Python has not lacked for libraries such as Hadoopy or Pydoop to work with Hadoop, but those libraries are designed more with Hadoop users in mind than data scientists proper.

Demand for big data skills are on the rise and aren't limited to just NoSQL and Hadoop but also include Python and general cloud skills.

Python and R data science running on containers and Kubernetes The fourth megatrend is containers and Kubernetes. Hadoop/Spark is not just a storage environment but also a compute environment.

As the Hadoop growth curves illustrate, the technological foundation for a data-oriented open-source ecosystem has been laid, and a family of related technology is starting to emerge.

Databricks believes that big data is a huge opportunity that is still largely untapped and wants to make it easier to deploy and use.

Open-source big-data platform Hadoop excels at batch-mode processing at scale but it was never designed for real-time analytics. ScaleOut Software has middleware that it thinks addresses the issue.

Hadoop is a must-know for all big data professionals. With courses, classes and bootcamps abound you’ll be a data master in no time.