DEV Community

The Weekly Squeak

Pepperdata - Bringing DevOps Practices to the Big Data World

Big Data analysis generally involves piping large data sets into processing systems such as Apache Spark and Hadoop, walking away for a very long coffee and returning later to a hopefully successful result. The DevOps and continuous build ecosystem has long employed linters, code analyzers and application profilers to discern potential trouble and optimization points in code, and Pepperdata want to bring that practice to the Big Data world. I spoke with CEO Ash Munshi, formerly CTO of Yahoo and consultant to other data heavy tech companies about what the company offers, and how their customers save hours from running inefficient processing jobs.

Episode source