DEV Community

Cover image for Three Interesting Apache Opensource Projects that you should take a look at!
Aravind Putrevu
Aravind Putrevu

Posted on

Three Interesting Apache Opensource Projects that you should take a look at!

Nowadays, I see many devs write about "Cloud-native" software projects like Kubernetes, Prometheus, Containerd, Envoy. But there are a few other similar OSS foundation backed projects which are excellent too.

This post is about three such projects from "Apache Software Foundation (ASF)" that one should take a look at.

ASF is a very old Opensource organization which has been the home for projects like Hadoop, Zookeeper, Kafka, Lucene. If you are interested in contributing to OSS, this might be the best place to start too.

1. Apache Airflow

Apache Airflow

Apache Airflow is an open-source workflow management platform started at Airbnb. It essentially helps with many operations in a software project. It can monitor cron jobs, manage data pipelines. It generates a Directed Acyclic Graph (DAG) visualization diagram. But, it is not a streaming or ETL solution.

Cloud Platforms like GCP has Cloud Composer, AWS has Managed Workflows - which are managed SaaS versions of Airflow.

Github Repository

2. Apache Beam

Apache Beam

Apache Beam is a combination of the Batch and Streaming model to design and develop data processing pipelines. There are Java, Python, Go SDKs available for Apache Beam. It also supports backends like Apache Flink, Apache Spark, and GCP Dataflow.

Github Repository

3. Apache Pulsar

Apache Pulsar

Apache Pulsar is popular is a cloud-native, distributed messaging and streaming platform created at Yahoo. You might have heard it while researching the famous OSS message queue Kafka. Born in the cloud-native world, Pulsar can be run on Docker or Kubernetes. Pulsar has built-in connectors to MongoDB, Elasticsearch, PostgreSQL, Redis.

Github Repository

I believe these projects are interesting to learn and try even on a side-project. I might have missed several other interesting projects or incubating one's. If you are using or have used something, write them in the comments.

Stay Safe
Aravind Putrevu

Top comments (0)