DEV Community

Cover image for Three Interesting Apache Opensource Projects that you should take a look at!
Aravind Putrevu
Aravind Putrevu

Posted on

2

Three Interesting Apache Opensource Projects that you should take a look at!

Nowadays, I see many devs write about "Cloud-native" software projects like Kubernetes, Prometheus, Containerd, Envoy. But there are a few other similar OSS foundation backed projects which are excellent too.

This post is about three such projects from "Apache Software Foundation (ASF)" that one should take a look at.

ASF is a very old Opensource organization which has been the home for projects like Hadoop, Zookeeper, Kafka, Lucene. If you are interested in contributing to OSS, this might be the best place to start too.

1. Apache Airflow

Apache Airflow

Apache Airflow is an open-source workflow management platform started at Airbnb. It essentially helps with many operations in a software project. It can monitor cron jobs, manage data pipelines. It generates a Directed Acyclic Graph (DAG) visualization diagram. But, it is not a streaming or ETL solution.

Cloud Platforms like GCP has Cloud Composer, AWS has Managed Workflows - which are managed SaaS versions of Airflow.

Github Repository

2. Apache Beam

Apache Beam

Apache Beam is a combination of the Batch and Streaming model to design and develop data processing pipelines. There are Java, Python, Go SDKs available for Apache Beam. It also supports backends like Apache Flink, Apache Spark, and GCP Dataflow.

Github Repository

3. Apache Pulsar

Apache Pulsar

Apache Pulsar is popular is a cloud-native, distributed messaging and streaming platform created at Yahoo. You might have heard it while researching the famous OSS message queue Kafka. Born in the cloud-native world, Pulsar can be run on Docker or Kubernetes. Pulsar has built-in connectors to MongoDB, Elasticsearch, PostgreSQL, Redis.

Github Repository

I believe these projects are interesting to learn and try even on a side-project. I might have missed several other interesting projects or incubating one's. If you are using or have used something, write them in the comments.

--
Stay Safe
Aravind Putrevu

Heroku

Build apps, not infrastructure.

Dealing with servers, hardware, and infrastructure can take up your valuable time. Discover the benefits of Heroku, the PaaS of choice for developers since 2007.

Visit Site

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Immerse yourself in a wealth of knowledge with this piece, supported by the inclusive DEV Community—every developer, no matter where they are in their journey, is invited to contribute to our collective wisdom.

A simple “thank you” goes a long way—express your gratitude below in the comments!

Gathering insights enriches our journey on DEV and fortifies our community ties. Did you find this article valuable? Taking a moment to thank the author can have a significant impact.

Okay