Why I Built DagSmith: Bringing a Visual Drag-and-Drop Editor straight into Apache Airflow 3

#python #airflow #dataengineering #opensource

Let’s be honest: code is great, but sometimes writing data pipelines by hand feels like unnecessary friction.

Every data engineer knows the drill. You need to connect a few tasks, so you start importing operators, setting up task relationships using >> and <<, and jumping back and forth between your IDE and the Airflow UI just to check if the graph looks right.

But it’s not just about data engineers. In almost every company, there is a growing group of business users, analysts, or analytics engineers who know exactly what data they need to move from A to B, but they don't necessarily want (or know how) to write complex Python scripts just to orchestrate a simple flow.

This gap—and the desire to make DAG creation faster for everyone—is exactly why I started building DagSmith.

The Problem: Coding vs. Visualizing

Airflow is incredibly powerful because it treats workflows as code. But that power comes with a cost: accessibility.

For Business and Less-Technical Users: They need a way to build or modify simple ingestion and transformation steps without drowning in boilerplate code, environments, or Git workflows. They just want a visual bridge.
For Data Engineers: Even if you are a Python wizard, there are days when you just want to sketch out a high-level architecture visually. You want to see the layout, drag a couple of operators onto a screen, and let the tool worry about formatting the code, so you can focus on the actual data logic.

I wanted to combine the best of both worlds: the visual clarity of a drag-and-drop tool with the native power of Apache Airflow.

From a Standalone React App to an Integrated Plugin

The project didn’t start this way. Initially, I built a completely standalone web application using React. The idea was to create an isolated canvas where you could draw your pipeline and then export the code.

But I quickly realized that a standalone tool creates its own friction. Moving files around and keeping things in sync felt clunky. If an engineering team already has an Airflow instance up and running, the last thing they want is another separate platform to maintain. So, I put the project on hold.

Then Airflow 3 came into the picture. With the massive updates and the fresh energy around the ecosystem, it clicked: Don't build a new platform. Build it directly into the native Airflow UI.

I pivoted the entire architecture, moved away from the standalone approach, and turned it into a native plugin.

What exactly is DagSmith?

DagSmith is an open-source, after-hours project designed to bridge the gap between visual design and production code:

Visual Drag-and-Drop Canvas: An interactive interface where anyone can place operators from a dedicated toolbox and visually link them together.
Properties Inspector: A simple sidebar to configure task parameters on the fly without writing custom dictionaries or boilerplate logic.
Integrated Code Editor: For the engineers who still want full control, it features a built-in code editor. You can manipulate the Python script directly and see the layout adapt, keeping the visual graph and the code perfectly in sync.

Give it a spin and share your thoughts!

Whether you want to empower your non-technical team members to build their own simple flows, or you're a seasoned Data Engineer who is just tired of typing out layout code, DagSmith is built for you.

This is still an early release (a true passion project built after hours), but it's ready for testing. I’ve put a short demo video on the homepage so you can see it in action before installing.