Same guarantees, without programming them: Hydra ETL vs Airflow + dlt + dbt vs Airflow + Airbyte + dbt

Most comparisons of data tools show the happy path. I wanted to see the other one: what happens when the source file is missing, when one value is unreadable, or when the second output fails after the first one was written.

So I built the same small pipeline three times, injected the same four failures into each, and measured what it took to keep the outputs correct. Everything is public, so you can check it or run it yourself: github.com/bejaouibechir/hydra-etl-benchmark.

The pipeline

• Input: 200,000 orders and 2,000 customers (CSV).
• Work: keep paid orders, join customers, compute revenue by country and revenue by month.
• Output: two CSV reports.
• Backup: yesterday's export, to use if today's data is unusable.

Before every run, the outputs hold the day-before-yesterday's reports. After the run, each output must come from today's data or the backup, and both outputs must come from the same one.

The three stacks:

1. Hydra ETL (declarative YAML pipelines, validated before they run). Disclosure: I'm the author.
2. Airflow + dlt + dbt (DuckDB).
3. Airflow + Airbyte + dbt (PostgreSQL).

Each stack exists in two variants: without protection (what you usually write first) and with protections (fallback to the backup, outputs published all-or-nothing, timeouts).

The four scenarios:

• Normal: nothing breaks.
• Missing source: today's orders.csv is not there.
• Unreadable amount: one amount is abc, so the first report fails.
• Impossible date: one date is 2026-13-45, so only the second report fails.

Without protection, every stack can leave inconsistent outputs. With protections, all three get it right.

"Inconsistent" means the country report is today's and the month report is the day-before-yesterday's. Nothing failed loudly enough to notice; the dashboard is simply wrong.

Without protection, the Airbyte stack stayed consistent, but not thanks to Airbyte: in that DAG, the CSV export is a separate step that only runs after dbt fully succeeds. It's a design choice in the DAG, not a property of the tool.

With protections, the guarantees are the same everywhere. The difference is how you get them.

How each stack gets there

Airflow + dlt + dbt: about 26 lines of Python. A backup task with trigger_rule="one_failed", staging folders, and a publish task that copies both reports next to the target and renames them, written by hand:

backup = build("build_backup", "data/backup/orders.csv", f"{HOME}/staging/backup")
backup.trigger_rule = "one_failed"

@task(trigger_rule="one_success")
def publish():
    """Publish both reports of the same run, or nothing."""
    for stage in ("main", "backup"):
        d = HOME / "staging" / stage
        if all((d / f).exists() for f in OUTPUTS):
            for f in OUTPUTS:
                shutil.copy(d / f, HOME / "output" / (f + ".tmp"))
            for f in OUTPUTS:
                os.replace(HOME / "output" / (f + ".tmp"), HOME / "output" / f)
            return stage
    raise RuntimeError("no complete set of reports to publish")

My first version of this DAG had a logic bug: the backup ran even after a success. That's the risk with guarantees you program yourself. And this atomic publish only works for files; for a database you'd still have to write a table swap.

Hydra ETL: about 21 lines of YAML. The workflow undoes what it did if a step fails, and the execution order declares a backup workflow and when to use it:

# workflows/daily_revenue.yaml
workflow:
  name: daily_revenue
  rollback: true            # if a step fails, undo the steps already done
  steps:
    - name: by_country
      type: job
      job: ../jobs/revenue_by_country
    - name: by_month
      type: job
      job: ../jobs/revenue_by_month
      depends_on: [by_country]

# workflows/thread.yaml
thread:
  - mode: sequential
    workflows:
      - name: daily_revenue
        timeout: 10m
        fallback:
          use: [daily_revenue_backup]
          when: "not file_exists('data/orders.csv')"   # checked before the start
          on_failure: true                             # also after a failure that left nothing published

Here is the "impossible date" run (Hydra ETL 0.19.1, long error lines trimmed):

  ▶  Workflow  daily_revenue
     ✓  Step 'by_country' OK (1.9s)
     ✗  Step 'by_month' FAILED: ValueError: Step 2/5: cast: cannot convert column='order_date' to type='datetime': time data "2026-13-45" …
     ↩  Step 'by_country' undone
     │ output/revenue_by_country.csv: previous file restored
  ↯  Workflow 'daily_revenue' failed (2.8s), nothing published — replaced by daily_revenue_backup
  ▶  Workflow  daily_revenue_backup  (fallback, in place of daily_revenue)
     ✓  Step 'use_backup_export' OK (0.0s)
     ✓  Step 'by_country' OK (1.1s)
     ✓  Step 'by_month' OK (1.5s)
  ✓  Workflow 'daily_revenue_backup' OK (2.6s)

Two differences in behaviour, beyond the syntax:

• Hydra ETL decides before starting. If today's file is missing, the backup runs directly. Airflow has to fail first, which leaves a failed task and an alert.
• Hydra ETL only falls back if nothing was published. If the failed run had already written something it couldn't undo, it stops and says why, instead of mixing two versions.

Is it less code? No.

The volume is comparable. The difference is that the guarantees are declared and enforced by the engine, instead of programmed in a DAG, and that it's one tool and one language instead of three. With Airbyte, add the configuration of sources, destination and connections: 58 lines of API calls here, or clicks in the UI.

How long a run takes

On one EC2 machine (Ubuntu 24.04, 4 vCPUs, 15 GB), Hydra ETL takes 1 to 2 seconds, Airflow + dlt + dbt 14 to 23 seconds, and Airflow + Airbyte + dbt 1 to 3 minutes; each Airbyte sync starts containers, which takes 60 to 80 seconds on its own.

Read these with care: the Airflow runs use airflow dags test, which pays Airflow's start-up on every run. In production the scheduler runs continuously.

What it costs to install and run

Hydra ETL is one pip install with nothing to operate. Airflow needs its scheduler in production. The Airbyte stack adds Docker, a local Kubernetes cluster (kind), Airbyte itself and PostgreSQL: 18 containers using about 3.7 GB of memory while idle.

A few things I ran into with Airbyte, which say more about my setup than about Airbyte: its file source in web mode requires https:// (I switched to SFTP); dbt views on the raw tables blocked Airbyte's full-refresh syncs until I made the intermediate models ephemeral; and the API error was a generic "Something went wrong in the connector" until I wrote a diagnostic script.

What about bigger volumes?

On the same pipeline without Airflow, Hydra ETL was the fastest up to about 2 million rows per run (up to 6 times faster than dlt + dbt at 200,000 rows, where dlt + dbt pay about 9 seconds of fixed start-up). Above that, a columnar engine like DuckDB took the lead. Since version 0.15, Hydra ETL can hand its aggregations, sorts and joins to DuckDB itself above that threshold.

Most pipelines are on the left side of that line: analyses of production warehouse workloads published in recent years (Amazon's Redset dataset, VLDB 2024) show that most queries scan modest amounts of data. You probably don't have a big-data problem; you probably have a correctness problem.

Limits of this test

One machine. 200,000 rows. One run per scenario for Airbyte (median of three for the others). airflow dags test instead of a running scheduler. Airbyte in --low-resource-mode. The measurements were taken with Hydra ETL 0.14.0 on October 28; I re-ran the four failures with 0.19.1 before publishing, with the same results.

Check it yourself

The three implementations, the scripts that run and measure them, the raw results, the logs and the script that draws these charts are here: github.com/bejaouibechir/hydra-etl-benchmark. python3 run.py replays the whole test on a fresh Ubuntu machine; the README shows how to replay only the four failures in a few minutes.

Takeaway

Every stack in this test can be made safe. The question is what that costs you: in Airflow you write the safety net yourself, in Python, and it's easy to get subtly wrong. In Hydra ETL it's a few declared lines that the engine checks before anything runs, with one pip install and nothing else to operate.

• Hydra ETL: github.com/bejaouibechir/Hydra · pip install hydra-etl
• Every release: hydraetl.com/changelog

It's in beta, with one maintainer. If you see a flaw in the method, open an issue on the benchmark repository: I'd rather fix the test than defend it.