DEV Community

IT-Craftsman
IT-Craftsman

Posted on Originally published at itcraftsman.hashnode.dev

Airflow 3 quietly changed what your cron schedule means

#airflow #dataengineering #python #etl

If you upgrade a DAG like this from Airflow 2 to Airflow 3 without changing a line, it can start processing a different day than before:

@dag(schedule="0 0 * * *", start_date=pendulum.datetime(2026, 10, 1, tz="UTC"))
def daily_orders():
    @task
    def extract(ds=None):
        load_orders_for(ds)   # which day is ds?
    extract()
Enter fullscreen mode Exit fullscreen mode

Nothing is broken. The default timetable changed. Here is what is going on and how to make your DAGs independent of it.

Airflow 2: a run covers an interval and starts after it ends

In Airflow 2, [scheduler] create_cron_data_intervals defaults to True, so a bare cron string becomes a CronDataIntervalTimetable. Each run covers a data interval and is started after the interval ends:

Run logical_date data interval actually starts
"Oct 1 run" 2026-10-01 00:00 [Oct 1 00:00, Oct 2 00:00) shortly after Oct 2 00:00

logical_date (and therefore ds) equals data_interval_start: the day being processed. This confused a generation of engineers ("why does the Oct 1 run start on Oct 2?"), but it made "process yesterday" DAGs work naturally.

Airflow 3: a run is triggered at the cron time

In Airflow 3 the default is create_cron_data_intervals = False, so a bare cron string becomes a CronTriggerTimetable. The run starts at the cron time, logical_date is the trigger time, and data_interval_start and data_interval_end are normally the same moment unless you configure an interval.

Run logical_date data interval actually starts
run at Oct 2 00:00 2026-10-02 00:00 Oct 2 00:00, Oct 2 00:00 Oct 2 00:00

So the run that starts at midnight on October 2nd has ds = 2026-10-02 in Airflow 3, but processed 2026-10-01 in Airflow 2. If your task does WHERE order_date = '{{ ds }}', it now loads the current (incomplete) day.

The official upgrade guide calls this out: if you rely on data_interval_start/data_interval_end or on ds/ts, either set create_cron_data_intervals=True before upgrading or pin an explicit timetable. (Another change while you are there: catchup now defaults to False.)

The fix: stop depending on the default

Pin the timetable in the DAG and give it an explicit interval. Then the DAG behaves the same regardless of the global config:

from datetime import timedelta
import pendulum
from airflow.sdk import dag, task
from airflow.timetables.trigger import CronTriggerTimetable

@dag(
    schedule=CronTriggerTimetable("0 0 * * *", timezone="UTC", interval=timedelta(days=1)),
    start_date=pendulum.datetime(2026, 10, 1, tz="UTC"),
    catchup=False,
)
def daily_orders():
    @task
    def extract(data_interval_start=None, data_interval_end=None):
        # run at 2026-10-02 00:00 gets [2026-10-01 00:00, 2026-10-02 00:00)
        load_orders(window_start=data_interval_start, window_end=data_interval_end)
    extract()

daily_orders()
Enter fullscreen mode Exit fullscreen mode

Note: even with interval=, logical_date (and ds) is the trigger time (Oct 2), so template on data_interval_start/data_interval_end, not on ds.

And make the task read the window it is given, never "now":

# bad: depends on when the task happens to run (retries, backfills, catchup all break)
load_orders(window_start=date.today() - timedelta(days=1))

# good: depends only on the run
load_orders(window_start=data_interval_start, window_end=data_interval_end)
Enter fullscreen mode Exit fullscreen mode

A checklist before you upgrade

  1. grep your DAGs for ds, ds_nodash, ts, logical_date, execution_date (removed in Airflow 3), data_interval_start.
  2. For each DAG with a bare cron string, decide: interval semantics or trigger semantics? Pin it explicitly.
  3. Check manual triggers: Airflow 3's upgrade guide warns not to assume a manually triggered run's data_interval equals the logical_date you typed in.
  4. Make tasks idempotent (delete-and-replace the run's window, or MERGE by key), so a run that processed the wrong day can simply be rerun.
  5. Add a test that imports every DAG and asserts the timetable class you expect.

This is a condensed version of one question from the Data Engineering Interview Kit 2026. Interviewers love it because it tests whether you understand why idempotent, interval-based tasks matter. The free sample with 20 full questions (answers, trade-offs, common mistakes, follow-ups) is here: https://payhip.com/b/18A5n

About me: I work as an independent data engineer and DevOps freelancer (IT-Craftsman) and have delivered several data engineering and DevOps projects for large companies. I also work for a technology consulting firm, where I conduct hiring interviews for data engineers and DevOps engineers.

Facts checked against the Airflow 3 documentation ("Upgrading to Airflow 3", "Timetables").

Top comments (0)