DEV Community

Cover image for Cron jobs without Celery, Redis, or Beat: how Fitz puts a distributed scheduler inside the language
Martin Palopoli
Martin Palopoli

Posted on

Cron jobs without Celery, Redis, or Beat: how Fitz puts a distributed scheduler inside the language

To run a task every 5 minutes in Python you need Celery + Redis + Celery Beat + a worker process + a dedicated Dockerfile. In Fitz it's a decorator. With retry, timezone, persistence, and catch-up inside the language.

The same story every time

Your client is happy with the API. It's running. Then they ask for three things:

  1. "Can we clean up expired sessions every night?"
  2. "Can the welcome email go out in the background so signup doesn't wait?"
  3. "The daily report needs to run at 9 AM Buenos Aires time."

Three small asks. If you're in Python, the honest answer is: "OK, give me two days to add Celery."

The typical Python stack

pip install celery redis
Enter fullscreen mode Exit fullscreen mode

celery_app.py:

from celery import Celery
from celery.schedules import crontab

celery_app = Celery(
    "myapp",
    broker="redis://redis:6379/0",
    backend="redis://redis:6379/1",
)

celery_app.conf.beat_schedule = {
    "cleanup-sessions-nightly": {
        "task": "myapp.tasks.cleanup_old_sessions",
        "schedule": crontab(hour=3, minute=0),
    },
    "daily-report": {
        "task": "myapp.tasks.generate_daily_report",
        "schedule": crontab(hour=12, minute=0),  # 9am Buenos Aires = 12:00 UTC
        # heads up: if DST shifts, this fires at the wrong time
    },
}
celery_app.conf.timezone = "UTC"
Enter fullscreen mode Exit fullscreen mode

tasks.py:

from .celery_app import celery_app
from .db import get_session

@celery_app.task(autoretry_for=(Exception,), retry_backoff=True, max_retries=3)
def cleanup_old_sessions():
    with get_session() as s:
        s.execute("DELETE FROM sessions WHERE expires_at < now()")
        s.commit()

@celery_app.task(autoretry_for=(SMTPError,), retry_backoff=True, max_retries=5)
def send_welcome_email(email: str):
    smtp.send(email, "Welcome!")
Enter fullscreen mode Exit fullscreen mode

api.py:

@app.post("/signup")
def signup(creds: Credentials):
    user = create_user(creds)
    send_welcome_email.delay(user.email)  # fire-and-forget via broker
    return user
Enter fullscreen mode Exit fullscreen mode

docker-compose.yml:

services:
  api:
    build: .
    command: uvicorn api:app --host 0.0.0.0
  celery-worker:
    build: .
    command: celery -A celery_app worker --loglevel=info
    depends_on: [redis]
  celery-beat:
    build: .
    command: celery -A celery_app beat --loglevel=info
    depends_on: [redis]
  redis:
    image: redis:7-alpine
Enter fullscreen mode Exit fullscreen mode

Plus:

  • A supervisord.conf or systemd unit to keep things alive when they crash.
  • (Optional) flower for visibility — a 4th process.
  • Timezone conversions done by hand (Celery beat works in UTC, you have to compute the offset).
  • When a worker crashes between hitting the broker and finishing the task: does it retry? Does it hang? Depends on how you configured acks_late and task_reject_on_worker_lost.

Four processes in production. Three new libraries. A broker. New conventions. A day of setup.

The same thing in Fitz

@cron("0 3 * * *", tz="UTC")
async fn cleanup_old_sessions(db: DbConn) {
    db.exec("DELETE FROM sessions WHERE expires_at < now()").await
}

@cron("0 9 * * *", tz="America/Argentina/Buenos_Aires")
async fn daily_report(db: DbConn) {
    // ...
}

@background
async fn send_welcome_email(email: Str) {
    // expensive thing
}

@post("/signup")
fn signup(creds: Credentials) -> User {
    let user = create_user(creds)
    spawn(send_welcome_email(user.email))  // typed fire-and-forget
    return user
}
Enter fullscreen mode Exit fullscreen mode

docker-compose.yml:

services:
  api:
    build: .
Enter fullscreen mode Exit fullscreen mode

That's it. One binary. One process. No broker. No dedicated worker. No beat. The scheduler runs inside the Fitz process.

The raw table

Item Python (Celery + Redis + Beat) Fitz
Initial setup 4 files + 3 services + 1 broker 1 decorator
Schedule crontab(hour=3, minute=0) in config @cron("0 3 * * *")
Timezone UTC + manual offset tz="America/Argentina/Buenos_Aires"
Retry/backoff autoretry_for=(...) + retry_backoff=True retry={max=3, backoff="exponential", initial_secs=1, max_secs=30}
Run persistence Redis result backend + viewing via Flower store=db, table auto-created in Postgres
Catch-up of missed runs Not native catch_up=true
Background fire-and-forget .delay(arg) via broker spawn(fn_call) direct
Type-checked args None (everything JSON-serialized) Static —spawn requires @background and refines to Future<T>
Production processes api + worker + beat + redis api
Docker images 3 (api, worker, beat) + redis 1

Opt-in persistence with store=db

When you want run history for auditing:

@cron("0 3 * * *", store=db, retry={max=3, backoff="exponential"})
async fn cleanup_old_sessions(db: DbConn) {
    db.exec("DELETE FROM sessions WHERE expires_at < now()").await
}
Enter fullscreen mode Exit fullscreen mode

On boot, Fitz creates (if missing) fitz_cron_jobs and fitz_cron_runs. Each attempt is persisted with started_at/finished_at/status/attempt/error. You query with plain SQL:

SELECT job_name, status, attempt, error, started_at
FROM fitz_cron_runs
WHERE started_at > now() - interval '24 hours'
ORDER BY started_at DESC;
Enter fullscreen mode Exit fullscreen mode

No Flower install. No Sentry webhook. The DB you already have.

Catch-up

If the binary was down between 3 AM and 7 AM and the cron was at 3 AM:

  • Celery beat: the run is lost. The task won't fire again until the next midnight.
  • Fitz with catch_up=true: on boot, it sees there was a missed run between last_run_at and now, fires ONE immediate run (not N — avoids spam), then resumes the normal schedule.
@cron("0 3 * * *", store=db, catch_up=true)
async fn cleanup_old_sessions(db: DbConn) { ... }
Enter fullscreen mode Exit fullscreen mode

Configurable retry with backoff

Three backoff modes: exponential (1s, 2s, 4s, 8s...), linear (1s, 2s, 3s, 4s...), constant (1s, 1s, 1s...). With cap max_secs:

@cron("*/10 * * * *",
    retry={ max=5, backoff="exponential", initial_secs=1, max_secs=60 })
async fn sync_external_api(db: DbConn) {
    // 1s → 2s → 4s → 8s → 16s (capped at 60s if it would go higher)
}
Enter fullscreen mode Exit fullscreen mode

In Python with Celery:

@celery_app.task(
    autoretry_for=(ConnectionError, TimeoutError),
    retry_backoff=True,
    retry_backoff_max=60,
    retry_jitter=False,
    max_retries=5,
)
def sync_external_api():
    ...
Enter fullscreen mode Exit fullscreen mode

Equivalent. But notice Celery requires you to enumerate the exceptions that trigger retry, and backoff is a boolean instead of a kind enum. Fitz retries on any error the fn returns (consistent with the language's Result model), and the backoff mode is explicit.

Real timezone, not hardcoded offsets

DST changes are the bug that wakes you up at 4 AM. Celery beat works in UTC and you have to remember that between March and November your 9 AM Buenos Aires cron runs at 12 UTC, but the rest of the year it shifts. Your client is in another timezone. Your app sells in three more countries.

Fitz accepts IANA timezones directly:

@cron("0 9 * * *", tz="America/Argentina/Buenos_Aires") async fn buenos_aires() { ... }
@cron("0 9 * * *", tz="America/New_York")              async fn new_york()      { ... }
@cron("0 9 * * *", tz="Asia/Tokyo")                    async fn tokyo()         { ... }
Enter fullscreen mode Exit fullscreen mode

Each runs at 9 AM local time in its tz, with DST handled by the underlying lib (chrono-tz). No manual conversions.

Background jobs without a broker

# Python
@app.post("/signup")
def signup(creds: Credentials):
    user = create_user(creds)
    send_welcome_email.delay(user.email)  # serialize args → Redis → worker
    return user
Enter fullscreen mode Exit fullscreen mode
// Fitz
@background
async fn send_welcome_email(email: Str) {
    smtp.send(email, "Welcome!")
}

@post("/signup")
fn signup(creds: Credentials) -> User {
    let user = create_user(creds)
    spawn(send_welcome_email(user.email))  // native tokio::spawn, typed
    return user
}
Enter fullscreen mode Exit fullscreen mode

The compiler requires the fn be decorated with @background to authorize the spawn(...) — without it, the callsite throws a type error at build time. And it refines the return to Future<Null> with the concrete type of the target, not Any. If send_welcome_email returns Result<()>, the spawn(...) gives it to you typed.

When NOT to use @background and go back to Celery?

  • If you need jobs to survive process crashes → explicit persistence with store=db covers cron jobs; for @background with persistence, that's residual debt.
  • If you need to distribute jobs across N workers on N nodes → Celery with a shared broker is still the answer. @background runs in-process.

For 90% of services (nightly cleanup, transactional email, KPI recompute, cache sync) Fitz's model is enough and then some.

Cron-only mode for systemd

For services that ONLY have jobs (no HTTP), fitz build produces a binary that starts the scheduler and blocks on ctrl+c:

@cron("0 3 * * *") async fn cleanup() { ... }
@cron("*/15 * * * *") async fn sync() { ... }
Enter fullscreen mode Exit fullscreen mode

As a systemd unit:

[Unit]
Description=Scheduled jobs for myapp
After=network.target

[Service]
ExecStart=/usr/local/bin/myapp-jobs
Restart=always
User=myapp

[Install]
WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode

Zero brokers. The binary is 5 MB. systemctl restart myapp-jobs and you're done.

Bit-for-bit parity fitz runfitz build

This is what makes it shippable: the binary produced by fitz build runs the same scheduler as fitz run, with the same cron expression parser, the same retries, the same timezone. Same syntax, same semantics, no "oh, this only works in production if you configure Celery."

What Fitz does NOT give you (yet)

Being honest about where it doesn't reach:

  • Distributing jobs across N workers/nodes sharing a queue. @cron runs in-process. If you run two binaries with the same @cron, both fire — bug, not feature. For horizontal scaling of jobs, it's still Celery (or NATS JetStream, or Temporal). There's explicit debt in the roadmap about distributed locks.
  • A Flower-like UI. The data is in fitz_cron_jobs and fitz_cron_runs. If you want dashboards, external dashboards (Grafana, Metabase) cover it.
  • @background with persistence across restarts. For @cron it's in v0.11.2 (close of 9.w.3.iter2). For @background the spawn fires but doesn't survive crashes — deferred to iter3 if demand appears.
  • In-flight job cancellation. Today there's no API to "cancel all queued runs of X job". Process dies → in-flight runs die with it.

If you're in one of those cases, Fitz isn't the tool today. If not, the one-binary model covers the real case with fewer moving parts.

Closing

The argument for adding Celery to a Python API is typically: "but later it scales better." In 90% of services I've shipped over the past decade, that "later" never came — the app spent its entire lifetime with fewer than 100 jobs per hour and never needed a worker cluster.

For that 90%, Fitz replaces 4 processes + 3 libraries + 1 broker with one decorator. When you reach the other 10% where you need real horizontal scaling, Celery is still there — from python import celery is also available, you can do it yourself.

But starting the project with the simplest possible model and raising complexity only when you need it is the feedback loop Fitz wants to give you.


Next post in the series: "Auth with JWT, RBAC, and token blacklist without gluing 5 libraries: Fitz vs FastAPI + python-jose + passlib + Redis blacklist" — the full auth flow, side by side.

Repo: github.com/Thegreekman76/fitz
Chapter 30 of the guide (Jobs without Celery): docs/guide.md

Top comments (0)