Windmill for Data Engineering: TypeScript/Python Scripts, Flows & Self-Hosted OSS

windmill is the pick-one architectural decision that finally collapses "we need an orchestrator and an internal-tool builder and a secret manager and a script runner" into a single Rust-native, self-hosted, AGPLv3-licensed platform — and it is the one platform senior data engineers evaluate in 2026 when they refuse to string together Airflow plus Retool plus HashiCorp Vault plus a custom cron server. Every internal workflow your organisation runs — a nightly Postgres → S3 dump written in Python, a customer-support triage form fronting a Postgres write, a Slack-triggered backfill flow, a data-quality retry job — needs a runtime, a scheduler, a UI surface, an audit trail, and a way to inject secrets without leaking them into logs and without pulling in six separate SaaS products. The engineering trade-off does not live in "should we adopt a workflow platform" — every data team past a certain scale needs one — but in which windmill orchestrator (or Airflow, or Dagster, or n8n) you standardise on and what it costs the polyglot team that has to live inside it.

This guide is the senior-DE walkthrough you wished existed the first time an interviewer asked "walk me through Windmill's script + flow + module primitives and why polyglot matters", or "your data team writes Python but the tooling team writes TypeScript — how does one platform serve both?", or "explain the OpenFlow spec and why the AGPL license changes the vendor calculus." It walks through the Windmill architecture — the OpenFlow spec, scripts and flows and modules, the Postgres-backed queue with a Rust-native worker fleet, Nsjail sandboxing, the low-code app builder atop scripts, typed resources and encrypted variables — and the head-to-head against windmill vs airflow, Dagster, n8n, and Retool with the interview signals senior architects listen for. Each section pairs a teaching block with a Solution-Tail interview answer — code, a step-by-step trace, an output table, then a concept-by-concept breakdown of why it works.

When you want hands-on reps immediately after reading, drill the SQL practice library →, rehearse system design against the design practice library →, and sharpen the streaming axis with the streaming practice library →.

---

On this page

• Why Windmill matters in 2026
• Scripts, flows & modules
• Workers, queue & executor architecture
• Apps + variables + resources
• Windmill vs Airflow / Dagster / n8n / Retool + interview signals
• Cheat sheet — Windmill recipes
• Frequently asked questions
• Practice on PipeCode

---

1. Why Windmill matters in 2026

One Rust-native platform unifies scripts + flows + apps + secrets — the choice binds your polyglot team for years

The one-sentence invariant: windmill is a Rust-native, self-hosted, AGPLv3-licensed platform that unifies polyglot scripts (TypeScript, Python, Bash, Go, SQL), durable multi-step flows (DAGs with branches, loops, and suspends), a low-code app builder that binds UI widgets to script outputs, encrypted variables, typed resources, workspace RBAC, and a Postgres-backed queue into one deployable binary — and each capability trades against the buy-vs-build calculus for orchestrators (Airflow), asset platforms (Dagster), integration hubs (n8n), and internal-tool builders (Retool) in a way that cannot be undone once your team has 200 scripts in the platform. The tool you pick in month one becomes the tool your entire team's tacit knowledge encodes; every downstream consumer, dashboard, and Slackbot hard-codes assumptions about which secret store to read, which webhook endpoint to hit, which UI is authoritative.

The four axes interviewers actually probe.

• Polyglot runtime. Windmill runs TypeScript (Deno + Bun), Python (per-script venvs), Bash, Go, SQL, and PowerShell — each with hermetic dependency isolation. Airflow is Python-only (bash callable via BashOperator, but ergonomics are second-class). Dagster is Python-only. n8n is JavaScript-only. Retool is a JS-only UI layer. If your team has both Python data engineers and TypeScript backend engineers writing internal tooling, polyglot is not a "nice to have" — it's the single largest ergonomic differentiator.
• Self-hostability + license. Windmill is AGPLv3 with a commercial license available; the core is genuinely open source, one Postgres + one binary, no vendor callouts. Airflow (Apache 2.0), Dagster (Apache 2.0), and n8n (Sustainable Use License) are all self-hostable; Retool is proprietary (closed-source, per-seat SaaS with a self-hosted enterprise tier). Interviewers open here because AGPL versus MIT versus proprietary shapes the vendor-independence story for the next five years.
• Orchestration vs internal-tool split. Most tools pick a side: Airflow / Dagster / Prefect are orchestrators; Retool / n8n / Appsmith are internal-tool builders; Vault / Doppler are secret managers. Windmill fuses all three surfaces — a script that runs on a schedule is a workflow step, that same script bound to a form widget is an internal tool, secrets are first-class variables. The unification is Windmill's headline pitch and the axis interviewers probe for architectural fluency.
• Worker isolation model. Windmill workers can run scripts natively (fast, trusts the script) or under Nsjail (Linux namespace sandbox — filesystem-isolated, seccomp-filtered, cannot see other scripts' secrets). Airflow relies on operator-level isolation (KubernetesPodOperator ships each task in its own pod). Dagster uses op-level executors. n8n runs everything in one Node process. Retool runs on the client. Interviewers probe worker isolation because it's the security boundary that separates "we let interns write scripts" from "we let one intern's script exfiltrate the prod DATABASE_URL."

The 2026 reality — Windmill is one of five viable options.

• Windmill. Rust-native binary + Postgres. AGPLv3. Polyglot, apps + orchestration + secrets in one product. The default when you want one self-hosted platform that serves both data pipelines and internal tooling with the same team, secrets, and RBAC surface. Managed offering also exists.
• Apache Airflow. Python-only orchestrator with the largest community. AGPLv3-free, Apache 2.0. Wins for pure Python DAG orchestration at any scale; less compelling if you want internal tools, TS scripts, or a low-friction UI for non-engineers.
• Dagster. Python-only, asset-lineage-first orchestrator. Apache 2.0. Wins for data-platform teams building software-defined assets with strong lineage + partitioning contracts; less compelling for arbitrary polyglot scripts or internal-tool UIs.
• n8n. Node-based no-code integration platform. Sustainable Use License (not OSI-approved but self-hostable). Wins for business-user-authored SaaS-to-SaaS integrations; less compelling for engineer-authored scripts, data workloads, or complex flow logic.
• Retool. Proprietary, closed-source, per-seat SaaS internal-tool builder. Wins for polished CRUD UI on top of arbitrary APIs; loses hard on self-hostability, AGPL freedom, backend flow orchestration, and cost-at-scale.

What interviewers listen for.

• Do you name the OpenFlow spec as the open standard Windmill emits for portability, not just "the flow YAML"? — senior signal.
• Do you distinguish "orchestrator" from "internal-tool builder" and note that Windmill fuses both surfaces on one runtime? — senior signal.
• Do you name the Rust worker + Postgres queue as the executor architecture (not Kubernetes-native, not Kafka-backed)? — required answer.
• Do you name Nsjail or namespace-based sandboxing as the per-script isolation primitive? — senior signal.
• Do you describe Windmill as polyglot (TS, Py, Bash, Go, SQL) rather than "Python-based"? — required answer.
• Do you name AGPLv3 and its implication ("your fork or hosted derivative must be open-sourced") as the license constraint? — senior signal.

Worked example — the five-tool comparison table

Detailed explanation. The single most useful artifact for a Windmill interview is a memorised 5x5 comparison table. Every senior orchestration + internal-tool discussion converges on this table within the first ten minutes; having it in your head is what separates a fluent answer from a stumbling one. Walk through building the table for a hypothetical mid-size data team that runs Postgres + Snowflake, has three data engineers on Python and two backend engineers on TypeScript, and needs both nightly ETL and a Slack-triggered backfill UI for support.

• Workload. Nightly Snowflake ETL (Python), an internal support UI (TS + form widgets), a manual GDPR-request runner, and a set of ad-hoc scripts anyone on the team can trigger.
• Team. 3 Python DEs, 2 TS backend, 1 PM who needs to run backfills without writing code.
• Constraints. Self-hosted only (no SaaS due to compliance); one Postgres; RBAC per team; audit trail for every run.
• Non-goals. Kubernetes-first, per-task container orchestration (Airflow's home turf); asset-lineage graphs (Dagster's home turf).

Question. Build the five-tool comparison and pick the platform each requirement drives you toward.

Input.

Tool	Polyglot?	Self-host OSS?	Apps + UI?	Orchestration scale	License
Windmill	yes (TS/Py/Bash/Go/SQL)	yes (AGPLv3)	yes (low-code app builder)	good, single-postgres bounded	AGPLv3 + commercial
Airflow	Python + Bash	yes (Apache 2.0)	no (UI is DAG viewer only)	best-in-class at scale	Apache 2.0
Dagster	Python only	yes (Apache 2.0)	limited (asset lens UI)	very good, asset-native	Apache 2.0
n8n	JavaScript nodes	yes (Sustainable Use)	limited (form triggers only)	ok, node-heavy	SUL (not OSI)
Retool	JS in UI, any-backend via API	no (proprietary, self-host enterprise tier)	yes (best-in-class polished UI)	poor (UI-first)	proprietary

Code.

# Windmill deployment — one docker-compose serves the entire stack
version: "3.8"
services:
  windmill_server:
    image: ghcr.io/windmill-labs/windmill:latest
    depends_on: [db]
    environment:
      DATABASE_URL: postgres://windmill:changeme@db:5432/windmill
      MODE: server
      BASE_URL: https://windmill.internal.example
    ports: ["8000:8000"]

  windmill_worker:
    image: ghcr.io/windmill-labs/windmill:latest
    depends_on: [db]
    environment:
      DATABASE_URL: postgres://windmill:changeme@db:5432/windmill
      MODE: worker
      WORKER_GROUP: default
    deploy:
      replicas: 3       # horizontally scale worker fleet

  db:
    image: postgres:16
    environment:
      POSTGRES_USER: windmill
      POSTGRES_PASSWORD: changeme
      POSTGRES_DB: windmill
    volumes: [pgdata:/var/lib/postgresql/data]

volumes:
  pgdata:

Step-by-step explanation.

1. The Windmill deployment is one binary plus one Postgres. Compared to Airflow's scheduler + webserver + workers + Redis + Postgres (five processes minimum), Windmill's operational surface is a fraction of the size. This matters at team-of-3 scale where nobody wants to babysit five services.
2. The comparison table drives the decision: polyglot rules out Dagster (Python-only) and Airflow-as-primary (Python + Bash but weak TS); self-host OSS rules out Retool; apps rule out Airflow and Dagster as internal-tool builders; single-postgres deployment rules against Airflow's operational overhead.
3. Windmill wins on four of five axes for the described team. The one axis where it loses is orchestration scale: at 100k+ tasks/day or Kubernetes-first isolation, Airflow's operator + KEDA + Kubernetes executor pattern is more mature. Below that scale Windmill's Postgres-queue is fine.
4. Retool would still win if the requirement were "polished pixel-perfect CRUD UI for external customers." Windmill's app builder is functional but positioned for internal ops, not customer-facing SaaS.
5. n8n would win if the primary users were business users authoring SaaS integrations by dragging boxes. For engineer-authored polyglot scripts, its ergonomics are wrong.

Output.

Requirement	Best fit	Why
Nightly Snowflake ETL (Python)	Windmill	polyglot + secret management + schedule
Internal support UI (TS + form)	Windmill	app builder binds form to TS script
Manual GDPR backfill runner	Windmill	one-off scripts with typed inputs
Ad-hoc runner-by-anyone	Windmill	workspace RBAC + audit trail
Pure Python 100k tasks/day	Airflow	proven scale, KEDA scaling
Polished customer-facing CRUD	Retool	UI polish + templates

Rule of thumb. Never pick a workflow tool based on "which is trendiest." Pick it based on (polyglot × self-host × apps × scale × license) — the five axes. Write the table on a whiteboard first; the tool falls out of the constraints.

Worked example — what interviewers actually probe

Detailed explanation. The senior data-engineering Windmill interview has a predictable structure: the interviewer opens with an ambiguous question ("how would you replace our current Airflow + Retool + Vault stack?"), then progressively narrows to test whether you know the axes. The candidates who name the platform in sentence one and cite the OpenFlow spec score highest; the candidates who describe "some low-code tool" score lowest. Walk through the interview grading rubric.

• Ambiguous opener. "We have Airflow for ETL, Retool for internal tools, and Vault for secrets — how would you consolidate?" — invites you to name Windmill (or defend the split).
• Follow-up 1. "What's Windmill's runtime story for TypeScript vs Python?" — probes polyglot axis.
• Follow-up 2. "Where does the task queue live and what's the worker isolation model?" — probes architecture.
• Follow-up 3. "Is Windmill really open source?" — probes AGPL literacy.
• Follow-up 4. "How does Windmill compare to Dagster's asset-lineage story?" — probes orchestration nuance.

Question. Draft a 5-minute senior Windmill answer that covers all four axes without waiting to be asked.

Input.

Interview signal	Weak answer	Senior answer
Platform named	"some workflow tool"	"Windmill — Rust-native OSS platform, one binary + Postgres"
Polyglot	"it runs Python and JS"	"TS via Deno or Bun, Py in per-script venvs, plus Bash / Go / SQL"
Executor	"it has workers"	"Rust workers pulling from a Postgres queue, tag-routed, Nsjail-sandboxed"
License	"open source"	"AGPLv3 core + commercial license; forks and hosted derivatives must open-source"
Comparison	"better than Airflow"	"orthogonal to Airflow — Windmill fuses orchestration + apps + secrets on one runtime; Airflow is pure Python orchestration at scale"

Code.

Senior Windmill answer template (5 minutes)
===========================================

Minute 1 — name the platform up front
  "I'd consolidate Airflow + Retool + Vault into Windmill: one Rust
   binary plus one Postgres, self-hosted, AGPLv3, that runs polyglot
   scripts (TS/Py/Bash/Go/SQL), orchestrates them as flow DAGs,
   surfaces them as internal-tool UIs, and stores secrets as
   first-class variables."

Minute 2 — polyglot runtime
  "TypeScript runs on Deno or Bun in a hermetic per-script env.
   Python runs in a per-script venv with pip caching.
   Bash / Go / SQL are all first-class. This is the single largest
   ergonomic differentiator over Airflow (Python-only), Dagster
   (Python-only) and Retool (JS-only)."

Minute 3 — executor architecture
  "The queue is a Postgres table (LISTEN/NOTIFY plus row-level
   locking) — no Kafka, no Redis, no Kubernetes required. Workers
   are stateless Rust binaries that pull jobs by tag; you scale the
   worker fleet horizontally. Per-script isolation is native (fast)
   or Nsjail (Linux namespace + seccomp sandbox)."

Minute 4 — license + governance
  "AGPLv3 core with a commercial license for teams that need to
   embed it in proprietary offerings. If you fork it or host it as
   a service you must open-source your changes. Internal use behind
   the corporate firewall does not trigger AGPL propagation."

Minute 5 — orthogonal comparisons
  "Airflow: better at 100k+ tasks/day Python-only orchestration.
   Dagster: better for asset-lineage-first data platforms.
   n8n: better for no-code SaaS integrations by non-engineers.
   Retool: better for polished customer-facing CRUD.
   Windmill: best when polyglot + self-host + apps + orchestration
   all live in one team's operational surface."

Step-by-step explanation.

1. Minute 1 is the crucial framing. Naming the platform immediately — "Windmill — Rust binary plus Postgres, AGPLv3" — signals you're a decision-maker, not a tool-tourist. Weak candidates describe capabilities before naming the tool.
2. Minute 2 addresses polyglot before the interviewer asks. This preempts the common trap where you commit to Windmill without acknowledging the Python-only alternatives. Naming Deno / Bun / venvs shows depth beyond marketing pages.
3. Minute 3 is the architecture probe. Every mature workflow tool has a queue and workers; naming Postgres as the queue (not Redis, not RabbitMQ) is the Windmill-specific signal. Nsjail is the isolation-model signal.
4. Minute 4 is the license axis. AGPL is a live topic in 2026 (post the Sentry / Elastic / Redis license churn), and interviewers listen for whether you understand copy-left propagation and the SaaS carve-out.
5. Minute 5 is the orthogonal comparison. The senior answer never says "Windmill is better than Airflow" — it says "Windmill fuses surfaces Airflow doesn't; Airflow wins at pure Python scale." Framing tools as orthogonal, not competing, is the senior signal.

Output.

Grading criterion	Weak score	Senior score
Names Windmill in minute 1	rare	mandatory
Names OpenFlow spec	rare	senior signal
Names Rust worker + Postgres queue	occasional	mandatory
Names AGPLv3 implications	rare	senior signal
Frames tools as orthogonal	rare	senior signal

Rule of thumb. The senior Windmill answer is a 5-minute monologue that covers polyglot, architecture, license, and orthogonal comparison without waiting for the follow-ups. Rehearse it once; deploy it every time.

Worked example — the "pick the platform" decision tree

Detailed explanation. Given a new consolidation project, the senior architect runs a 5-question decision tree in their head. Codifying the tree makes the interview answer reproducible: any interviewer can hand you a scenario and you can walk the tree out loud. Walk through the tree with three canonical scenarios: a polyglot team consolidating three tools, a Python-only data platform at 500k tasks/day, and a no-code business team.

• Q1. Do you need polyglot scripts (TS + Python + Bash in the same platform)? → yes = go to Q2; no = go to Q3.
• Q2. Do you also need internal-tool UIs bound to those scripts? → yes = Windmill; no = still likely Windmill (or a lighter runner).
• Q3. Is the workload Python-first orchestration at ≥100k tasks/day with asset-lineage requirements? → yes = Dagster; no = go to Q4.
• Q4. Is the workload Python-first orchestration at ≥100k tasks/day without lineage? → yes = Airflow; no = go to Q5.
• Q5. Are the primary authors non-engineers dragging SaaS boxes? → yes = n8n; polished customer-facing CRUD = Retool.

Question. Walk the decision tree for the three scenarios and record the platform each ends up with.

Input.

Scenario	Q1 (polyglot?)	Q2 (apps?)	Q3 (Py+lineage?)	Q4 (Py-scale?)	Q5 (no-code/CRUD?)
Polyglot team, 3 tools	yes	yes	—	—	—
Python data platform 500k/day	no	no	yes	—	—
Business ops team	no	no	no	no	yes (no-code)

Code.

# Decision-tree helper (illustrative)
def pick_workflow_platform(polyglot: bool,
                            wants_apps: bool,
                            python_lineage: bool,
                            python_scale: bool,
                            no_code: bool,
                            polished_crud: bool) -> str:
    """Return the primary workflow platform for a scenario."""
    if polyglot:
        return "Windmill"       # Q1 dominates
    if python_lineage:
        return "Dagster"        # Q3 asset-lineage
    if python_scale:
        return "Airflow"        # Q4 pure Python scale
    if polished_crud:
        return "Retool"         # Q5b polished UI
    if no_code:
        return "n8n"            # Q5a business-user drag
    return "Windmill"           # sensible default


print(pick_workflow_platform(True,  True,  False, False, False, False))
# → 'Windmill'

print(pick_workflow_platform(False, False, True,  True,  False, False))
# → 'Dagster'

print(pick_workflow_platform(False, False, False, False, True,  False))
# → 'n8n'

Step-by-step explanation.

1. Scenario 1 — a polyglot team of Python + TS engineers consolidating Airflow + Retool + Vault. Q1 = yes → Windmill. This is the modal Windmill adoption path in 2026: consolidation, not greenfield.
2. Scenario 2 — a Python-only data platform at 500k tasks/day with software-defined-asset requirements. Q1 = no, Q3 = yes → Dagster. Windmill would technically work but Dagster's asset lineage is a headline capability Windmill does not match.
3. Scenario 3 — a business-ops team wiring HubSpot to Slack to Google Sheets. Q1-Q4 = no, Q5 = yes → n8n. Windmill's engineer-first ergonomics are wrong for the no-code drag-and-drop use case.
4. The default fall-through is Windmill: absent a specific reason to pick something else, its breadth (polyglot + orchestration + apps + secrets) beats the specialised alternatives. This is the "sensible default" bias senior architects encode.
5. If Q1 through Q5 are all "no," you probably don't need a workflow platform at all — a cron + a bash script + sops is fine. Refuse to adopt a platform until the scenario justifies one.

Output.

Scenario	Primary platform	Why
Polyglot team, 3 tools	Windmill	consolidation, polyglot, apps
Python data platform 500k/day + lineage	Dagster	asset-lineage-native
Python data platform 500k/day, no lineage	Airflow	scale, community, ecosystem
Business ops team, SaaS wiring	n8n	no-code drag-and-drop
Polished customer-facing CRUD	Retool	UI polish

Rule of thumb. The five-question decision tree is a whiteboard-friendly answer. Practice walking it end-to-end so an interviewer can hand you any scenario and get a platform name in under 60 seconds.

Senior interview question on Windmill platform selection

A senior interviewer often opens with: "You inherit a small data team running Airflow for ETL, Retool for a support-team GDPR UI, and HashiCorp Vault for secrets. The team is 3 Python + 2 TS engineers, self-hosted only, and the CFO wants to cut the SaaS bill. Walk me through consolidating to Windmill, the migration order, the operational win, and the risks you'd guard against."

Solution Using Windmill as the consolidation target with a script-first migration order

# Step 1 — deploy Windmill via docker-compose (one binary + Postgres)
version: "3.8"
services:
  windmill_server:
    image: ghcr.io/windmill-labs/windmill:latest
    environment:
      DATABASE_URL: postgres://windmill:${WM_PASSWORD}@db:5432/windmill
      MODE: server
      BASE_URL: https://windmill.internal.example
      RUST_LOG: info
    ports: ["8000:8000"]

  windmill_worker:
    image: ghcr.io/windmill-labs/windmill:latest
    environment:
      DATABASE_URL: postgres://windmill:${WM_PASSWORD}@db:5432/windmill
      MODE: worker
      WORKER_GROUP: default
      WORKER_TAGS: "default,python,ts,bash"
    deploy:
      replicas: 3

  db:
    image: postgres:16
    environment:
      POSTGRES_USER: windmill
      POSTGRES_PASSWORD: ${WM_PASSWORD}
      POSTGRES_DB: windmill

# Step 2 — migrate one Airflow DAG at a time as a Windmill flow
# Old: Airflow DAG (~120 lines Python + operators + XComs)
# New: Windmill flow (declarative YAML + per-step scripts)

# /f/etl/snowflake_load.script.py
# 'main' is the entry point Windmill expects
import wmill                       # the Windmill Python SDK (auto-installed)
import psycopg2, snowflake.connector

def main(source_date: str) -> dict:
    """Nightly Postgres → Snowflake load for a single date partition."""
    pg   = wmill.get_resource("u/team/pg_prod")       # typed resource
    sf   = wmill.get_resource("u/team/snowflake_dw")
    slack_hook = wmill.get_variable("u/team/SLACK_HOOK")

    conn = psycopg2.connect(**pg)
    with conn.cursor() as cur:
        cur.execute(
            "SELECT id, ts, payload FROM events WHERE date(ts) = %s",
            (source_date,),
        )
        rows = cur.fetchall()

    sfc = snowflake.connector.connect(**sf)
    with sfc.cursor() as cur:
        cur.executemany(
            "INSERT INTO raw.events(id, ts, payload) VALUES (%s, %s, %s)",
            rows,
        )

    return {"loaded": len(rows), "date": source_date, "slack": slack_hook is not None}

# Step 3 — bind the script into a scheduled flow
# /f/etl/nightly_snowflake.flow.yaml (Windmill flow spec — a subset of OpenFlow)
summary: Nightly Snowflake load
schedule:
  cron: "0 2 * * *"        # 02:00 UTC daily
  timezone: UTC
value:
  modules:
    - id: load_yesterday
      value:
        type: script
        path: f/etl/snowflake_load
        input_transforms:
          source_date:
            type: javascript
            expr: "new Date(Date.now() - 86400000).toISOString().slice(0,10)"
    - id: notify_slack
      value:
        type: script
        path: f/ops/slack_notify
        input_transforms:
          message:
            type: javascript
            expr: "`Loaded ${results.load_yesterday.loaded} rows for ${results.load_yesterday.date}`"

// Step 4 — migrate the Retool GDPR UI to a Windmill app + TS script
// /f/support/gdpr_erase.script.ts
import * as wmill from "windmill-client"

type Input = { user_email: string, confirmed: boolean }
type Output = { deleted_rows: number, ticket_url: string }

export async function main({ user_email, confirmed }: Input): Promise<Output> {
    if (!confirmed) throw new Error("must confirm the erasure")
    const pg = await wmill.getResource("u/team/pg_prod")
    // ... run the actual erase against Postgres with typed connection ...
    const deleted = 42
    return {
        deleted_rows: deleted,
        ticket_url: `https://tickets.example.com/gdpr/${user_email}`,
    }
}

Step-by-step trace.

Step	Before (Airflow + Retool + Vault)	After (Windmill)
Deployment surface	5 processes (scheduler, webserver, workers, Redis, Postgres) + Retool cloud + Vault	1 binary + Postgres
Secret storage	Vault (separate service, custom auth flow)	Windmill variables (encrypted, workspace RBAC)
ETL DAG count	12 Airflow DAGs	12 Windmill flows
Internal UI count	8 Retool apps	8 Windmill apps bound to scripts
Polyglot scripts	Python only (Bash via BashOperator)	TS / Py / Bash / Go / SQL
Per-seat cost	~$50/user/month Retool	zero (self-hosted, unlimited users)
Audit trail	Airflow logs + Retool audit (per-tool)	one Windmill audit log
Migration order	—	1) deploy; 2) migrate scripts; 3) migrate flows; 4) migrate apps; 5) decommission

After the migration, the team runs on one platform: 12 flows on cron, 8 apps bound to the same scripts (form widget → script call → typed output), 40 secrets in Windmill variables, workspace RBAC enforced per team. The CFO's SaaS bill drops (Retool cancelled); the operational surface drops (Vault + Airflow schedulers decommissioned); onboarding drops from "learn three tools" to "learn Windmill."

Output:

Metric	Before	After
Distinct platforms	3 (Airflow + Retool + Vault)	1 (Windmill)
Monthly SaaS spend	$2,400 (Retool 40 seats)	$0
Onboarding time	3 tools × ~2 days	~3 days total
Secret propagation	Vault → env vars in Airflow	Windmill variables (native)
Audit trail	3 log streams	1 log stream
Polyglot support	Python + Bash	full polyglot

Why this works — concept by concept:

• One binary + one Postgres — the deployment surface shrinks by 5x. Postgres becomes the queue (LISTEN/NOTIFY + row-level locking), the metadata store, the audit log, and the secret store. One backup strategy covers everything.
• Typed resources via wmill.get_resource — the connection details (host, port, credentials) live as encrypted resources; the script fetches them by path. No env-var soup, no hard-coded secrets, no Vault sidecar.
• Scripts as first-class entities — a script is the atomic unit. Flows compose scripts. Apps bind widgets to scripts. Schedules trigger scripts. The unification means one script backs the ETL cron, the internal-tool UI, and the ad-hoc runner — no duplication.
• Workspace RBAC + audit — every run is attributed to a user, every flow has explicit can_view / can_run / can_edit grants, and the audit log is a Postgres table you can query. This is the compliance story that replaces Retool's audit + Airflow's user permissions + Vault's ACLs.
• Cost — one Rust worker pod costs ~50 MB RAM idle; three workers handle the team's load. Postgres storage is dominated by the audit log. The eliminated cost is 40 Retool seats + Vault licensing + Airflow scheduler ops. Net one-tool operational cost versus three-tool operational cost — the migration ROI is measured in weeks.

SQL
Topic — sql
SQL orchestration and warehouse-load problems

Practice →

Design Topic — design Design problems on workflow platform selection

Practice →

---

2. Scripts, flows & modules

The three Windmill primitives — script, flow, module — cover every workflow shape you have, all under the OpenFlow spec

The mental model in one line: Windmill's runtime is built on exactly three primitives — a script is a single function in TypeScript / Python / Bash / Go / SQL with typed inputs and a returned object; a flow is a DAG of script steps with branches, for-each loops, and suspend-resume nodes; a module is a reusable script imported by other scripts and flows — and all three are serialised in the OpenFlow spec, an open-standard YAML format that makes every Windmill workflow portable and Git-diffable. Every senior DE who adopts Windmill learns these three shapes in the first hour and never needs a fourth abstraction.

Scripts — the atomic unit.

• What it is. A single exported function (main in Python, export async function main in TS) with typed parameters and a typed return. Windmill introspects the signature to generate an auto-form in the UI and validates inputs on every call.
• Runtimes. TypeScript on Deno (fast startup, secure by default) or Bun (faster startup, npm-compatible). Python in a per-script venv (hermetic dependencies). Bash for shell workflows. Go for statically compiled hot paths. SQL scripts that run against a resource-connected database with typed input parameters.
• Deps. Python uses # requirements: pandas==2.2.0-style comments; TS uses inline import { … } from "npm:package"; both are cached per version. First call is slow (dep install); subsequent calls hit the cache.
• Storage. Every script lives at a path (f/etl/snowflake_load — folder-organised) with versioning; every publish creates a new version accessible by hash.

Flows — the DAG composer.

• What it is. A YAML document (or graphical builder) that chains script steps into a DAG. Each step can consume the outputs of previous steps via results.<step_id>.<field>.
• Control flow. Branch-one (pick one path via a JS expression), branch-all (parallel fan-out), for-each (iterate an array with configurable parallelism), while-loop (until a condition), sleep (delay), suspend (durable pause awaiting an external signal like a webhook or Slack approval).
• Failure modes. Each step has explicit retry policy (attempts, backoff, on-failure branch); the flow either succeeds fully or ends up in a failure branch you defined.
• Suspend + resume. A flow can suspend on an approval step (durable in Postgres), await an HTTP POST to a resume URL, and pick up hours or days later — critical for human-in-the-loop workflows.

Modules — the reusable script.

• What it is. A script written once and imported by other scripts or referenced by other flows. The distinction between "script" and "module" is convention (a module is any script others reuse), not a separate primitive.
• Import. From TypeScript: import { helper } from "/f/lib/helpers". From Python: from wmill import require; helper = require("f/lib/helpers"). Windmill resolves the path at runtime.
• Versioning. Modules are versioned like every other script; pin to a hash for reproducibility or float on latest.

OpenFlow spec — the portability layer.

• What it is. An open YAML schema documenting every flow, script, resource, and variable shape Windmill uses. Published by Windmill Labs and adopted (at least in intent) as a portability standard for the workflow-tool ecosystem.
• Why it matters. A flow expressed in OpenFlow is diffable in Git, reviewable in PRs, and — in theory — executable by any compatible runtime. In practice Windmill is the reference implementation, but the spec is what makes the "export your flows to Git" story credible.
• Git sync. Windmill's wmill sync CLI round-trips the workspace to a Git repo (f/etl/nightly_snowflake.flow.yaml on disk = the flow in the workspace). Every workspace change is a diff; every PR is a workflow review.

Common interview probes on scripts / flows / modules.

• "What's a Windmill script exactly?" — required answer: a single typed function in one of the supported languages, path-addressed, versioned, with auto-generated form UI.
• "How do flows compose scripts?" — YAML DAG of script steps with typed input transforms and typed output binding.
• "What's OpenFlow?" — the open YAML spec that describes flows portably.
• "How do you reuse code across scripts?" — modules (import a script by path).

Worked example — TypeScript extract-transform script

Detailed explanation. The canonical Windmill TS script: fetches rows from Postgres via a typed resource, transforms them, and returns a typed output. Walk through the whole file — the signature, the resource injection, the return shape, the deployment path.

• Path. f/etl/postgres_extract (folder-organised, workspace-scoped).
• Runtime. Deno + TypeScript.
• Resource. u/team/pg_prod — a Postgres connection resource.
• Input. { table: string, limit: number }.
• Output. { rows: Row[], count: number, extracted_at: string }.

Question. Write the extract-transform script with typed inputs, a Postgres resource, and a JSON-serialisable return.

Input.

Parameter	Type	Purpose
table	string	fully qualified source table
limit	number	max rows to return
resource u/team/pg_prod	Postgres	typed connection object

Code.

// /f/etl/postgres_extract.script.ts
// Runtime: Deno
// Deps: npm:pg (Windmill caches per-version)
import * as wmill from "windmill-client"
import { Client } from "npm:pg"

type Input = {
    table: string
    limit: number
}
type Row = {
    id: number
    ts: string
    payload: Record<string, unknown>
}
type Output = {
    rows: Row[]
    count: number
    extracted_at: string
}

export async function main({ table, limit }: Input): Promise<Output> {
    // Fetch the typed Postgres resource (host / port / user / password / db)
    const pg = await wmill.getResource<{
        host: string; port: number; user: string; password: string; dbname: string
    }>("u/team/pg_prod")

    const client = new Client({
        host: pg.host, port: pg.port,
        user: pg.user, password: pg.password,
        database: pg.dbname,
    })
    await client.connect()
    try {
        // Parameter-bound query (SQL-injection-safe; Windmill also lints)
        const res = await client.query(
            `SELECT id, ts, payload FROM ${table} ORDER BY id DESC LIMIT $1`,
            [limit],
        )
        const rows: Row[] = res.rows.map(r => ({
            id: Number(r.id),
            ts: new Date(r.ts).toISOString(),
            payload: r.payload as Record<string, unknown>,
        }))
        return {
            rows,
            count: rows.length,
            extracted_at: new Date().toISOString(),
        }
    } finally {
        await client.end()
    }
}

Step-by-step explanation.

1. The Input and Output type aliases are Windmill's contract. The main function is introspected at deploy time; Windmill emits a form UI (a text field for table, a number field for limit) and a JSON-schema validator that rejects bad inputs before the script runs.
2. wmill.getResource<T>("u/team/pg_prod") fetches the typed resource. The u/team/ prefix is workspace-scoped; pg_prod is the resource name. The generic <T> gives you type-checked access to the fields.
3. The Postgres client uses npm:pg — Deno's npm interop means any npm package works out of the box, with Windmill caching the resolved dependency graph per-version.
4. The query uses parameter binding ($1) rather than string interpolation — this is SQL-injection-safe and Windmill's static analysis warns you if you interpolate directly. Table names cannot be parameterised in Postgres, so the interpolation of table is a controlled trust boundary the script accepts.
5. The return shape is a plain JSON-serialisable object. Windmill stores the return, feeds it to downstream flow steps as results.<this_step>.<field>, and displays it in the UI as a typed panel.

Output.

Return field	Type	Example
rows	Row[]	100-row array of {id, ts, payload}
count	number	100
extracted_at	ISO string	"2026-07-21T10:15:03.421Z"

Rule of thumb. Every Windmill script exports one main function with typed input and typed output. The types drive the UI, the validation, and the flow-step wiring. Treat the type signature as the API contract for the script.

Worked example — Python S3 loader with hermetic deps

Detailed explanation. The canonical Windmill Python script: reads a JSON-array input, uploads to S3 via a typed AWS resource, and returns an upload manifest. The hermetic per-script venv means Python 3.11 + boto3 1.34 in this script doesn't fight a different boto3 version in another script.

• Path. f/etl/s3_json_upload.
• Runtime. Python 3.11 in a per-script venv.
• Resource. u/team/aws_s3 — an AWS credentials resource.
• Input. { rows: list[dict], bucket: str, prefix: str }.
• Output. { s3_key: str, byte_count: int, uploaded_at: str }.

Question. Write the S3 loader with typed AWS resource injection and a hermetic venv.

Input.

Parameter	Type	Purpose
rows	list of dicts	the rows to upload as JSONL
bucket	string	S3 bucket
prefix	string	S3 key prefix
resource u/team/aws_s3	AWS creds	typed connection

Code.

# /f/etl/s3_json_upload.script.py
# Runtime: Python 3.11 (hermetic venv)
# requirements:
#   boto3==1.34.90
#   ujson==5.9.0
import json
import ujson
from datetime import datetime, timezone
from typing import TypedDict, Any

import boto3
import wmill                            # auto-provided by Windmill

class Aws(TypedDict):
    access_key_id: str
    secret_access_key: str
    region: str

class Output(TypedDict):
    s3_key: str
    byte_count: int
    uploaded_at: str

def main(rows: list[dict[str, Any]], bucket: str, prefix: str) -> Output:
    """Serialise `rows` as newline-delimited JSON and upload to s3://{bucket}/{prefix}/…"""
    aws: Aws = wmill.get_resource("u/team/aws_s3")

    body = "\n".join(ujson.dumps(r) for r in rows).encode("utf-8")
    key  = f"{prefix.rstrip('/')}/{datetime.now(timezone.utc):%Y/%m/%d/%H%M%S}.jsonl"

    client = boto3.client(
        "s3",
        aws_access_key_id=aws["access_key_id"],
        aws_secret_access_key=aws["secret_access_key"],
        region_name=aws["region"],
    )
    client.put_object(Bucket=bucket, Key=key, Body=body, ContentType="application/x-ndjson")

    return {
        "s3_key":      f"s3://{bucket}/{key}",
        "byte_count":  len(body),
        "uploaded_at": datetime.now(timezone.utc).isoformat(),
    }

Step-by-step explanation.

1. The # requirements: comment declares the pip dependencies. Windmill parses this at deploy time, creates a venv, and installs the deps once per version — subsequent invocations reuse the cached venv (~ms startup vs seconds for a cold install).
2. The wmill.get_resource("u/team/aws_s3") call returns a typed dict Windmill has decoded from the encrypted resource store. The TypedDict annotation is optional but makes static analysis catch typos.
3. ujson is imported instead of stdlib json for ~3x faster serialisation on large row-arrays. Because each script has its own venv, this doesn't pollute other scripts' sys.modules.
4. The S3 key is time-partitioned by the current UTC clock. Windmill scripts run on workers whose clock is authoritative — no per-writer clock skew like the CDC world worries about, because there's one worker executing the script.
5. The return shape uses TypedDict (not a class) so the return value is a plain JSON dict, which Windmill can serialise into the run's output panel and pipe into a downstream flow step.

Output.

Return field	Example
s3_key	s3://acme-bronze/etl/orders/2026/07/21/102503.jsonl
byte_count	48,213
uploaded_at	2026-07-21T10:25:03.812+00:00

Rule of thumb. Declare pip deps via the # requirements: comment for hermetic venvs; use TypedDict for typed resources; return JSON-serialisable objects so downstream flow steps can consume them. Windmill's caching makes the first call slow but every subsequent call fast.

Worked example — forEach fan-out flow with suspend + approval

Detailed explanation. The canonical Windmill flow: for-each over an array of items, run a script per item, then suspend for human approval before a "commit" step, then resume on webhook and finish. This is the human-in-the-loop pattern every ops team eventually needs.

• Extract step. Query the DB for pending items.
• For-each step. Parallel fan-out over each item; call the transform script.
• Suspend step. Pause the flow; emit a Slack message with the approve/reject links.
• Approve step. On resume, run the commit script.

Question. Design the flow that fans-out, suspends for approval, and resumes.

Input.

Component	Value
Extract	f/etl/pending_items
Transform	f/etl/transform_item
Slack notify	f/ops/slack_approval
Commit	f/etl/commit_batch
Suspend timeout	24 hours
Approvers	user group data-ops

Code.

# /f/etl/nightly_approval.flow.yaml (OpenFlow-compliant)
summary: Nightly batch — fan out, await approval, commit
schedule:
  cron: "0 3 * * *"
  timezone: UTC
value:
  modules:
    # 1. Extract pending items
    - id: extract
      value:
        type: script
        path: f/etl/pending_items

    # 2. For each item, run the transform in parallel (up to 8 concurrent)
    - id: transform_each
      value:
        type: forloopflow
        iterator:
          type: javascript
          expr: "results.extract.items"
        parallel: true
        parallelism: 8
        modules:
          - id: transform
            value:
              type: script
              path: f/etl/transform_item
              input_transforms:
                item:
                  type: javascript
                  expr: "flow_input.iter.value"

    # 3. Suspend the flow — send a Slack message with the resume URL
    - id: await_approval
      value:
        type: script
        path: f/ops/slack_approval
        suspend:
          required_events: 1
          timeout: 86400          # 24h
          resume_form:
            schema:
              type: object
              properties:
                approved:
                  type: boolean
                  description: Approve the batch?

    # 4. Branch: commit if approved, otherwise skip
    - id: maybe_commit
      value:
        type: branchone
        branches:
          - summary: approved
            expr: "results.await_approval.approved === true"
            modules:
              - id: commit
                value:
                  type: script
                  path: f/etl/commit_batch
                  input_transforms:
                    items:
                      type: javascript
                      expr: "results.transform_each"
        default:
          - id: notify_rejected
            value:
              type: script
              path: f/ops/slack_notify
              input_transforms:
                message:
                  type: static
                  value: "Batch rejected — no commit"

Step-by-step explanation.

1. Step 1 (extract) is a normal script call. Its return object becomes results.extract, referenceable by every downstream step via a JS expression.
2. Step 2 (transform_each) is a forloopflow — Windmill iterates over results.extract.items and runs the inner transform script per item, up to 8 in parallel. Each iteration's return is collected into results.transform_each as an array.
3. Step 3 (await_approval) uses suspend — the script runs (posting to Slack with a resume URL), then the flow pauses durably in Postgres. The 24-hour timeout means an un-approved flow auto-cancels; the resume_form schema defines the payload the resume webhook expects.
4. Step 4 (maybe_commit) is a branchone — a JS expression on results.await_approval.approved picks the approved path (running commit) or falls through to the default (notify rejected). Branches are typed subflows.
5. The whole flow is one YAML document, Git-diffable, PR-reviewable, and portable — any OpenFlow-compatible runtime could in theory execute it. This is the flow-as-code promise.

Output.

Step	State after
extract	{items: [...]} — 42 items
transform_each	array of 42 transform results
await_approval	suspended for up to 24h in Postgres
Slack approver clicks approve → resume webhook	results.await_approval = {approved: true}
maybe_commit → commit	commit script runs on the 42 items
Flow done	success in ~1 minute + human time

Rule of thumb. Suspend + resume is the killer feature for human-in-the-loop workflows. Design the resume form schema explicitly (never accept freeform JSON on resume) and always set a timeout so orphaned flows don't accumulate in Postgres.

Senior interview question on Windmill scripts and flows

A senior interviewer might ask: "Design a Windmill flow that ingests a nightly batch of orders, validates them, requires ops approval before a commit, and retries failed items with exponential backoff. Include the script signatures, the flow YAML shape, the resource + variable use, and the failure-branch behaviour."

Solution Using scripts + a forloopflow + suspend/resume + a failure branch

# /f/etl/fetch_orders.script.py
import wmill, psycopg2
from datetime import date, timedelta
from typing import Any

def main() -> dict[str, Any]:
    pg = wmill.get_resource("u/team/pg_prod")
    yday = date.today() - timedelta(days=1)
    conn = psycopg2.connect(**pg)
    with conn.cursor() as cur:
        cur.execute(
            "SELECT id, customer_id, total_cents, status FROM orders WHERE date(created_at) = %s",
            (yday,),
        )
        rows = [dict(id=r[0], customer_id=r[1], total_cents=r[2], status=r[3]) for r in cur.fetchall()]
    return {"orders": rows, "date": yday.isoformat()}

// /f/etl/validate_order.script.ts
import * as wmill from "windmill-client"

type Order = { id: number, customer_id: number, total_cents: number, status: string }
type Result = { id: number, valid: boolean, reason?: string }

export async function main({ order }: { order: Order }): Promise<Result> {
    if (order.total_cents <= 0)  return { id: order.id, valid: false, reason: "non-positive total" }
    if (order.customer_id <= 0)  return { id: order.id, valid: false, reason: "invalid customer" }
    if (!["pending","shipped","cancelled"].includes(order.status))
        return { id: order.id, valid: false, reason: "unknown status" }
    return { id: order.id, valid: true }
}

# /f/etl/nightly_orders.flow.yaml
summary: Nightly order ingest with validation + approval + retry
schedule:
  cron: "15 3 * * *"
  timezone: UTC
value:
  modules:
    - id: fetch
      value:
        type: script
        path: f/etl/fetch_orders

    - id: validate_each
      value:
        type: forloopflow
        iterator:
          type: javascript
          expr: "results.fetch.orders"
        parallel: true
        parallelism: 16
        modules:
          - id: validate
            value:
              type: script
              path: f/etl/validate_order
              input_transforms:
                order:
                  type: javascript
                  expr: "flow_input.iter.value"
              retry:
                constant: { attempts: 3, seconds: 2 }

    - id: split_results
      value:
        type: script
        path: f/etl/partition_valid
        input_transforms:
          results:
            type: javascript
            expr: "results.validate_each"

    - id: await_approval
      value:
        type: script
        path: f/ops/slack_approval
        input_transforms:
          summary:
            type: javascript
            expr: "`${results.split_results.valid.length} valid / ${results.split_results.invalid.length} invalid`"
        suspend:
          required_events: 1
          timeout: 86400

    - id: maybe_commit
      value:
        type: branchone
        branches:
          - expr: "results.await_approval.approved === true"
            modules:
              - id: commit
                value:
                  type: script
                  path: f/etl/commit_orders
                  input_transforms:
                    orders:
                      type: javascript
                      expr: "results.split_results.valid"
        default:
          - id: notify_rejected
            value:
              type: script
              path: f/ops/slack_notify

  failure_module:
    id: on_failure
    value:
      type: script
      path: f/ops/notify_pagerduty

Step-by-step trace.

Layer	Config	Result
Extract	one Python script	fetches ~1000 orders/night
Validate	forloopflow, parallelism 16	1000 parallel validations in ~5s
Retry	constant backoff, 3 attempts	transient DB blips absorbed
Suspend	Slack approval, 24h timeout	durable pause in Postgres
Commit	branch on approved==true	only valid rows commit
Failure branch	failure_module on flow-level fail	PagerDuty alert

After deployment, the nightly flow fetches yesterday's orders, validates each in parallel with retries, summarises the split (valid vs invalid) to Slack, suspends for human approval, and either commits or notifies. The failure branch fires PagerDuty if any step exhausts its retries — one alert per failed run, not one per failed item.

Output:

Metric	Value
Nightly runtime (steady state)	~10 s + human approval
Parallelism during validate	16 workers
Retry attempts per item	up to 3
Suspend durability	Postgres row (survives worker restart)
Failure alert	PagerDuty page on any flow failure
Audit trail	one run row per invocation, per-step details

Why this works — concept by concept:

• forloopflow with parallelism — Windmill's fan-out iterates the array on the workers themselves, not on the flow driver. Each iteration gets its own worker slot; parallelism 16 means 16 concurrent transforms; the total wall-clock is ceil(N / 16) × per-item time.
• Retry policy per-step — the retry.constant block gives each validate call up to 3 attempts with a 2-second constant backoff. Transient Postgres blips or upstream flakes are absorbed without failing the whole flow.
• suspend + resume with schema — the flow pauses durably in Postgres. The 24h timeout auto-cancels orphaned approvals. The resume payload schema is validated, so an approver clicking a bad link cannot inject arbitrary JSON.
• branchone with JS expression — the branch predicate is a small JS expression evaluated on the flow driver; the chosen branch's sub-modules run inline as a sub-flow. This is the idiomatic if/else in Windmill flows.
• failure_module — the flow-level failure_module runs exactly once per failed flow, giving you a single deduplicated alert per run. This beats Airflow's per-task alerting for reducing noise.
• Cost — one Windmill flow row per invocation (~a few KB), one job row per step + iteration (dominated by the fan-out arity), and the suspend keeps the flow in Postgres until resumed. Total Postgres overhead is ~O(steps × iterations) per run. The eliminated cost is the Airflow scheduler + XCom + manual retry logic. Net one-flow definition versus 300+ lines of Airflow operator code.

SQL
Topic — sql
SQL fan-out and batch-approval workflows

Practice →

Design Topic — design Design problems on human-in-the-loop pipelines

Practice →

---

3. Workers, queue & executor architecture

Postgres is the queue, Rust binaries are the workers, Nsjail is the sandbox — a three-piece architecture that fits on one host or scales across a fleet

The mental model in one line: Windmill's executor is a Rust-native worker binary that polls a Postgres task queue (via LISTEN/NOTIFY and SELECT ... FOR UPDATE SKIP LOCKED), executes the picked-up script inside a hermetic runtime (Deno / Bun for TS, per-script venv for Python, native for Bash / Go), and — when configured — wraps the execution in Nsjail (Linux namespaces + seccomp) so a rogue script cannot see other scripts' secrets, filesystem, or network; the same three pieces (Postgres + worker + optional sandbox) scale from a single laptop to a multi-node fleet with pool tags for GPU / memory / priority routing. This architecture is the reason Windmill deploys as one binary + one Postgres, and it is the axis senior interviewers probe hardest.

Postgres as the queue.

• The queue table. A single queue table holds pending, running, and just-finished jobs. Each row has a JSON payload, a tag list, a scheduled-for timestamp, and a running-worker id.
• Job pickup. Workers SELECT ... FROM queue WHERE state='pending' AND scheduled_for <= now() AND tags && $worker_tags ORDER BY scheduled_for FOR UPDATE SKIP LOCKED LIMIT 1 — the row-lock plus SKIP LOCKED ensures no two workers pick the same job.
• Wake-up. Postgres LISTEN/NOTIFY wakes workers on new-job arrival (sub-100ms), falling back to a 500ms poll if the LISTEN connection drops.
• Durability. Because the queue is Postgres, the same backup / replica strategy that covers your app data covers your queue. No separate Redis / RabbitMQ cluster to backup and monitor.

Rust workers — stateless and horizontal.

• What they are. A single Rust binary, ~50MB RAM idle. Configured via MODE=worker, DATABASE_URL, and WORKER_TAGS. Ships with Deno, Bun, Python bootstrap, Go compiler as embedded/adjacent tooling.
• Scale-out. Add more worker replicas → more parallel jobs. There is no coordinator; workers race for jobs via SKIP LOCKED. Scale down: kill workers; running jobs complete then the worker exits.
• Tag-based routing. WORKER_TAGS=default,python,ts means the worker will pick up jobs tagged with any of those tags. Scripts declare their tag via WORKER_TAGS or per-script frontmatter — this is how you route heavy Python jobs to python-heavy workers, GPU jobs to GPU-equipped workers, high-memory jobs to high-mem workers.
• Failure semantics. At-least-once: if a worker crashes mid-job, the row's lock is released, another worker picks it up, and the script runs again. Idempotency is the caller's responsibility — Windmill guarantees delivery, not exactly-once execution.

Isolation modes.

• Native. The script runs in the worker's own process (Deno / Bun sub-process for TS, sub-process for Py in the shared venv). Fast, no sandbox overhead. Trust boundary: any script can read any env var the worker sees.
• Nsjail. The script runs inside an nsjail invocation — new Linux namespaces (mount, pid, network, user), seccomp filter, dropped capabilities. Each script gets its own filesystem view (no other scripts' cache visible), no host network unless explicitly enabled, no access to worker memory. Overhead ~5-20ms per invocation.
• Container-per-job. Not the default; some deployments front the worker with a Kubernetes Job launcher for full container isolation. Higher latency but stronger isolation.

Per-language runtime.

• TypeScript. Deno by default — secure-by-default (no filesystem or network unless granted), TypeScript-native, npm-compatible via npm: specifiers. Bun as an alternative for lower cold-start and full npm compatibility.
• Python. Per-script venv. Windmill parses the # requirements: comment, creates a venv, installs pinned deps, caches the venv by hash. First call is dep-install-slow; subsequent calls hit the cache.
• Bash / Go. Bash runs in a subshell. Go compiles to a tiny binary and runs.
• SQL. SQL scripts run against a resource-connected database with typed input parameters — you get a script-shaped SQL query that behaves like a first-class function.

Common interview probes on the executor.

• "Where does Windmill's queue live?" — required answer: Postgres, one table, FOR UPDATE SKIP LOCKED for job pickup, LISTEN/NOTIFY for wake.
• "Is it exactly-once?" — no, at-least-once; scripts must be idempotent.
• "What's the isolation model?" — native for speed, Nsjail for sandbox, per-language hermetic env for dep isolation.
• "How do you scale horizontally?" — add worker replicas; no coordinator needed.

Worked example — worker deployment with tag-based routing

Detailed explanation. The canonical multi-worker deployment: three worker pools — default, python-heavy, and gpu-enabled — each with different resource limits and tag configurations. Jobs tagged appropriately route to the right pool automatically.

• Default pool. 3 replicas, 1 CPU / 2GB RAM each, tags default,python,ts,bash.
• Python-heavy pool. 2 replicas, 4 CPU / 16GB RAM, tags python-heavy,pandas,spark.
• GPU pool. 1 replica, 1 GPU / 32GB RAM, tags gpu,ml.

Question. Write the docker-compose that deploys three worker pools, then show a script that opts into the GPU pool via its tag.

Input.

Pool	Replicas	Tags	Resources
default	3	default,python,ts,bash	1 CPU / 2 GB
python-heavy	2	python-heavy,pandas,spark	4 CPU / 16 GB
gpu	1	gpu,ml	1 GPU / 32 GB

Code.

# docker-compose.yml — three worker pools serving one Postgres queue
version: "3.8"
services:
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: windmill
      POSTGRES_PASSWORD: ${WM_PASSWORD}
      POSTGRES_DB: windmill
    volumes: [pgdata:/var/lib/postgresql/data]

  server:
    image: ghcr.io/windmill-labs/windmill:latest
    depends_on: [db]
    environment:
      DATABASE_URL: postgres://windmill:${WM_PASSWORD}@db:5432/windmill
      MODE: server
    ports: ["8000:8000"]

  worker_default:
    image: ghcr.io/windmill-labs/windmill:latest
    depends_on: [db]
    environment:
      DATABASE_URL: postgres://windmill:${WM_PASSWORD}@db:5432/windmill
      MODE: worker
      WORKER_GROUP: default
      WORKER_TAGS: "default,python,ts,bash"
    deploy:
      replicas: 3
      resources: { limits: { cpus: "1.0", memory: "2G" } }

  worker_python_heavy:
    image: ghcr.io/windmill-labs/windmill:latest
    depends_on: [db]
    environment:
      DATABASE_URL: postgres://windmill:${WM_PASSWORD}@db:5432/windmill
      MODE: worker
      WORKER_GROUP: python_heavy
      WORKER_TAGS: "python-heavy,pandas,spark"
    deploy:
      replicas: 2
      resources: { limits: { cpus: "4.0", memory: "16G" } }

  worker_gpu:
    image: ghcr.io/windmill-labs/windmill:latest
    depends_on: [db]
    environment:
      DATABASE_URL: postgres://windmill:${WM_PASSWORD}@db:5432/windmill
      MODE: worker
      WORKER_GROUP: gpu
      WORKER_TAGS: "gpu,ml"
    deploy:
      replicas: 1
      resources: { limits: { cpus: "8.0", memory: "32G" } }
      # nvidia-container-runtime block elided

# /f/ml/embed_batch.script.py
# Route to the GPU pool via the WORKER_TAG frontmatter
# WORKER_TAG: gpu
import wmill, torch
from typing import Any

def main(texts: list[str]) -> dict[str, Any]:
    # This script runs on the gpu-tagged worker
    device = "cuda" if torch.cuda.is_available() else "cpu"
    # ... run an embedding model ...
    return {"n": len(texts), "device": device}

Step-by-step explanation.

1. The docker-compose.yml defines four Windmill service groups: the server, the default worker pool (3 replicas), the python-heavy pool (2 replicas, 16GB RAM), and the GPU pool (1 replica, GPU). All four talk to the same Postgres queue.
2. Each worker's WORKER_TAGS env var declares which script tags it will pick up. The default pool takes default,python,ts,bash; the python-heavy pool takes python-heavy,pandas,spark; the GPU pool takes gpu,ml. Tags are OR-matched — a script tagged python-heavy will be picked up by any worker whose tag list contains python-heavy.
3. Scripts opt into a pool via a per-script WORKER_TAG frontmatter comment. The embed_batch script tags itself gpu, so only the GPU worker picks it up. If the GPU worker is offline, the job queues (does not fall back to a non-GPU worker) — this is the correctness property tag routing gives you.
4. Because the queue is Postgres and workers race via SKIP LOCKED, adding a new worker is zero-config: point it at the same DATABASE_URL, set its tag list, and it starts pulling matching jobs.
5. Scale-down is graceful: killing a worker releases its in-flight job's lock (once the worker exits); another matching-tagged worker picks up the released job. In-flight scripts finish before the worker exits (via SIGTERM handling with a drain period).

Output.

Job tag	Which pool runs it	Concurrency
default	default (3 replicas)	3
python-heavy	python-heavy (2 replicas × 1 slot)	2
gpu	gpu (1 replica)	1
Untagged fallback	none — jobs queue until a matching worker starts	0

Rule of thumb. Use tags to route, not to rate-limit. Each worker is single-slot by default — if you need higher concurrency for a tag, add more workers with that tag, not more slots per worker. Tag routing is the closest Windmill comes to a "resource class" abstraction.

Worked example — Nsjail isolation for untrusted scripts

Detailed explanation. A common Windmill deployment scenario: contractors or interns can author scripts, but they must not be able to cat /etc/passwd, exfiltrate the DATABASE_URL, or reach other scripts' filesystem cache. Nsjail isolation is the answer. Walk through enabling it and quantifying the overhead.

• Config. Set NSJAIL=true on the worker; Windmill wraps every script execution in an nsjail invocation.
• What it blocks. Filesystem access outside the script's private /tmp; network access unless explicitly allowed; syscalls outside the seccomp allowlist.
• Overhead. ~5-20 ms per script invocation (namespace + seccomp setup). For sub-second scripts this doubles latency; for multi-second scripts it's negligible.

Question. Enable Nsjail on the default worker pool, write a script that tries (and fails) to read /etc/passwd, and quantify the added latency.

Input.

Component	Change
Worker env	NSJAIL=true, DISABLE_NSJAIL=false
Script under test	reads /etc/passwd
Latency probe	1000 no-op invocations native vs Nsjail

Code.

# docker-compose.yml (default worker with Nsjail)
worker_default:
  image: ghcr.io/windmill-labs/windmill:latest
  privileged: true                    # nsjail needs CAP_SYS_ADMIN
  environment:
    DATABASE_URL: postgres://windmill:${WM_PASSWORD}@db:5432/windmill
    MODE: worker
    WORKER_GROUP: default
    WORKER_TAGS: "default,python,ts,bash"
    NSJAIL: "true"                    # wrap every script in nsjail
    DISABLE_NSJAIL: "false"
  deploy: { replicas: 3 }

# /f/tests/read_passwd.script.py — should fail under Nsjail
def main() -> str:
    with open("/etc/passwd") as f:
        return f.read()

# Latency probe — measure per-invocation overhead
wmill script run f/tests/noop --count 1000 --measure
# Native worker  → mean 8ms, p99 15ms
# Nsjail worker  → mean 25ms, p99 45ms

Step-by-step explanation.

1. Enabling Nsjail requires NSJAIL=true and privileged: true on the container (Nsjail uses CAP_SYS_ADMIN to create namespaces). In a Kubernetes deployment you grant the equivalent security context.
2. When Nsjail is enabled, every script execution is wrapped in nsjail --user 65534 --group 65534 --disable_clone_newipc --iface_no_lo …. New mount, pid, and network namespaces; the script sees an empty /tmp, no host processes, no host network.
3. The read_passwd script fails with FileNotFoundError: [Errno 2] No such file or directory: '/etc/passwd' — the namespace's minimal rootfs does not include /etc. This is the security property: even a script written to exfiltrate host files cannot see them.
4. Latency probe shows native ~8ms mean vs Nsjail ~25ms mean per no-op — a ~17ms tax per invocation. For a script that does real work (query a DB, transform rows), this is noise; for a "call this script 10k times a second" hot path, it's meaningful.
5. Trust-boundary decision: if the workspace has only trusted authors, run native for speed. If contractors, students, or auto-generated scripts run in the workspace, run Nsjail. Windmill supports mixed pools — one native pool for trusted teams, one Nsjail pool for untrusted.

Output.

Metric	Native	Nsjail
/etc/passwd readable?	yes	no
Mean invocation latency	~8 ms	~25 ms
p99 latency	~15 ms	~45 ms
Additional overhead	0	~17 ms
Trust model	high (trusted authors only)	low (any author, sandboxed)

Rule of thumb. Turn Nsjail on for any workspace where non-employees write scripts. The 17ms overhead is a rounding error for real work and a hard security requirement for shared platforms. Split pools if you have both trusted and untrusted authors — one native pool for internal DE code, one Nsjail pool for everything else.

Worked example — measuring queue depth and worker saturation

Detailed explanation. The two operational metrics every Windmill deployment must expose are (a) queue depth (pending jobs) and (b) worker saturation (percent of workers currently executing). Both are simple Postgres queries. Write them, wire them to Prometheus, and know your on-call response for each.

• Queue depth. SELECT count(*) FROM queue WHERE state = 'pending'. Grows when workers can't keep up.
• Worker saturation. SELECT count(*) FROM queue WHERE state = 'running' divided by total worker count.
• Runbook. Depth > 1000 for 5 min → add workers. Saturation = 100% for 10 min → add workers or shed load.

Question. Write the Postgres queries and a Prometheus exporter script that publishes both metrics every 15 seconds.

Input.

Metric	Postgres query	Prometheus name
pending	count(*) WHERE state='pending'	windmill_queue_pending
running	count(*) WHERE state='running'	windmill_queue_running
workers	count(*) FROM worker	windmill_worker_count

Code.

-- Queue depth (pending)
SELECT count(*) AS pending FROM queue WHERE state = 'pending';

-- Running jobs (in-flight)
SELECT count(*) AS running FROM queue WHERE state = 'running';

-- Worker count (heartbeated in last 60s)
SELECT count(*) AS active_workers
FROM   worker
WHERE  last_ping > now() - INTERVAL '60 seconds';

-- Combined saturation view
CREATE OR REPLACE VIEW windmill_saturation AS
SELECT
  (SELECT count(*) FROM queue WHERE state = 'pending')  AS pending,
  (SELECT count(*) FROM queue WHERE state = 'running')  AS running,
  (SELECT count(*) FROM worker WHERE last_ping > now() - INTERVAL '60 seconds') AS workers;

# /f/ops/prometheus_exporter.script.py — schedule this every 15s
# requirements:
#   prometheus_client==0.20.0
#   psycopg2-binary==2.9.9
import wmill
import psycopg2
from prometheus_client import CollectorRegistry, Gauge, push_to_gateway

def main() -> dict:
    pg = wmill.get_resource("u/team/pg_windmill")   # the Windmill queue DB
    registry = CollectorRegistry()

    g_pending = Gauge("windmill_queue_pending", "pending jobs", registry=registry)
    g_running = Gauge("windmill_queue_running", "running jobs", registry=registry)
    g_workers = Gauge("windmill_worker_count",  "active workers", registry=registry)

    conn = psycopg2.connect(**pg)
    with conn.cursor() as cur:
        cur.execute("SELECT pending, running, workers FROM windmill_saturation")
        pending, running, workers = cur.fetchone()

    g_pending.set(pending)
    g_running.set(running)
    g_workers.set(workers)

    push_to_gateway("pushgateway.internal:9091", job="windmill", registry=registry)

    saturation = round(running / max(workers, 1) * 100, 1)
    return {"pending": pending, "running": running, "workers": workers, "saturation_pct": saturation}

Step-by-step explanation.

1. The queue-depth query is a single count(*) WHERE state='pending'. It runs in sub-millisecond time even on multi-million-row queue tables because the state column is indexed.
2. The active_workers count filters on last_ping > now() - 60s — workers heartbeat into a worker table every 30s; anything older is either dead or restarted.
3. The saturation view is the atomic snapshot: three counts pulled in one transaction. This avoids the race where pending is measured, then running is measured 5 seconds later, and the numbers no longer reflect the same instant.
4. The Prometheus exporter script itself runs on Windmill on a 15s schedule — a Windmill flow monitoring itself. This is idiomatic; there's no "outside the system" monitoring layer required.
5. Runbook: pending > 1000 for 5 min → add worker replicas via docker-compose up --scale worker_default=5. Saturation = 100% for 10 min → same fix; consider whether one script is CPU-bound and needs a python-heavy tag route instead.

Output.

Metric	Green	Yellow	Red
Pending	< 100	100–1000	> 1000 for 5 min
Saturation	< 70%	70–95%	100% for 10 min
Workers	count(deploy)	1 short	> 1 short

Rule of thumb. Queue depth and worker saturation are the two SLIs every Windmill deployment must expose. Wire them to Prometheus + PagerDuty on day one; both queries are cheap and both alerts pay for themselves the first time a runaway flow saturates the pool.

Senior interview question on Windmill workers and queue

A senior interviewer might ask: "Your Windmill deployment is running 5000 flows/day across a mix of quick TS scripts and long-running Python jobs. Latency for interactive scripts has climbed from 200ms to 3s. Walk me through diagnosing whether it's queue depth, worker saturation, or dep-install cold starts, and design the fix."

Solution Using tag segregation + a warm venv cache + horizontal worker scaling

-- Step 1 — diagnose the split
-- Are interactive TS jobs waiting behind long Python jobs on the same workers?
SELECT
  tag,
  count(*) FILTER (WHERE state = 'pending') AS pending,
  count(*) FILTER (WHERE state = 'running') AS running,
  percentile_disc(0.95) WITHIN GROUP (ORDER BY duration_ms) FILTER (WHERE state = 'success') AS p95_dur_ms
FROM queue
WHERE started_at > now() - INTERVAL '1 hour'
GROUP BY tag
ORDER BY pending DESC;

-- Typical output showing the bottleneck:
--  tag              | pending | running | p95_dur_ms
--  ------------------+---------+---------+-----------
--  python-heavy    |     123 |       3 |     45000
--  default         |      87 |       3 |      3200
--  ts-interactive  |      41 |       0 |       180

# Step 2 — split the pools: one interactive-only, one heavy-only
services:
  worker_interactive:
    image: ghcr.io/windmill-labs/windmill:latest
    environment:
      MODE: worker
      WORKER_GROUP: interactive
      WORKER_TAGS: "ts-interactive,default"    # picks quick TS jobs first
    deploy:
      replicas: 6                              # 6 quick workers
      resources: { limits: { cpus: "1.0", memory: "1G" } }

  worker_python_heavy:
    image: ghcr.io/windmill-labs/windmill:latest
    environment:
      MODE: worker
      WORKER_GROUP: python_heavy
      WORKER_TAGS: "python-heavy"              # only picks long jobs
      PIP_INDEX_URL: "https://pypi-cache.internal/simple"   # local wheel cache
    deploy:
      replicas: 4
      resources: { limits: { cpus: "4.0", memory: "8G" } }

# Step 3 — warm the shared venv cache so cold Python starts don't dominate
# /f/ops/warm_pip_cache.script.py — run once per version deploy
# requirements:
#   pandas==2.2.0
#   pyarrow==15.0.0
#   duckdb==0.10.0
def main() -> str:
    """Trigger dep install so subsequent runs hit the shared wheel cache."""
    import pandas, pyarrow, duckdb
    return f"warmed: pandas {pandas.__version__}, pyarrow {pyarrow.__version__}"

# Step 4 — tag heavy Python scripts explicitly
# /f/etl/heavy_pandas_job.script.py
# WORKER_TAG: python-heavy
def main(day: str) -> dict:
    import pandas as pd
    df = pd.read_parquet(f"s3://acme-bronze/events/{day}/*.parquet")
    # ~45s of work
    return {"rows": len(df)}

Step-by-step trace.

Layer	Config	Result
Diagnosis	queue split-by-tag query	ts-interactive p95 180ms; python-heavy pending 123
Root cause	interactive + heavy shared a pool → queue-head blocking	quick jobs waited behind 45s Python jobs
Fix 1	separate pools by tag	interactive pool never sees heavy jobs
Fix 2	warm pip cache	cold starts drop from 8s to 100ms
Fix 3	scale interactive pool	6 replicas absorb burst load
Result	p95 interactive latency	180 ms restored

After the split + cache + scale, interactive TS scripts see their pre-degradation latency; long Python jobs still take 45s each but run on a dedicated pool that doesn't block quick work; the warm pip cache means the first-run tax is paid once per version, not once per invocation.

Output:

Metric	Before	After
Interactive p95 latency	3.0 s	180 ms
Heavy job throughput	3 concurrent	4 concurrent (dedicated pool)
Cold-start Python dep install	8 s per invocation	100 ms (cache hit)
Queue-head blocking on interactive	present	eliminated
Worker cost	3 shared × 4 CPU	6 small (1 CPU) + 4 heavy (4 CPU)

Why this works — concept by concept:

• Queue-head-of-line blocking — mixing quick and slow jobs on the same tag creates a HOL problem: a long job holds a worker slot; quick jobs queue behind it. Splitting by tag gives each workload its own worker-pool budget.
• Warm venv cache via pre-warm script — the first call to a Python script installs its deps, which can take seconds. Pre-running the deps in a warm script populates the wheel cache; subsequent invocations reuse the cached venv in ~100ms.
• Tag routing as the isolation primitive — Windmill's tags are the only "resource class" abstraction. Use them liberally: ts-interactive, python-heavy, gpu, high-memory. Each tag maps to a worker-pool; each worker-pool sizes for its workload.
• Horizontal scaling for interactive — quick jobs benefit from many small workers (concurrency = replica count). Long jobs benefit from few large workers (each gets the CPU and RAM to run). Sizing the pools separately is the win.
• PIP_INDEX_URL to a local wheel cache — a shared Devpi or a Nexus proxy in your network eliminates the "reach out to pypi.org, pay 5s of network round trips" cold-start tax. Every serious Windmill deployment has this.
• Cost — the split doubles the worker count on paper (6+4=10 vs 3 shared) but halves wall-clock latency for the interactive workload. In CPU-seconds terms the change is neutral; in user-observed latency it's a 15x improvement. The eliminated cost is the on-call escalation triggered by the initial 3s p95.

SQL
Topic — sql
SQL queue-analysis and saturation problems

Practice →

Design Topic — design Design problems on worker pools and queue routing

Practice →

---

4. Apps + variables + resources

The low-code app builder atop scripts + typed resources + encrypted variables is what makes Windmill the internal-tool platform, not just a workflow tool

The mental model in one line: Windmill's apps are a low-code drag-and-drop canvas where UI widgets (tables, forms, buttons, charts) bind to script inputs and outputs — combined with variables (encrypted secrets) and resources (typed connection objects like Postgres, S3, HTTP endpoints), all with workspace-scoped RBAC — this turns the same platform that runs your ETL cron into the platform that hosts your GDPR-erasure form, your data-quality dashboard, your on-call runbook launcher, and your customer-support triage UI, without ever leaving one login or one audit trail. The unification is the axis that separates Windmill from Airflow (no apps), Dagster (limited UI), and Retool (proprietary + no orchestration).

Apps — low-code UIs bound to scripts.

• What they are. A canvas of UI widgets (input fields, buttons, tables, charts, maps, custom React components) laid out on a grid. Each widget's data source or action is a Windmill script — either an existing one or an inline snippet.
• The binding model. A button's onClick fires a script; the script's output feeds a table below; a form's submit runs a script with the form's fields as typed input. Everything traceable, everything auditable.
• Publishing. Apps get a public URL (workspace-scoped ACL) — internal teams navigate to /apps/f/support/gdpr_erase and get the UI without any deploy step separate from the script itself.
• What they're not. They are not pixel-perfect customer-facing UIs. Retool wins that battle. They are the fastest way to put a UI on top of an already-written script for internal ops.

Variables — first-class encrypted secrets.

• What they are. Named key-value entries stored in Postgres, encrypted with a workspace master key. Referenced from scripts via wmill.get_variable("u/team/SLACK_HOOK").
• Access control. Workspace RBAC — every variable has can_read / can_write grants. A script can only read variables the executing user has grants for.
• Rotation. Rotate the value in the UI; every script picks up the new value on next invocation (no restart, no config-reload).
• Audit. Every read is logged to the audit table with user_id, variable_path, script_id, timestamp. The audit trail is the compliance story that replaces a Vault sidecar.

Resources — typed connection objects.

• What they are. JSON objects with a schema; each schema is a "resource type" (Postgres, S3, HTTP API, Snowflake, Kafka, custom). Instances are named resources referenceable by path.
• Why typed. The Postgres resource type has fields host, port, user, password, dbname — declared once in the resource type schema. Scripts get autocompletion; the UI generates a matching input form; secrets in the resource are encrypted-at-rest by the same master key as variables.
• Composition. A script asks for a resource by path (u/team/pg_prod); Windmill fetches the decrypted JSON and hands it to the script. No hard-coded credentials; no env-var soup.
• Custom resource types. Define your own YAML schema for internal systems ("our custom auth gateway with a client_id + shared_secret + region"). Once defined, every script and every resource instance benefits from the typed shape.

RBAC + workspace isolation.

• Workspace. Top-level tenant boundary. Users, groups, folders, and resources are all workspace-scoped. Bigger orgs run one workspace per team.
• Folders. Path-based grouping (/f/etl, /f/ops, /f/support). Each folder has explicit ACLs — can_view, can_run, can_edit, can_admin.
• Groups. Named sets of users; assign groups to folder ACLs rather than individual users.
• Publishable. A script can be marked "publishable as webhook" (external POST triggers the script) or "publishable as UI" (browser-visitable form). Both respect the workspace ACL.

Common interview probes on apps / variables / resources.

• "How do apps relate to scripts?" — apps are UI widgets bound to scripts as their data source or action.
• "Where do secrets live?" — variables, encrypted with the workspace master key, RBAC-controlled.
• "What's a typed resource?" — a JSON connection object with a schema; scripts fetch by path.
• "How does Windmill compare to Retool for UI?" — Windmill is functional for internal ops; Retool is polished for customer-facing.

Worked example — an internal GDPR-erasure app bound to a TS script

Detailed explanation. The canonical Windmill app: a support-team form with a user-email field, a "confirm" checkbox, and a submit button. On submit, a TS script runs the erasure; the output panel shows the result. Everything workspace-audited.

• Script. /f/support/gdpr_erase — TS, takes {email, confirmed}, returns {deleted_rows, ticket_url}.
• App. /apps/f/support/gdpr_ui — form with two fields + submit button + result panel.
• RBAC. Only support-ops group has can_run on the script and can_view on the app.

Question. Write the erasure script and describe the app layout that binds to it.

Input.

Component	Value
Script path	f/support/gdpr_erase
App path	apps/f/support/gdpr_ui
Script inputs	email: string, confirmed: boolean
Script output	{deleted_rows: number, ticket_url: string}
RBAC group	support-ops

Code.

// /f/support/gdpr_erase.script.ts
import * as wmill from "windmill-client"

type Input  = { email: string, confirmed: boolean }
type Output = { deleted_rows: number, ticket_url: string, ran_by: string }

export async function main({ email, confirmed }: Input): Promise<Output> {
    if (!confirmed) throw new Error("must confirm the erasure")
    if (!/^\S+@\S+\.\S+$/.test(email)) throw new Error(`invalid email: ${email}`)

    const pg  = await wmill.getResource("u/team/pg_prod")
    const tix = await wmill.getVariable("u/team/TICKET_BASE_URL")
    const me  = await wmill.getVariable("WM_USER")   // built-in — the invoking user

    // ... run the actual erase against Postgres (elided) ...
    const deleted = 42

    return {
        deleted_rows: deleted,
        ticket_url:   `${tix}/gdpr/${encodeURIComponent(email)}`,
        ran_by:       me,
    }
}

# /apps/f/support/gdpr_ui.yaml — app manifest (illustrative)
grid:
  - id: title
    type: text
    text: "GDPR Erasure Runner"
    layout: { x: 0, y: 0, w: 12, h: 1 }
  - id: email
    type: input
    label: "User email"
    layout: { x: 0, y: 1, w: 6, h: 1 }
  - id: confirmed
    type: checkbox
    label: "I confirm the erasure"
    layout: { x: 0, y: 2, w: 6, h: 1 }
  - id: submit
    type: button
    label: "Erase"
    color: red
    onClick:
      type: runScript
      path: f/support/gdpr_erase
      args:
        email:     { component: email,     field: value }
        confirmed: { component: confirmed, field: value }
    layout: { x: 0, y: 3, w: 3, h: 1 }
  - id: result
    type: json_viewer
    source: { component: submit, field: result }
    layout: { x: 0, y: 4, w: 12, h: 4 }

Step-by-step explanation.

1. The gdpr_erase script is a normal Windmill TS script — nothing app-specific about it. The same script can be called from a scheduled flow, from a webhook, from an ad-hoc runner, or (as here) from an app. This is the unification win.
2. The app manifest is a YAML document (in the UI you drag components onto a grid; the YAML is the source of truth). Each widget has an id, a type, a layout position, and — for interactive widgets — an event handler.
3. The submit button's onClick binds to runScript. The script's args are populated from other components' values (args.email = the email input's value). The button's own result field holds the script's return.
4. The result panel (json_viewer) reads from submit.result — so as soon as the script returns, the JSON viewer displays {deleted_rows: 42, ticket_url: "…", ran_by: "alice@corp"}.
5. RBAC: the support-ops folder has can_run on gdpr_erase for the support-ops group and can_view on gdpr_ui for the same group. Non-support-ops users can't navigate to the app or invoke the script — enforced at the API layer, not just the UI.

Output.

Interaction	State
Support agent visits /apps/f/support/gdpr_ui	app renders (or 403 if wrong group)
Types alice@corp, checks confirm, clicks Erase	script runs on a worker
Script returns {deleted_rows: 42, ticket_url: "…", ran_by: "bob@corp"}	JSON panel updates
Audit log row	run(script=gdpr_erase, user=bob, args={email:"alice@corp",…})

Rule of thumb. For any internal ops UI, the pattern is: (1) write the script with typed input/output; (2) build a form app on top; (3) grant the relevant group can_run on the script and can_view on the app; (4) done. The pipeline from "we need a UI" to "shipped" is minutes, not sprints.

Worked example — typed resource + custom resource type

Detailed explanation. Beyond built-in resource types (Postgres, S3, HTTP), Windmill lets you declare custom resource types for internal systems. Walk through defining a resource type for a proprietary auth gateway and instantiating a resource of that type.

• Resource type. custom_auth_gateway — schema with client_id, client_secret, region, token_url.
• Resource instance. u/team/auth_prod — one instance of that type.
• Consumer script. Reads the resource, mints a token, calls an internal API.

Question. Define the resource type schema, create the resource instance, and consume it from a script.

Input.

Object	Value
Resource type	custom_auth_gateway
Instance path	u/team/auth_prod
Schema fields	client_id, client_secret (secret), region, token_url
Consumer script	/f/api/mint_token

Code.

# /resource_types/custom_auth_gateway.yaml — one-time definition
name: custom_auth_gateway
description: "Proprietary auth gateway — mints a bearer token per region."
schema:
  type: object
  required: [client_id, client_secret, region, token_url]
  properties:
    client_id:
      type: string
      description: "OAuth client id"
    client_secret:
      type: string
      format: password           # marks as secret; encrypted at rest
      description: "OAuth client secret"
    region:
      type: string
      enum: [us-east-1, eu-west-1, ap-south-1]
    token_url:
      type: string
      format: uri

// /resources/u/team/auth_prod.json — one instance of the type
{
  "resource_type": "custom_auth_gateway",
  "value": {
    "client_id":     "acme-etl-batch",
    "client_secret": "$var:u/team/AUTH_SECRET",
    "region":        "us-east-1",
    "token_url":     "https://auth.acme.internal/token"
  }
}

# /f/api/mint_token.script.py — consumer
# requirements:
#   requests==2.32.0
import requests
import wmill
from typing import TypedDict

class AuthGw(TypedDict):
    client_id:     str
    client_secret: str
    region:        str
    token_url:     str

def main() -> dict:
    gw: AuthGw = wmill.get_resource("u/team/auth_prod")   # returns decrypted dict
    r = requests.post(gw["token_url"], data={
        "grant_type":    "client_credentials",
        "client_id":     gw["client_id"],
        "client_secret": gw["client_secret"],
        "scope":         f"region:{gw['region']}",
    }, timeout=10)
    r.raise_for_status()
    return {"access_token": r.json()["access_token"], "expires_in": r.json()["expires_in"]}

Step-by-step explanation.

1. The resource type is a schema — a JSON-schema-shaped YAML document. Windmill validates every instance of the type against this schema at write time (client_secret must be a password-format string, region must be one of the enum values).
2. format: password marks the field as secret — Windmill's UI hides the field by default and the underlying storage encrypts it with the workspace master key. Non-secret fields are stored plaintext (for auditability + queryability).
3. The $var:u/team/AUTH_SECRET reference in the resource JSON is a variable reference — Windmill resolves it at read time by fetching the variable's decrypted value. This is how you rotate a secret without touching every resource that references it.
4. The consumer script asks for the resource by path; Windmill returns a decrypted dict with the same shape as the schema. Type-checked at write time, type-checked at read time (via the TypedDict), rotated in one place.
5. Any script anywhere in the workspace can consume the same u/team/auth_prod resource. Change the token_url once, all consumers pick up the new value on next invocation.

Output.

Actor	Sees
Resource-type author	schema saved; new instances of this type get a matching form
Resource-instance author	form with client_id (plaintext), client_secret (password), region (dropdown)
Consumer script	decrypted AuthGw dict at runtime
Audit log	one read_resource(path=u/team/auth_prod, by=<user>, at=<ts>) row
Secret rotation	update u/team/AUTH_SECRET variable; next script invocation gets new value

Rule of thumb. For any internal system your scripts need to talk to, define a custom resource type once. The upfront cost (writing the schema) pays back immediately in typed autocomplete, encrypted secret storage, form-based UI, and single-place rotation. The alternative (env vars + hard-coded URLs) is a slow-burning tech-debt fire.

Worked example — workspace RBAC for a multi-team deployment

Detailed explanation. A common scenario: Windmill hosts scripts for three teams — ETL, Support, and Data Science. Each team should see and run only its own scripts + resources + variables. Walk through the folder + group + ACL setup that enforces isolation.

• Folders. /f/etl (ETL team), /f/support (Support team), /f/ds (Data Science team), /f/shared (utilities all can read).
• Groups. etl-team, support-team, ds-team.
• ACLs. Each team gets can_admin on its own folder, can_view on /f/shared.

Question. Configure the folder + group ACLs to enforce team isolation while allowing shared utility use.

Input.

Folder	Group	Grant
/f/etl	etl-team	can_admin
/f/support	support-team	can_admin
/f/ds	ds-team	can_admin
/f/shared	etl-team, support-team, ds-team	can_view + can_run
/f/shared	platform-team	can_admin

Code.

# /folders/f/etl.yaml
name: etl
display_name: "ETL Team"
owners: [etl-team]
extra_perms:
  "g/etl-team": true      # can_admin (implied by owners)

---
# /folders/f/support.yaml
name: support
display_name: "Support Team"
owners: [support-team]

---
# /folders/f/shared.yaml
name: shared
display_name: "Shared utilities"
owners: [platform-team]
extra_perms:
  "g/etl-team":     read    # can_view + can_run only
  "g/support-team": read
  "g/ds-team":      read

// /f/shared/http_get.script.ts — a shared utility every team can use
import * as wmill from "windmill-client"

type Input  = { url: string, headers?: Record<string,string> }
type Output = { status: number, body: unknown }

export async function main({ url, headers }: Input): Promise<Output> {
    const r = await fetch(url, { headers })
    return { status: r.status, body: await r.json() }
}

# /f/etl/pull_customers.script.py — ETL script uses the shared utility
import wmill

def main() -> dict:
    result = wmill.run_script_sync(
        "f/shared/http_get",
        args={"url": "https://api.acme.com/customers"},
    )
    return {"customers": result["body"]}

Step-by-step explanation.

1. Each team's folder (/f/etl, /f/support, /f/ds) has its team-group as owners, which grants can_admin implicitly. Non-team-members cannot even list scripts inside the folder.
2. The /f/shared folder is owned by the platform-team and grants read (which equals can_view + can_run) to each team's group. Any team can call f/shared/http_get; only the platform team can edit it.
3. The pull_customers script calls the shared utility via wmill.run_script_sync — Windmill checks the invoking user's ACL against the shared script (must have can_run), runs it, returns the result. RBAC applied per-call.
4. Variables and resources under /f/etl are similarly ACL'd — only the ETL team sees them. Support scripts cannot read the ETL Postgres resource; ETL scripts cannot read the support ticket API key. The team boundary is a first-class isolation primitive.
5. This model scales: adding a new team is "create a group, create a folder owned by that group." No custom auth code, no manual permission wiring — the ACL grammar handles it.

Output.

User	Can view /f/etl	Can view /f/shared	Can edit /f/shared
ETL engineer	yes	yes	no
Support engineer	no	yes	no
Data scientist	no	yes	no
Platform admin	yes (root)	yes	yes

Rule of thumb. Model each team as (group + folder + owner-grant) and each shared utility as a platform-team-owned folder with read grants to consuming teams. This is the smallest RBAC vocabulary that supports team isolation without becoming a permissions swamp.

Senior interview question on Windmill apps + variables + resources

A senior interviewer might ask: "You're consolidating three Retool apps and a HashiCorp Vault instance into Windmill for a 40-user support team. The apps need typed inputs, RBAC per team, secret rotation without redeploys, and an audit trail. Design the folder + resource + variable + app layout, and describe how you'd migrate one Retool app end-to-end."

Solution Using folder-scoped RBAC + typed resources + encrypted variables + a script-first app rebuild

# Step 1 — folder + group layout (as before)
# /folders/f/support.yaml
name: support
owners: [support-team]

# /folders/f/support/apps.yaml
name: apps
owners: [support-team]

# /folders/f/support/scripts.yaml
name: scripts
owners: [support-team]

// Step 2 — resources for the Postgres + ticket API
// /resources/u/team/pg_support.json
{
  "resource_type": "postgresql",
  "value": {
    "host":     "pg-support.internal",
    "port":     5432,
    "user":     "wm_support",
    "password": "$var:u/team/PG_SUPPORT_PWD",
    "dbname":   "support"
  }
}

// /resources/u/team/tickets_api.json
{
  "resource_type": "http_bearer",
  "value": {
    "base_url": "https://tickets.acme.internal",
    "token":    "$var:u/team/TICKETS_TOKEN"
  }
}

// Step 3 — the migrated GDPR-erasure script (was: Retool JS query + REST call)
// /f/support/scripts/gdpr_erase.script.ts
import * as wmill from "windmill-client"
import { Client } from "npm:pg"

type Input  = { email: string, confirmed: boolean }
type Output = { deleted_rows: number, ticket_url: string, ran_by: string }

export async function main({ email, confirmed }: Input): Promise<Output> {
    if (!confirmed) throw new Error("must confirm")
    const pg     = await wmill.getResource("u/team/pg_support")
    const tix    = await wmill.getResource("u/team/tickets_api")
    const runBy  = await wmill.getVariable("WM_USER")

    const client = new Client(pg)
    await client.connect()
    let deleted = 0
    try {
        const res = await client.query(
            "DELETE FROM users WHERE email = $1 RETURNING id",
            [email],
        )
        deleted = res.rowCount ?? 0
    } finally {
        await client.end()
    }

    const ticket = await fetch(`${tix.base_url}/gdpr`, {
        method: "POST",
        headers: {
            "Authorization": `Bearer ${tix.token}`,
            "Content-Type": "application/json",
        },
        body: JSON.stringify({ email, deleted, actor: runBy }),
    }).then(r => r.json())

    return {
        deleted_rows: deleted,
        ticket_url:   `${tix.base_url}/tickets/${ticket.id}`,
        ran_by:       runBy,
    }
}

# Step 4 — the app that binds to the script
# /apps/f/support/apps/gdpr_ui.yaml
grid:
  - id: title
    type: text
    text: "## Support · GDPR Erasure"
  - id: email
    type: input
    label: "User email"
  - id: confirmed
    type: checkbox
    label: "I confirm the erasure is authorised"
  - id: submit
    type: button
    label: "Erase"
    color: red
    onClick:
      type: runScript
      path: f/support/scripts/gdpr_erase
      args:
        email:     { component: email,     field: value }
        confirmed: { component: confirmed, field: value }
  - id: result
    type: json_viewer
    source: { component: submit, field: result }

Step-by-step trace.

Layer	Before (Retool + Vault)	After (Windmill)
Runtime	Retool cloud SaaS ($50/user/mo × 40 = $2000/mo)	Windmill self-hosted ($0/user)
Secret storage	Vault sidecar per env	Windmill variable, workspace-encrypted
Script surface	Retool JS + REST + SQL blocks	one TS script gdpr_erase
RBAC	Retool group + Vault ACL (two systems)	one workspace folder ACL
Audit trail	Retool audit + Vault audit (join across)	one Windmill audit table (single-query)
Migration effort per app	—	~1 day per app; 3 apps = ~1 week

After the migration, the support team logs into one URL, navigates to /apps/f/support/apps/gdpr_ui, submits the form, sees the same result — but the ticket API token lives in one Windmill variable (rotatable in the UI), the Postgres password lives in another (RBAC-controlled), and every action leaves one audit row that includes the invoking user, script path, arguments, and result. The Vault + Retool tab in the SaaS bill closes.

Output:

Metric	Before	After
Distinct tools (support workflows)	2 (Retool + Vault)	1 (Windmill)
Monthly SaaS spend	$2000	$0
Secret rotation flow	Vault UI → redeploy Retool	Windmill UI, immediate
Audit-log join	2 systems, custom pipeline	one Postgres query
Onboarding for a new support agent	2 tool logins + roles	1 tool login + group membership

Why this works — concept by concept:

• One binary, one audit, one login — the consolidation win. Every action lives in one system with one auth flow, one RBAC surface, one audit table. Compliance auditors love this; on-call engineers love this; new hires love this.
• Typed resources for connections — pg_support and tickets_api are typed objects the script fetches by path. Rotating either is a one-line UI edit; every consumer picks up the new value on next invocation.
• Variable references inside resources — the $var:u/team/… pattern lets a resource's secret field point to a variable, so secrets can rotate independently of resource metadata. This is the layered-indirection pattern that Vault also uses but for free.
• Folder-scoped RBAC — support-team owns /f/support; only they see and run its contents. No custom permission code; no per-script ACL wiring; the folder is the permission unit.
• Script + app are the same underlying object — the app is a UI on top of the script. Changing the script changes both the app's behaviour and any programmatic caller (webhook, cron, other script). One source of truth.
• Cost — the migration is engineer-days per app (rewrite Retool JS + SQL blocks as a single script; drag components onto the app canvas). The eliminated cost is 40 seats × Retool per-seat pricing plus Vault operational overhead. ROI is measured in weeks. Net one-tool operational surface versus two.

SQL
Topic — sql
SQL RBAC and audit-log query problems

Practice →

Design Topic — design Design problems on internal-tool consolidation

Practice →

---

5. Windmill vs Airflow / Dagster / n8n / Retool + interview signals

The head-to-head — Windmill wins for polyglot self-hosted teams that also want app-building; every other tool wins a narrower axis

The mental model in one line: windmill vs airflow (and Dagster, and n8n, and Retool) is not a "which is better" contest — it is a pick per workload decision where Windmill wins the fused-polyglot-self-hosted-plus-apps quadrant, Airflow wins the pure Python orchestration-at-scale quadrant, Dagster wins the asset-lineage-first quadrant, n8n wins the no-code business-user quadrant, and Retool wins the polished customer-facing UI quadrant — and senior interviewers listen for whether you can frame the tools as orthogonal rather than competing, whether you name OpenFlow / AGPLv3 / Nsjail as the Windmill-specific signals, and whether you refuse to say "Windmill is better than Airflow" when the honest answer is "they serve different quadrants". Every senior data-engineering interview in 2026 probes this axis because it's the proxy for whether you can pick tools for constraints, not for hype.

Windmill's wins.

• Polyglot runtime. TS + Py + Bash + Go + SQL all as first-class script types with hermetic per-script envs. No other tool in this comparison offers all five with the same ergonomics.
• Self-host + AGPL. One binary + one Postgres, AGPLv3 core with commercial license. Genuine vendor independence; forks are copyleft-protected.
• Apps + orchestration on one runtime. The same script backs a schedule, a webhook, and a UI. No cross-tool sync, no drift, no double-defined secrets.
• Postgres as queue. Operational simplicity — one backup strategy, one auth surface, one monitoring pane. Scales to tens of thousands of jobs/day on a single Postgres.

Windmill's losses.

• Orchestration at Airflow scale. At hundreds of thousands of tasks/day with Kubernetes-native isolation, Airflow's operator + KEDA + Kubernetes-executor pattern is more mature. Windmill's Postgres queue is bounded by Postgres row-lock throughput.
• Asset-lineage graphs. Dagster's software-defined assets, partitions, and lineage lens are a first-class capability Windmill does not match.
• No-code SaaS integration. n8n's drag-and-drop node library for HubSpot / Slack / Google Sheets / etc. beats Windmill for business-user-authored workflows.
• Polished customer-facing UI. Retool's UI components and templates are more polished than Windmill's app builder for pixel-perfect external UIs.

Airflow — where it wins.

• Pure Python orchestration at scale. 100k+ tasks/day, Kubernetes-first, mature operator ecosystem (500+ prebuilt operators).
• Community + ecosystem. Largest community by far; every SaaS has an Airflow operator; every hiring manager recognises "Airflow" on a resume.
• Where it loses. Python-only, no apps, heavy operational footprint (scheduler + webserver + workers + queue + DB), UI is DAG viewer only.

Dagster — where it wins.

• Software-defined assets. First-class asset lineage, partitions, materialisation graphs, freshness policies. The "data platform for data engineers" pitch.
• Asset checks + observability. Built-in data-quality primitives; asset lens shows what materialised when.
• Where it loses. Python-only, apps are limited, orchestration mental model is "assets first" which doesn't map to arbitrary polyglot scripts.

n8n — where it wins.

• No-code SaaS integration. 500+ prebuilt nodes (HubSpot, Salesforce, Slack, Google Sheets, Airtable, …). Non-engineer authored workflows.
• Self-host + fair source. Sustainable Use License (not OSI-approved but self-hostable); node ecosystem is under a permissive license.
• Where it loses. JavaScript-only (custom logic), weak on engineer-authored polyglot scripts, weak on typed inputs, weak on RBAC compared to enterprise tools.

Retool — where it wins.

• Polished CRUD UI. Best-in-class UI components, templates, and pixel-perfect layouts. Non-engineer-friendly UI builder.
• Backend-agnostic. Talks to any REST / GraphQL / SQL backend; not tied to a specific runtime.
• Where it loses. Proprietary (closed-source), per-seat SaaS (self-host enterprise tier available), no orchestration primitives (no schedules, no flows, no DAGs), no polyglot script runtime.

The five interview signals.

• Name OpenFlow. The open YAML spec for Windmill flows. Signals awareness of the portability story.
• Name AGPLv3. The license and its copyleft implications (fork or hosted derivative must open-source). Signals awareness of the license landscape.
• Name the Rust + Postgres executor. Not Kafka, not Redis, not Kubernetes-native. Signals architecture depth.
• Name Nsjail. The Linux-namespace sandbox for per-script isolation. Signals security-model literacy.
• Frame as orthogonal. "Windmill wins X quadrant; Airflow wins Y." Signals decision-making maturity.

Worked example — the pick-per-workload matrix

Detailed explanation. The single most useful artifact for a comparison interview is a matrix that maps workload characteristics to platforms. Every senior discussion converges on this matrix within ten minutes. Walk through building it for six representative workloads.

• Workload 1. Python-only ETL, 200k tasks/day, need Kubernetes-native isolation.
• Workload 2. Polyglot scripts (Py + TS + Bash), self-host, apps + orchestration on one runtime.
• Workload 3. Asset-lineage-first data platform, materialisation freshness policies.
• Workload 4. Business ops wiring SaaS-to-SaaS, non-engineer authored.
• Workload 5. Customer-facing polished CRUD dashboard.
• Workload 6. Internal ops UIs on top of already-written scripts.

Question. Build the workload → platform matrix and defend each pick.

Input.

Workload	Polyglot	Scale	Apps needed	Lineage	Author	Deploy
1. Py ETL scale	no	200k/day	no	no	engineers	K8s
2. Consolidated ops	yes	5k/day	yes	no	engineers	self-host
3. Asset platform	no (Py)	50k/day	limited	yes	data eng	self-host
4. Biz SaaS wiring	no (JS)	1k/day	no	no	biz users	self-host
5. Customer CRUD	no	n/a	yes (polished)	no	app team	SaaS ok
6. Internal ops UI	yes	500/day	yes	no	engineers	self-host

Code.

# Decision matrix as a Python function
def pick_platform(polyglot: bool,
                   scale_tasks_per_day: int,
                   needs_apps: bool,
                   needs_lineage: bool,
                   author_is_engineer: bool,
                   needs_polished_ui: bool,
                   self_host: bool) -> str:
    if needs_polished_ui and not self_host:
        return "Retool"
    if not author_is_engineer and polyglot is False:
        return "n8n"
    if needs_lineage:
        return "Dagster"
    if not polyglot and scale_tasks_per_day > 100_000:
        return "Airflow"
    if polyglot and needs_apps:
        return "Windmill"
    if polyglot:
        return "Windmill"
    return "Windmill"    # sensible default for medium-scale


# Six workloads
print(pick_platform(False, 200_000, False, False, True,  False, True))
# → 'Airflow'
print(pick_platform(True,    5_000, True,  False, True,  False, True))
# → 'Windmill'
print(pick_platform(False,  50_000, False, True,  True,  False, True))
# → 'Dagster'
print(pick_platform(False,   1_000, False, False, False, False, True))
# → 'n8n'
print(pick_platform(False,       0, True,  False, True,  True,  False))
# → 'Retool'
print(pick_platform(True,      500, True,  False, True,  False, True))
# → 'Windmill'

Step-by-step explanation.

1. Workload 1 (Python ETL at 200k/day, K8s) → Airflow. Windmill's Postgres queue can be pushed to ~50k jobs/day comfortably; beyond that Airflow's KubernetesExecutor + KEDA scaling is the more proven pattern.
2. Workload 2 (polyglot + apps + self-host, 5k/day) → Windmill. This is the modal Windmill quadrant — nothing else fuses these axes on one runtime.
3. Workload 3 (asset-lineage) → Dagster. Windmill has scripts and flows but no software-defined-asset abstraction; Dagster's asset lens is the differentiator.
4. Workload 4 (business-user SaaS wiring) → n8n. Windmill's script-first ergonomics are wrong for non-engineers; n8n's node library is purpose-built for this.
5. Workload 5 (polished customer CRUD) → Retool. Windmill's app builder is functional but not pixel-perfect; Retool's component library and templates ship UI polish out of the box.
6. Workload 6 (internal ops UIs) → Windmill. Same as workload 2 — polyglot, apps, self-host, engineers.

Output.

Workload	Platform	Key axis
1. Py ETL scale	Airflow	scale + K8s
2. Consolidated ops	Windmill	polyglot + apps + self-host
3. Asset platform	Dagster	asset lineage
4. Biz SaaS wiring	n8n	no-code + node library
5. Customer CRUD	Retool	UI polish
6. Internal ops UI	Windmill	polyglot + apps + self-host

Rule of thumb. Frame every comparison as "which axis does this workload maximise?" and let the axis pick the tool. Never let the tool pick the axis. The senior signal is fluency in both directions — you can defend any of the five tools when the axis calls for it.

Worked example — the AGPL license conversation

Detailed explanation. Every enterprise Windmill adoption hits a legal-review question: "AGPLv3 is copyleft; what does that mean for our internal use?" Senior architects need a crisp answer that distinguishes internal use (fine) from hosted derivatives (must open-source). Walk through the conversation.

• The AGPL trigger. Distributing modified source or offering the software as a network service triggers the source-availability obligation.
• The "internal use" carve-out. Running Windmill behind the corporate firewall for internal employees is not a "network service to third parties" — no propagation.
• The commercial license option. Windmill Labs sells a commercial license that removes the AGPL obligations for organisations that need to embed or resell.

Question. Draft the two-paragraph legal-review answer that unblocks Windmill adoption in an enterprise.

Input.

Scenario	AGPL propagation?	Action needed
Internal use behind firewall	no	none
Fork with local patches, internal use	no	none
Fork with local patches, offered as SaaS to external customers	yes	open-source the fork or buy commercial license
Embed Windmill in a proprietary product sold to customers	yes	buy commercial license
Use unmodified Windmill for internal use	no	none

Code.

Legal-review answer template (2 paragraphs)
===========================================

Paragraph 1 — the internal-use case
  "Windmill is licensed under AGPLv3. We plan to run the unmodified
   OSS distribution behind our corporate firewall for internal
   employee use. AGPLv3's source-availability obligation triggers
   on (a) distribution of modified source code to third parties, or
   (b) offering the software as a network service to third parties.
   Neither applies to our internal-only, unmodified deployment, so
   we incur no source-disclosure obligation."

Paragraph 2 — the escalation paths
  "If we ever (a) modify Windmill's source with proprietary
   extensions, (b) offer Windmill-based services to external
   customers, or (c) embed Windmill in a product we sell, the AGPL
   propagation triggers and we would need to either open-source our
   modifications under AGPLv3 or purchase Windmill Labs' commercial
   license. We do not anticipate any of these scenarios for the
   current internal-consolidation project. If they arise, legal
   review is required before deployment."

Step-by-step explanation.

1. Paragraph 1 addresses the immediate legal concern: does running Windmill trigger any obligation? The answer is no, because internal use behind a firewall is neither distribution nor external network service.
2. Paragraph 2 preempts the follow-up: what if that changes? The three escalation triggers (modification + distribution, external network service, embedding in a sold product) each have a resolution path (open-source or buy commercial).
3. The distinction between "internal network service" (fine) and "external network service" (triggers AGPL) hinges on who the users are, not on the transport. Employees using the internal Windmill via HTTPS behind the VPN are still internal users.
4. Modifying Windmill for internal use is fine — the source-availability obligation only triggers on distribution, not on private modification. Every enterprise patches OSS internally; AGPL does not forbid this.
5. The commercial license from Windmill Labs is the escape hatch for scenarios where AGPL propagation would be disruptive. This is the same model Sentry, MongoDB, and Elastic use — dual-licensing OSS + commercial.

Output.

Deployment scenario	AGPL obligation	Recommended action
Unmodified internal use	none	proceed
Modified internal use	none (private modification)	proceed
Fork + external SaaS	must open-source fork	open-source or buy commercial
Embed in customer product	must open-source integration	buy commercial
Contribute upstream	none (contributor voluntary)	contribute if desired

Rule of thumb. For internal-only deployments AGPL is a non-issue. For anything customer-facing or embedded, the commercial license is the standard escape hatch. Address both cases up front in legal review so the enterprise adoption doesn't stall on ambiguity.

Worked example — the "won't Windmill die if the company goes away?" question

Detailed explanation. A frequent late-stage-interview question: "What happens if Windmill Labs goes bankrupt or gets acquired and killed? Are we stuck?" The senior answer is that OSS + open protocol + community fork = bounded downside. Walk through the risk-mitigation story.

• The OSS core. The AGPLv3 source is public and forever. Any organisation can fork and maintain it.
• The OpenFlow spec. Flows are portable — any compatible runtime can execute them.
• The community. Growing enough (2026) that a fork would find volunteer maintainers.
• The exit ramp. Migrate flows to Airflow / Dagster / a fork — the OpenFlow spec + Postgres data are the migration substrate.

Question. Draft the risk-mitigation section of the adoption proposal for a risk-averse enterprise.

Input.

Risk	Likelihood	Mitigation
Windmill Labs bankruptcy	low	OSS fork; migrate flows
Windmill Labs acquisition + product kill	medium	OSS fork; migrate flows
License change to non-OSS	medium (industry trend)	pin to last AGPL version; fork
Community fork abandonment	low (given growth)	migrate to Airflow/Dagster
Data lock-in	low	OpenFlow spec + Postgres export

Code.

Adoption risk-mitigation template
=================================

Risk 1 — vendor viability
  Windmill Labs is a well-funded startup as of 2026; bankruptcy is a
  low-probability but non-zero risk. Because the core is AGPLv3 and
  the source is public, a community fork is always possible.

Risk 2 — product kill after acquisition
  Larger risk than bankruptcy. Mitigated by (a) OSS core, (b) the
  OpenFlow spec, (c) Postgres as the queue (data is in our DB, not
  a proprietary format).

Risk 3 — license change
  Industry trend (Sentry, Elastic, Redis, MongoDB have all changed
  licenses). Mitigation: version-pin the currently-AGPL release; if
  the next release is non-OSS, evaluate the fork or migration path.

Risk 4 — migration path
  Scripts are just Python / TS / Bash / Go / SQL files — portable
  to any runtime. Flows are OpenFlow YAML — importable into a
  compatible runtime or migratable to Airflow DAGs / Dagster ops
  with a straightforward transformation script.

Risk 5 — data lock-in
  All Windmill state lives in our Postgres. Backup the DB; the state
  is portable. No proprietary metadata store; no vendor callouts.

Step-by-step explanation.

1. The vendor-viability risk is real but bounded — the OSS licence guarantees that even in the worst case (Windmill Labs vanishes), the source stays public and forkable.
2. Product-kill risk after acquisition is the higher-probability scenario (see history of many acquired OSS companies). The same mitigations apply: source is public, data is in our Postgres, flows are OpenFlow-portable.
3. License-change risk is the 2026 industry trend. Every OSS-adopting enterprise now pins to the last known-OSI-compatible version and evaluates each new release for license drift.
4. The migration path — OpenFlow YAML flows + polyglot scripts + Postgres queue data — is more portable than most tools. Compare to a Retool app (proprietary JSON) or a Zapier workflow (proprietary SaaS) — those have zero migration path.
5. Data lock-in is minimal because Postgres owns the state. A pg_dump gives you your entire Windmill state as SQL; you can restore into a fork, into a maintenance mode, or into a migration tool.

Output.

Risk	Bounded by	Escape hatch
Vendor bankruptcy	AGPL source public	fork
Product kill	AGPL + OpenFlow + Postgres data	fork / migrate
License change	version-pin currently-OSS release	pin + evaluate
Community abandonment	migration path exists	migrate to Airflow/Dagster
Data lock-in	Postgres owns state	pg_dump / restore

Rule of thumb. Frame OSS adoption risk as "bounded downside" — worst-case, you fork or migrate; best-case, the tool keeps improving. This is the honest and interview-quality answer for any AGPL / OSS adoption discussion.

Senior interview question on Windmill vs alternatives

A senior interviewer might ask: "Your VP of Engineering wants to consolidate three tools — Airflow, Retool, and Vault — into one platform for a 50-engineer data + tools team. Two candidate platforms are on the table: Windmill and 'roll our own on Airflow + Kubernetes + Backstage + custom UIs.' Compare the total cost, the ergonomic wins and losses, the migration risk, and the two-year outlook. Deliver a recommendation."

Solution Using a scored comparison + explicit migration order + a two-year outlook

COMPARISON — Windmill vs "roll our own on Airflow + K8s + Backstage + custom UIs"
==============================================================================

Axis                       Windmill                Roll-our-own
---                        ---                     ---
Polyglot runtime           TS/Py/Bash/Go/SQL       Py-first + N-lang via K8s
Deployment                 1 binary + Postgres     Airflow (5 procs) + K8s + custom
Apps + internal tools      built-in low-code       must be custom-built
Secret management          built-in (variables)    Vault + custom wiring
RBAC                       workspace + folder      manual per-tool ACL
Audit trail                one table               join across N tools
Onboarding time            ~1 week                 ~1 quarter
Buy-vs-build effort        buy (adopt + configure) build (~4 engineer-quarters)
Ongoing ops burden         low (one binary)        high (5+ services)
Vendor risk                AGPL fork exit          in-house code exit
Two-year TCO (est.)        $60k                    $1.2M (4 eng-quarters + ops)
Ergonomic ceiling          medium (functional)     high (custom-built)
Ecosystem                  growing, ~2026-mature   Airflow-mature + fragmented

# Scored recommendation
def score(platform: str) -> dict:
    if platform == "Windmill":
        return {
            "polyglot": 5, "self_host_oss": 5, "apps": 4, "orchestration": 4,
            "operations": 5, "vendor_risk": 4, "cost_2y": 5, "eng_effort": 5,
        }
    if platform == "RollOurOwn":
        return {
            "polyglot": 4, "self_host_oss": 5, "apps": 5, "orchestration": 5,
            "operations": 2, "vendor_risk": 5, "cost_2y": 1, "eng_effort": 1,
        }
    return {}

# Simple sum
sum_wm = sum(score("Windmill").values())        # 37/40
sum_ro = sum(score("RollOurOwn").values())      # 28/40
print(f"Windmill: {sum_wm}  RollOurOwn: {sum_ro}")
# → Windmill: 37  RollOurOwn: 28

RECOMMENDATION
==============
Adopt Windmill.

Migration order (4 sprints):
  Sprint 1 — deploy Windmill; migrate top-5 Airflow DAGs; validate
             performance; wire Prometheus + PagerDuty.
  Sprint 2 — migrate top-5 Retool apps; migrate variables from Vault;
             onboard 10 engineers; decommission unused Retool seats.
  Sprint 3 — migrate remaining Airflow DAGs + Retool apps; run parallel;
             finalize RBAC folder layout.
  Sprint 4 — decommission Airflow scheduler + Retool + Vault; cut over
             on-call runbooks; run all-hands demo.

Two-year outlook:
  - Year 1: consolidated to one platform; SaaS bill down $50k/year;
            on-call surface down 3 tools → 1.
  - Year 2: evaluate Windmill scale; if approaching 100k jobs/day plan
            a hybrid (Windmill for interactive + apps, Airflow for
            batch scale). Escape hatch: OpenFlow flows + polyglot
            scripts migratable.

Step-by-step trace.

Sprint	Deliverable	Risk mitigation
1	Windmill deployed; 5 DAGs migrated	run in parallel with Airflow
2	5 Retool apps migrated; Vault → variables	Vault stays until all secrets moved
3	Remaining migrations	parallel operation until sign-off
4	Decommission old stack	last-day cutover; keep DB snapshots

After the four sprints, the team runs on Windmill; the Airflow + Retool + Vault stack is decommissioned; the SaaS + operational bill drops; onboarding new engineers takes days instead of weeks. The recommendation is defensible because it's scored, not asserted.

Output:

Metric	Windmill	Roll-our-own
Two-year total cost of ownership	~$60k	~$1.2M
Engineer-quarters to build	~1 (configure)	~4 (build from scratch)
Operational surface	1 binary + Postgres	5+ services + custom UI code
Migration reversibility	pg_dump + OpenFlow flows	in-house code — you own it
Two-year outlook	proven adoption path	proven build risk

Why this works — concept by concept:

• Score the axes, don't argue them — a 4-vs-5 dot on each axis converts a subjective debate into an arithmetic decision. VPs love this; interviewers love this even more.
• Sprint-by-sprint migration order — running Windmill in parallel with the existing stack for 3 sprints lets you validate performance and roll back cheaply. Big-bang cutovers fail; parallel operation succeeds.
• Two-year outlook — every buy-vs-build decision hides a five-year commitment; explicitly naming the year-2 evaluation point ("if we hit 100k jobs/day, add Airflow for batch") is the mature architectural signal.
• Escape hatch clarity — OpenFlow + polyglot scripts + Postgres data means the migration cost out of Windmill is bounded. This is the AGPL + OSS story translated into buy-vs-build language.
• Cost model transparency — $60k for Windmill (adopt + configure) vs $1.2M for roll-our-own (4 engineer-quarters at $300k/quarter loaded cost) is the arithmetic that closes the deal. Every senior recommendation cites numbers, not adjectives.
• Cost — the total cost of the recommendation is the migration effort (4 sprints × 3 engineers = ~$300k loaded) plus the ongoing $0 in per-seat SaaS + one Postgres worth of infra. The eliminated cost is 40 Retool seats × $50/mo × 24 months + 5-service ops surface + Vault operational overhead. Net one-tool operational cost versus three-tool ops surface — the two-year ROI is 4-5x.

SQL
Topic — sql
SQL data-platform selection and comparison problems

Practice →

Design
Topic — design
Design problems on buy-vs-build for workflow platforms

Practice →

---

Cheat sheet — Windmill recipes

• When to pick Windmill. Polyglot self-hosted teams that also want internal-tool building on the same runtime as their orchestrator. TS + Py + Bash + Go + SQL first-class; one binary + Postgres deploy surface; AGPLv3 core + commercial license option. Airflow wins pure Python at 100k+ tasks/day; Dagster wins asset-lineage-first data platforms; n8n wins no-code SaaS wiring for business users; Retool wins polished customer-facing CRUD. Pick per workload; frame the tools as orthogonal.
• The three Windmill primitives. Script = single typed function in TS / Py / Bash / Go / SQL, path-addressed and versioned, auto-form UI from the signature. Flow = DAG of script steps with branches, for-each loops with parallelism, suspend + resume for human-in-the-loop, retry policies per step, and a flow-level failure_module. Module = a script imported by other scripts (convention, not a separate primitive). All three serialise as OpenFlow YAML for portability + Git sync.
• OpenFlow spec. Open YAML schema for scripts, flows, resources, and variables. wmill sync round-trips a workspace to a Git repo; every workspace change is a diff; every PR is a workflow review. The spec is Windmill's portability + governance layer and the differentiator interviewers listen for.
• Executor architecture. Postgres is the queue — one table with state IN ('pending','running','success','failure'), SELECT ... FOR UPDATE SKIP LOCKED for pickup, LISTEN/NOTIFY for wake-up. Rust workers (~50 MB RAM idle) poll the queue by tag. Add worker replicas horizontally; no coordinator. Failure semantics = at-least-once (scripts must be idempotent).
• Isolation modes. Native — script runs in the worker's own process; fast, trusts the script. Nsjail — Linux namespaces + seccomp; each script sees an isolated /tmp, no other scripts' cache, no host network unless granted; ~17ms overhead per invocation; mandatory for workspaces with untrusted script authors.
• Tag routing recipe. Set WORKER_TAGS=default,python,ts,bash on default workers; python-heavy,pandas on beefy python workers; gpu,ml on GPU workers. Scripts declare their tag via # WORKER_TAG: python-heavy frontmatter. Route by workload characteristics, not by rate-limit; each worker is single-slot by default.
• Warm venv + wheel cache. First Python invocation installs deps (5-8 s cold); pre-run a "warmer" script with the same requirements at deploy time to populate the cache. Point PIP_INDEX_URL at an internal Devpi / Nexus for zero-network wheel installs. Cold-start latency drops from 8 s to 100 ms.
• Resources + variables template. Define a resource type (JSON schema — postgresql, s3, http_bearer, or custom). Create instances at u/team/<name> with $var:u/team/<SECRET_NAME> references for password fields. Scripts fetch via wmill.get_resource("u/team/name") and wmill.get_variable("u/team/NAME") — both decrypted at read time. Rotate secrets in the UI; every script picks up new values on next invocation.
• App builder recipe. Layout widgets on a grid (input, checkbox, button, table, json_viewer, chart, custom React). Bind a button's onClick to runScript(path, args); bind a display widget's source to {component: button, field: result}. The script + app share one identity; RBAC is folder-scoped; the audit log is one Postgres table.
• Workspace RBAC recipe. Model each team as (group + folder + owner-grant). Shared utilities live in a platform-team-owned folder with read grants to consuming groups. Publishable-as-webhook and publishable-as-UI both respect the same ACL surface. Never bolt custom permission code on top; the built-in ACL grammar covers the standard cases.
• Operational SLIs. Queue depth = count(*) WHERE state='pending'; worker saturation = count(*) WHERE state='running' / worker_count; worker heartbeat = last_ping > now() - 60s. Wire all three to Prometheus + PagerDuty on day one. Alert on pending > 1000 for 5 min or saturation = 100% for 10 min.
• Migration order (from Airflow + Retool + Vault). Sprint 1 — deploy Windmill in parallel; migrate top-5 DAGs. Sprint 2 — migrate top-5 Retool apps + variables. Sprint 3 — migrate remaining. Sprint 4 — decommission old stack. Run parallel through sprint 3 so rollback is cheap; big-bang cutovers fail.
• AGPL playbook. Internal use behind firewall = no propagation. Modified internal use = no propagation. Fork + external SaaS or embed in sold product = must open-source or buy commercial license. Version-pin the currently-OSI-compatible release; evaluate each upgrade for license drift.
• Vendor-risk mitigation. OSS source is public + forever. OpenFlow flows are portable. Postgres owns all state; pg_dump is your escape hatch. Migration cost out of Windmill is bounded — polyglot scripts run anywhere, OpenFlow flows import into any compatible runtime, worst case you migrate to Airflow/Dagster with a transformation script.
• Debugging + observability. Every run has a job_id; the UI shows step-by-step logs, timing, retries, and the exact args/result. The job and queue Postgres tables are directly queryable — join to worker and resource for full context. Prometheus exporter is a Windmill script; the platform monitors itself.

Frequently asked questions

What is Windmill in one sentence?

windmill is a Rust-native, self-hosted, AGPLv3-licensed platform that fuses a polyglot script runtime (TypeScript on Deno or Bun, Python in per-script venvs, Bash, Go, and SQL), a durable multi-step flow engine (DAGs with branches, for-each loops with parallelism, and suspend-plus-resume for human-in-the-loop), a low-code app builder that binds UI widgets to scripts, and first-class encrypted variables and typed resources into one deployable binary + one Postgres — replacing the "Airflow for orchestration + Retool for internal tools + Vault for secrets" stack that most mid-size data teams accreted between 2020 and 2024. Every operational surface (schedule, webhook, form UI, ad-hoc runner) invokes the same underlying script, every secret is workspace-scoped and audit-logged, and every workflow serialises as OpenFlow YAML for Git diffability. Senior data-engineering interviewers probe Windmill because it's the load-bearing consolidation pattern for teams that have outgrown "one Airflow DAG per problem" but haven't fallen into "we built our own Backstage plugin."

Windmill vs Airflow — when do I pick each?

Default to windmill when your team is polyglot (Python + TypeScript + Bash), needs both orchestration and internal-tool UIs on the same runtime, wants a self-hosted deployment with a small operational surface (one binary + one Postgres), and runs a bounded task volume (comfortably to tens of thousands of jobs/day, workable further with worker fleet scaling). Pick Airflow when the workload is Python-only, needs to scale to hundreds of thousands of tasks/day with Kubernetes-native per-task isolation (KubernetesExecutor + KEDA), or needs the mature 500+-operator ecosystem for SaaS integrations. Airflow wins pure Python orchestration at scale; Windmill wins the fused-polyglot-plus-apps-plus-self-host quadrant. Never frame it as "Windmill is better than Airflow" — the honest senior answer is "they win different quadrants." Many mature teams end up running both — Windmill for interactive + apps + medium-scale orchestration; Airflow for the batch-orchestration workload that has outgrown Windmill's Postgres queue.

What is the OpenFlow spec?

openflow is the open YAML schema Windmill Labs published for describing scripts, flows, resources, and variables — a portability contract that lets any compatible runtime execute a flow the same way. In practice Windmill is currently the reference (and dominant) implementation, but the spec is what makes several downstream stories credible: (a) Git-based flow-as-code — every flow round-trips to a YAML file the team can PR-review, diff, and version; (b) workspace export/import — wmill sync moves flows between workspaces or between organisations; (c) vendor-risk mitigation — if Windmill Labs ever pivots or is acquired-and-killed, an OpenFlow-compatible fork or an alternative runtime is a bounded migration path rather than a rewrite; (d) tooling — linters, type-checkers, security scanners can all target the spec rather than an internal Windmill API. Interviewers listen for whether you name OpenFlow as the portability layer, not just "the YAML format" — it's the signal that separates candidates who read the marketing page from candidates who understand the strategic play.

Is Windmill really self-hostable and truly open source?

Yes. Windmill's core is windmill self-hosted under AGPLv3 — a genuinely open source (OSI-approved) licence with a copyleft trigger. The Docker image is public (ghcr.io/windmill-labs/windmill), the source is on GitHub, and a standard deployment is one binary (server + worker modes) plus one Postgres — no vendor callouts, no proprietary metadata store, no telemetry-required beacons. The AGPL trigger fires on (a) distribution of modified source or (b) offering the software as a network service to third parties. Internal-only use behind a corporate firewall triggers neither, so no source-availability obligation attaches. Modifying Windmill internally is fine. If you ever offer Windmill-based services to external customers, or embed Windmill in a product you sell, the AGPL requires either open-sourcing your modifications under AGPLv3 or purchasing the Windmill Labs commercial licence. This dual-licence model is the same pattern Sentry, MongoDB, and Elastic (pre-license-change) adopted; enterprise legal review is a standard well-worn path.

What runtimes does Windmill support?

Windmill runs TypeScript on Deno (secure-by-default, TypeScript-native, npm: specifiers work) or Bun (faster cold-start, full npm compatibility), Python in per-script venvs (declare pip deps via # requirements: comment; venvs cached per-version), Bash for shell workflows, Go compiled to a tiny binary per script, and SQL as first-class scripts that run against a resource-connected database with typed input parameters. Each runtime is hermetic — a Python script pinning pandas==2.2.0 doesn't conflict with another script pinning pandas==1.5.0; a TS script using npm:pg@8.11.0 doesn't fight another script using npm:pg@8.10.0. The polyglot story is Windmill's headline differentiator over Airflow (Python + Bash-second-class), Dagster (Python), and n8n (JavaScript only). PowerShell, Deno-standard-lib, and various flavours of "run this in Docker" are also supported as script-type extensions. Worker isolation (native vs Nsjail) is orthogonal to runtime — you can Nsjail-sandbox a Python script, a Bash script, or a TS script identically.

How does Windmill compare to Retool for internal tools?

Retool is proprietary, closed-source, per-seat SaaS (with a self-host enterprise tier) that ships best-in-class polished UI components — datagrids, chart libraries, form templates, mobile-first layouts — and connects to any REST / GraphQL / SQL backend via a large connector library. It wins for polished customer-facing or pixel-perfect internal CRUD, and its non-engineer-friendly UI builder is a genuine differentiator. Windmill is Rust-native OSS, AGPLv3, self-hosted, with a functional (not pixel-perfect) low-code app canvas that binds UI widgets to first-class Windmill scripts running on the same platform as your orchestrator. It wins when the internal-tool UI is (a) engineer-authored, (b) built on top of already-written scripts (so no separate backend to maintain), (c) needs to share RBAC + secrets + audit trail with your orchestration surface, and (d) cannot be per-seat SaaS due to compliance or budget. In a mixed enterprise the pattern that emerges is Retool for customer-facing dashboards + Windmill for internal ops UIs on top of scripts — they solve different problems and the honest senior answer is "use both when the axes call for it."

Practice on PipeCode

• Drill the SQL practice library → for the queue-analysis, watermark, and RBAC problems senior interviewers love.
• Rehearse system design against the design practice library → for the workflow-platform selection, worker-pool sizing, and consolidation-migration scenarios.
• Sharpen the streaming axis with the streaming practice library → for the human-in-the-loop, suspend-resume, and event-triggered flow patterns.
• Stack the prerequisites against PipeCode's broader 450+ data-engineering catalogue to anchor the five-tool decision matrix against real graded inputs.

Lock in windmill muscle memory

Docs explain platforms. PipeCode drills explain the decision — when Windmill's polyglot runtime beats Airflow, when Retool's polished UI beats Windmill's app builder, when Dagster's asset lineage is the axis that matters, when Nsjail's sandbox is non-negotiable, when the AGPL license conversation unblocks adoption, when the pick-per-workload matrix earns its place on the whiteboard. Pipecode.ai is Leetcode for Data Engineering — pattern-first practice tuned for the production trade-offs senior data engineers actually face.

Practice SQL problems →
Practice design problems →