DEV Community: Developer Service

I Installed Free-Threaded Python - The GIL Came Back Anyway

Developer Service — Mon, 21 Sep 2026 07:20:32 +0000

Python 3.14 made the free-threaded build officially supported. Nice change log line.

I installed 3.14t the way I install anything I'm excited about, way too fast, and with one bad assumption: that the build was enough.

python -VV said free-threading build. I spun up threads on a CPU-bound job and waited for the cores to light up. Wall time barely moved. Four threads, one core's worth of progress.

Nothing crashed. Nothing warned.

My threads didn't fail, they went back to taking turns. A C extension I'd imported wasn't ready, so the runtime quietly put the GIL back on.

I only knew because I finally asked sys._is_gil_enabled().

That afternoon is why I wrote this down.

Why I cared

For twenty years I lived with the same rule everyone else did. The global interpreter lock means only one thread runs Python byte code at a time. Threads were fine when I was waiting on sockets. Useless when I wanted CPU. Need cores? Pay for processes.

Python 3.13 shipped a free-threaded build as an experiment. Python 3.14 made it officially supported, although still optional, still not the default.

Most people who install 3.14 still get the GIL. I wanted a narrower answer: if I install the other build, what actually gets faster, and what quietly puts the lock back?

So I set up a small lab experiment with uv, a stdlib-only benchmark, and one unmarked C extension.

Setting up the lab

Free-threading isn't a flag on normal 3.14. It's a separate interpreter: python3.14t.

With uv I ask for 3.14t, pin it in the project, and keep requires-python = ">=3.14" in pyproject.toml, the t is a build variant, not a version number, and uv will refuse >=3.14t.

$ uv python install 3.14 3.14t
Installed 2 versions in 2.32s
 + cpython-3.14.5+freethreaded-linux-x86_64-gnu (python3.14t)
 + cpython-3.14.5-linux-x86_64-gnu (python3.14)

$ uv python pin 3.14t
Pinned `.python-version` to `3.14+freethreaded`

When I want the normal GIL build back for a comparison, I ask for it by name. A pinned 3.14t plus an existing .venv will swallow a vague --python 3.14, and I'll think I'm comparing when I'm not.

uv run --no-project --isolated --python 3.14+gil python benchmark.py
uv run --no-project --isolated --python 3.14t python benchmark.py

What I measured

I kept the first benchmark pure Python on purpose: format a row, split it, fold integers.

Same total work, one thread then four.

Libraries that drop the lock from C would fake a win on the default build, and I'd be measuring the wrong thing.

from concurrent.futures import ThreadPoolExecutor
import time

def crunch(n: int) -> int:
    acc = 0
    for i in range(n):
        a, b, c = f"{i},{i * i},{i % 97}".split(",")
        acc += int(a) + int(b) + int(c)
        acc ^= acc << 1
        acc &= 0xFFFFFFFF
    return acc

def wall_time(workers: int, total: int) -> float:
    chunk, rem = divmod(total, workers)
    sizes = [chunk + (1 if i < rem else 0) for i in range(workers)]
    started = time.perf_counter()
    with ThreadPoolExecutor(max_workers=workers) as pool:
        list(pool.map(crunch, sizes))
    return time.perf_counter() - started

The full script in the lab repo prints interpreter metadata, repeats the run, and keeps the best wall time.

Let's now run the benchmark:

$ uv run --no-project --isolated --python 3.14+gil python benchmark.py
Python          3.14.5 (main, May 10 2026, 19:28:16) [Clang 22.1.3 ]
Py_GIL_DISABLED 0
GIL enabled     True
CPU count       4
total rows      2,000,000  (split across threads)

 threads      wall s   speedup
       1       1.210     1.00x
       4       2.133     0.57x

$ uv run --no-project --isolated --python 3.14t python benchmark.py
Python          3.14.5 free-threading build (main, May 10 2026, 19:27:52) [Clang 22.1.3 ]
Py_GIL_DISABLED 1
GIL enabled     False
CPU count       4
total rows      2,000,000  (split across threads)

 threads      wall s   speedup
       1       1.174     1.00x
       4       0.320     3.67x

Default 3.14: four threads were slower than one (0.57×). Contention for the lock, not a free lunch. 3.14t: about 3.7× on four cores.

That speedup is the clean-lab ceiling, not a promise. This job shares nothing: no shared dict, no dataframe, no lock.

Contention and native code that still serializes will pull real work toward 2×. If I see 1.1×, I'm paying a single-thread tax (often a few percent, sometimes closer to 10%) for nothing.

But the reason I still don't trust a fresh 3.14t pin isn't bad scaling. It's quieter than that.

The thing that ate my afternoon

The build can say free-threaded and still be running with the GIL on. python -VV and a compile-time flag only tell me which interpreter I installed. sys._is_gil_enabled() tells me what's happening right now.

import sys
import sysconfig

sysconfig.get_config_var("Py_GIL_DISABLED")  # 1 on a free-threaded build
sys._is_gil_enabled()                        # False — until something flips it

I need both. That's how I burned the afternoon.

Import a C extension that hasn't said it can run without the GIL, and the interpreter turns the lock back on for the rest of the process. It prints a warning. It doesn't raise.

The benchmark keeps running. The threads keep looking busy. Throughput goes back to taking turns.

The detector I use is a loop:

import importlib, sys, warnings

print("before", sys._is_gil_enabled())  # False on a fresh 3.14t

for name in ["numpy", "pandas", "gil_trap"]:
    with warnings.catch_warnings(record=True) as caught:
        warnings.simplefilter("always")
        importlib.import_module(name)
        print(name, sys._is_gil_enabled())
        for w in caught:
            if "gil" in str(w.message).lower():
                print(" ", w.message)

I start free-threaded. Import dependencies one by one. Watch for the flip.

The lab at python-threads-gil-article ships trap/: a tiny C extension that does no work and never declares itself safe. That's enough.

$ uv run --no-project --isolated --python 3.14t --with ./trap python gil_detector.py gil_trap

      Built gil-trap @ file:///.../python-threads-gil-article/trap
Installed 1 package in 0.79ms
Python          3.14.5 free-threading build (main, May 10 2026, 19:27:52) [Clang 22.1.3 ]
Py_GIL_DISABLED 1

before imports               gil_enabled=False

gil_trap                     gil_enabled=True  <-- GIL re-enabled
                             warning: The global interpreter lock (GIL) has been enabled to load module 'gil_trap', which has not declared that it can run safely without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0.

after all imports            gil_enabled=True

PYTHON_GIL=0 after that warning is not a fix. It's a dare. The module told me it isn't safe. I should believe it.

I re-ran the benchmark after importing gil_trap. The 3.7× was gone: back to taking turns, same story as default 3.14.

What about the stack I already run? On this machine, current NumPy, pandas, SciPy, Pydantic, and FastAPI imported clean: lock still off after each.

Having a free-threaded wheel isn't the same as scaling under threads either. Some paths still take their own locks. Async and I/O don't care much: the GIL was never what made await work.

The thing that worried me was the private wheel, the pinned 18-month-old native module, the "works on 3.14" package that only shipped the normal build. I look for the t in the package tag and keep an eye on py-free-threading and free-threaded wheels.

Where I landed

I stay on default 3.14 when the work is mostly waiting: sockets, databases, HTTP. Async already overlaps that. Free-threading won't make FastAPI handlers magically use four cores while they talk to Postgres.

I stay off it for single-threaded CLIs too. A little slower, no parallelism, nothing to show for the pin. And if the hot path is a native library that still serializes, or a wheel that re-enables the GIL, I'd just be paying a tax to keep the lock.

I still reach for processes when isolation matters more than shared memory: crash domains, messy native code.

I reach for 3.14t when the work is CPU-bound Python that already shares memory awkwardly across processes: parallel transforms, image or PDF pipelines, batch work over local files.

And only when I control the dependency set: a greenfield worker or a small internal tool, not the forty-wheel monolith I inherited. Before I trust threads on that build, I run the detector on a cold start, and in CI, after every import that matters.

Free-threading in 3.14 is real. On my lab box, four threads finished that CPU job in about a third of the time. It's also a different interpreter and a dependency veto. One unmarked native module and I'm back to one core, with a warning that looks like noise.

Try the lab on your hottest CPU path. Then tell me what broke.

Follow me on Twitter: https://twitter.com/DevAsService

Follow me on Instagram: https://www.instagram.com/devasservice/

Follow me on TikTok: https://www.tiktok.com/@devasservice

Follow me on YouTube: https://www.youtube.com/@DevAsService

Building an MCP Server for Dokku with FastMCP

Developer Service — Fri, 11 Sep 2026 11:11:37 +0000

You are already in Cursor, mid-debug, when you need to know whether an app is up, what its domains are, or why the last deploy blew up.

The usual path is a context switch: open a terminal, SSH into the Dokku host, run the right plugin commands, paste the output back into chat. By then the thread of the problem has gone cold.

What if you could stay in the conversation instead? Type “is api running, and show me the last fifty log lines” and let the agent hit Dokku for you: same host, same SSH credentials you already trust, without shuttling shell output by hand.

That is what an MCP server is for: tools the model can call while you keep working in the editor.

This post walks through dokku-mcp, a FastMCP server that talks to Dokku over SSH, returns structured results, and ships with guardrails so “convenient” does not become “unrestricted shell access.”

What MCP actually is

The Model Context Protocol is a small contract between an AI client and a side process that owns capabilities the model should not invent on its own: database queries, ticket APIs, or your Dokku host.

In the local setup Cursor and Claude Desktop use most often, the client starts your server as a subprocess and talks to it over stdio (JSON-RPC on stdin/stdout). At connect time the server advertises a catalog of tools: name, description, and a JSON schema for arguments.

When you ask something in chat, the model can choose a tool, the client invokes it on the server, and the result is fed back into the conversation as structured data, not as something you pasted from a terminal.

FastMCP turns Python async functions into that catalog with almost no configuration: decorate or register a function, document side effects in the docstring (clients surface those to the model), and call mcp.run()

The protocol handles discovery and invocation; your job is deciding which operations exist and what they are allowed to do.

Setup: skeleton and SSH

The rest of this post follows the code in dokku-mcp as it ships today. The package lives under src/dokku_mcp/ with a thin layout:

server.py — FastMCP instance and tool registration
ssh.py — shared asyncssh session
config.py — env settings and allow/deny lists
parsers.py — CLI stdout → structured data
tools/ — one module per Dokku concern

Dependencies are the obvious ones: fastmcp, asyncssh, pydantic-settings, python-dotenv.

One Dokku-specific gotcha matters more than the framework choice. Dokku’s SSH user uses a forced command. You do not run dokku apps:list in a remote shell. You SSH as dokku and send the plugin invocation as the remote command:

ssh dokku@your-host apps:list

The connection layer mirrors that. One shared session per process, a lock so concurrent tool calls don’t interleave, and reconnect if the link drops:

async def run(self, *args: str) -> str:
    """Run a Dokku plugin command over SSH.

    Dokku's forced SSH command expects the plugin invocation as the remote
    command (e.g. ``apps:list``), not ``dokku apps:list``.
    """
    if not args:
        raise ValueError("DokkuSSH.run requires at least one argument")
    command = " ".join(_shell_quote(a) for a in args)
    async with self._lock:
        return await self._run_unlocked(command)

Config comes from the environment (DOKKU_HOST, DOKKU_SSH_USER, DOKKU_SSH_KEY_PATH, …). Defaults favor the Dokku convention: user dokku, port 22, mode read-only.

Tool #1: `list_apps`

The first useful tool is also the simplest: list apps, prefer JSON, fall back to quiet text, then filter through the allowlist.

async def list_apps() -> list[str]:
    """List Dokku apps on the host.

    Side effects: none (read-only).
    Respects DOKKU_APP_ALLOWLIST / DOKKU_APP_DENYLIST — denied apps are filtered out.
    """
    settings = get_settings()
    try:
        raw = await run_dokku("apps:list", "--format", "json")
    except DokkuSSHError:
        raw = await run_dokku("--quiet", "apps:list")
    apps = parse_apps_list(raw)
    return [app for app in apps if settings.is_app_allowed(app)]

from fastmcp import FastMCP
from dokku_mcp.tools import apps

mcp = FastMCP("dokku")
mcp.tool(apps.list_apps)

def main() -> None:
    mcp.run()

That is enough for the server process. Wire Cursor (or Claude Desktop) as in the the Example section below, the client config is the same whether you expose one tool or the full set.

Expand the tool set (and face the parsing problem)

Read-only coverage grows the same way: one Dokku command, one tool, one parser.

Tool	Dokku command	Notes
`app_report`	`ps:report <app>`	Process / running state
`app_config`	`config:export <app>`	Secrets masked by default
`app_logs`	`logs <app> -n <n>`	Default 50 lines, hard max 500
`app_domains`	`domains:report <app>`	VHOST report
`app_url`	derived	HTTPS URLs from domains

Dokku isn’t a JSON API. Newer plugins often accept --format json; older output is banner text and Key: value lines. The server prefers JSON when it works, then falls back to text parsers, never a generic “run whatever string the model invents” tool.

That last point is deliberate. An open-ended dokku_exec would be convenient for demos and disastrous in production. Structured tools keep the surface small, the docstrings honest about side effects, and the parsers testable against fixture files of real CLI output.

def parse_apps_list(raw: str) -> list[str]:
    """Parse ``apps:list`` JSON or quiet text output."""
    text = raw.strip()
    if not text:
        return []
    if text.startswith("["):
        data = json.loads(text)
        if isinstance(data, list):
            return [str(item) for item in data]
        raise ValueError("apps:list JSON was not a list")
    apps: list[str] = []
    for line in text.splitlines():
        line = line.strip()
        if not line or line.startswith("=====") or line.lower() == "my apps":
            continue
        apps.append(line)
    return apps

Same pattern for reports, config:export, and logs: small functions, fixture-backed unit tests, no scraping surprises at runtime.

Guardrails: don’t blindly trust AI with infra

Giving a model SSH into your PaaS is only interesting if you assume it will eventually ask for something you don’t want. Guardrails are that protection.

Mode. DOKKU_MCP_MODE defaults to read-only. Mutating tools call require_write_mode() and fail closed unless you set read-write.

Allowlist / denylist. Empty allowlist means all apps; a non-empty list is an explicit permit set. Denylist always wins. Every app-scoped tool checks before SSH.

Secret masking. app_config returns env vars with DATABASE_URL, *_TOKEN, *_PASSWORD, and friends replaced by *** unless the caller passes reveal_secrets=True. The model can reason about which keys exist without leaking credentials into the chat transcript by default.

Confirm for mutations. Even in read-write, restart and scale refuse to run unless confirm=True is passed explicitly, never inferred from chatty assent:

async def restart_app(app: str, confirm: bool = False) -> dict[str, str]:
    """Restart all processes for a Dokku app.

    Side effects: restarts the application (downtime possible).
    Requires DOKKU_MCP_MODE=read-write and confirm=True.
    """
    settings = get_settings()
    settings.require_write_mode()
    settings.require_app_allowed(app)
    if not confirm:
        raise PermissionError(
            "restart_app requires confirm=True to proceed — pass confirm=True explicitly"
        )
    output = await run_dokku("ps:restart", app)
    return {"app": app, "status": "restarted", "output": output.strip()}

set_config follows the same pattern and returns the keys that were set, not the values. create_app validates names (^[a-z0-9-]+$ with alphanumeric edges) and still requires write mode.

What I deliberately left out: destroy_app. Destroying an app is irreversible enough that “the model said yes” is not a controlled safety.

Tool docstrings spell out side effects because MCP clients surface them to the model. That is part of the safety features, not just decoration.

Example: from chat to Dokku

Assume the package is installed (or run from the repo with uv sync). Before wiring Cursor, confirm SSH works the Dokku way, the same forced-command path the server will use:

ssh dokku@your-host apps:list

If that fails, fix the key, host, or Dokku user first; the MCP layer will not rescue a broken SSH setup.

Then add the server to ~/.cursor/mcp.json (or the project .cursor/mcp.json). From the repo with uv:

{
  "mcpServers": {
    "dokku": {
      "command": "uv",
      "args": [
        "run",
        "--directory",
        "~/dokku-mcp-server",
        "dokku-mcp"
      ],
      "env": {
        "DOKKU_HOST": "xxx.xxx.xxx.xxx",
        "DOKKU_SSH_USER": "dokku",
        "DOKKU_SSH_KEY_PATH": "~/.ssh/id_rsa",
        "DOKKU_MCP_MODE": "read-only"
      }
    }
  }
}

Once published, uvx dokku-mcp with the same env block is enough.

Reload MCP in Cursor, then stay in the agent chat.

If dokku does not appear under MCP tools, open the MCP panel and check the server logs: bad host, key path, or a failed uv start show up there before any chat error does.

" width="481" height="674">

Prompts that map cleanly to tools:

You type	Tools the model should use
“What apps are on my Dokku host?”	`list_apps`
“Is `api` running, and what’s its public URL?”	`app_report`, `app_url`
“Show the last 50 log lines for `api`.”	`app_logs`
“Which env keys does `api` have? Don’t show secrets.”	`app_config` (secrets stay masked)

Example of “What apps are on my Dokku host?”:

" width="697" height="575">

You never open a terminal for those checks. SSH still happens, but inside the MCP server, on a shared session, with mode and allowlists applied.

When you intentionally enable writes (DOKKU_MCP_MODE=read-write), a prompt like “restart api” only succeeds if the tool call includes confirm=true. If the model omits it, the server refuses. That is the difference between a convenience wrapper and a controlled safety you would actually leave connected.

Conclusion

dokku-mcp is a small, opinionated MCP server: FastMCP on the outside, Dokku over SSH on the inside, dedicated parsers instead of a free-form shell, and defaults that keep day-to-day use read-only.

Issues and PRs are welcome, especially fixture captures from real Dokku versions and extra read-only reports that fit the “one command, one tool, one parser” rule.

If you already run Dokku at home or in a small shop, this is a practical way to keep Dokku in the Cursor chat instead of a separate SSH session.

Follow me on Twitter: https://twitter.com/DevAsService

Follow me on Instagram: https://www.instagram.com/devasservice/

Follow me on TikTok: https://www.tiktok.com/@devasservice

Follow me on YouTube: https://www.youtube.com/@DevAsService

Your Voice Pipeline Is Too Flexible

Developer Service — Thu, 03 Sep 2026 08:11:29 +0000

Voice does not change what a tool needs. It changes what arrives first: audio, then a string, then a candidate payload that might still be wrong.

In Your Data Structure Is Too Flexible, a string crossed an HTTP boundary where the ledger expected an int. The stack trace blamed infrastructure. The bug was the handler that accepted the body.

Voice puts another door in front of that one. A reservation tool still wants a contract. Speech-to-text gives you a transcript. That transcript is not a tool argument any more than a raw JSON dict is a charge request.

The rule: Strict at the edges. A transcript is untrusted input. Only validated JSON crosses into the tool.

This piece follows one booking from mic to side effect: voice → transcribe → extract → validate → tool.

The runnable code lives in this repository.

Staging lied about the mic

Every voice demo uses some version of this utterance:

“Book me a table for four tomorrow at seven, outdoor if you have it.”

Whisper returns a readable transcript. You paste it into a prompt that says “return JSON”.

The model fills party_size, date, time, and seating. The booking lands. The pipeline looks finished.

Your reservation tool never wanted readable text. It wanted:

party_size: int
date: date
time: time
seating: Literal["indoor", "outdoor"] | None

If the extractor invents guests: "four", drops the date, or maps “patio” into seating, the booking is wrong even when every word in the transcript was transcribed correctly.

Staging did not lie about speech-to-text. It lied about the contract. The take was clean, the speaker was you, and nobody asked what happens when speech is partial, ambiguous, or wrong.

Production gets a different speaker. They trail off before the date. They say “Friday” with no week in context. They correct themselves mid-sentence: “actually make that eight, not four”. The model fills a required slot with a guess, coerces “a few people” into 3, or passes a free-text blob into a typed argument.

Those are not Word Error Rate (WER) failures. Whisper heard the words. The payload still does not match the tool. The bug is the same family as the flexible dict: untrusted input crossed the door, and nothing checked the shape.

Transcription quality is a UX problem. The boundary that protects the tool is the one that refuses the call until the payload validates.

Don’t ask one prompt to do two jobs

The staging lie often hides in a single instruction: “Transcribe this audio and return JSON for the booking tool.”

It works on a clean take. Under real speech it breaks in ways you cannot debug. The user corrects a time mid-utterance; the model has to choose between the transcript text and the fields it already filled. A required slot is missing; the model invents one to satisfy JSON rather than fail closed.

When party_size comes back wrong, you cannot tell whether Whisper misheard “four” or the extractor guessed. There is no stable artifact to re-run extraction against, and no clean place to validate before the tool call.

Keep a handoff.

Speech-to-text produces a Transcript: text, plus optional confidence and timestamps.

The tool never sees audio or raw STT output.

If extraction fails, retry or clarify against the same transcript. If STT fails, fix recognition. Do not ask the schema to guess from noise.

from pydantic import BaseModel, Field


class Transcript(BaseModel):
    """STT artifact. Text only — not a tool argument."""

    text: str
    confidence: float | None = Field(default=None, ge=0.0, le=1.0)
    timestamps: list[tuple[float, float, str]] | None = None

WER measures recognition accuracy. It does not measure whether the payload matches the tool.

The candidate payload still needs an explicit contract: which fields, which types, what is required.

That contract is the tool schema, the same Pydantic model extraction targets and validation checks against.

Name what each stage owns

Follow one utterance through five stages. If a design review cannot name what each stage owns, the boundary will leak, and the leak will look like “the model is bad at voice”.

Voice owns capture and audio quality. It does not own meaning or fields.

Transcribe owns speech-to-text. It produces a Transcript. WER belongs here.

Extract owns turning that string into a candidate payload shaped like the tool contract, usually with an LLM and PydanticAI. It may guess. It must not be the last gate before the tool.

Validate owns the schema: types, required fields, enums, constraints. On failure it rejects or returns structured errors. It does not “improve” the transcript or invent missing slots to be helpful.

Tool owns side effects: booking, query, write. It receives only validated data, never a raw transcript, an untyped dict, or best-effort JSON from the model.

In the repository, that map is wired in src/voice_structured/pipeline.py. Capture and transcribe produce a Transcript. Extraction and validation produce a ReservationRequest. The tool receives only what passed both gates.

The invariant from the earlier article, applied twice: once at transcript → payload, once at payload → tool. Fail at the door with a message about the contract. Do not debug a wrong booking three layers down.

The contract is the spine

Once transcription and extraction are separate, the tool schema does the same job ChargeRequest did for inbound JSON.

Write it as a Pydantic model: field names, types, required vs optional, and the constraints the tool will enforce anyway.

Extraction targets that type. Validation checks against it. The tool re-validates before any side effect.

from datetime import date, time
from typing import Literal

from pydantic import BaseModel, Field


class ReservationRequest(BaseModel):
    party_size: int = Field(ge=1, le=20)
    date: date
    time: time
    seating: Literal["indoor", "outdoor"] | None = None

PydanticAI puts that contract on the agent. The transcript is the input.

The agent’s job is to produce a ReservationRequest, not free-form JSON.

Relative dates such as “tomorrow” resolve against today in the system prompt, still extraction, not STT.

from datetime import date

from pydantic_ai import Agent

from voice_structured.schema import ReservationRequest


def build_reservation_agent() -> Agent[None, ReservationRequest]:
    today = date.today().isoformat()
    return Agent(
        "openai:gpt-4o-mini",
        output_type=ReservationRequest,
        retries={"output": 2},
        system_prompt=(
            "Extract reservation fields from the transcript. "
            "Do not invent values for missing slots. "
            f"Today's date is {today}. "
            "Resolve relative dates such as 'tomorrow' against that. "
            "party_size must be an integer. "
            "seating is indoor, outdoor, or omit if unknown."
        ),
    )

The agent proposes a payload. The schema accepts or rejects it inside run().

Output retries feed structured validation errors back to the model, not a brand-new free-form prompt.

The side-effect tool receives validated JSON only:

from voice_structured.schema import ReservationRequest


async def book_table(payload: dict) -> dict:
    """Side-effect tool. Receives validated JSON only — never a raw transcript."""

    ReservationRequest.model_validate(payload)
    return {"status": "booked", "reservation": payload}

handle_transcript in pipeline.py is the orchestration point.

Empty text refuses. STT confidence below 0.5 clarifies and skips extraction.

A valid ReservationRequest is the only path that calls book_table.

When the output-retry budget is exhausted, PydanticAI raises UnexpectedModelBehavior, catch it and clarify. Do not call the tool.

async def handle_transcript(
    transcript: str | Transcript,
    *,
    agent: Agent[None, ReservationRequest] | None = None,
) -> PipelineResult:
    artifact = (
        transcript if isinstance(transcript, Transcript) else Transcript(text=transcript)
    )
    extractor = agent or build_reservation_agent()

    if not artifact.text.strip():
        return await clarify_or_refuse(artifact)
    if artifact.confidence is not None and artifact.confidence < LOW_CONFIDENCE:
        return await clarify_or_refuse(artifact)

    try:
        result = await extractor.run(artifact.text)
    except UnexpectedModelBehavior:
        return await clarify_or_refuse(artifact)

    payload: ReservationRequest = result.output
    await book_table(payload.model_dump())
    return PipelineResult(
        outcome="booked",
        message="Reservation booked.",
        payload=payload,
    )

The CLI --transcript flag skips the mic so you can test extract → validate → tool as text. LOW_CONFIDENCE lives in transcribe.py.

Tune it per domain and STT vendor; 0.5 is a starting point for whole-utterance scores.

The repository wires the model from OPENAI_API_KEY / VOICE_STRUCTURED_MODEL; the snippet above is the spine.

When speech is wrong

The clean demo utterance is not the story. The story is what happens when the user says “book a table for four” and stops.

Validation failure is normal with voice. The useful question is which kind you have, and whether you retry, clarify, or refuse.

Partial: a required slot was never spoken. “Book a table for four” with no date.

Ambiguous: the transcript has a value the schema cannot resolve. “Friday” with no week. “Around seven” with no time.

Wrong: extraction or coercion produces an invalid payload. party_size: "four". seating: "patio" outside the enum.

Each maps to a different response at the boundary.

Retry extraction inside Agent(retries=…) when the speech might be mappable but the first pass guessed or dropped a field. PydanticAI feeds structured validation errors back to the LLM.

Clarify when a required slot is genuinely missing or ambiguous. Ask for the date, the time, or seating. Keep the partial payload out of the tool until the user fills the gap. Voice UX usually tolerates one visible clarify round better than three silent retries. Cap the attempts and fail where the user can see it.

Refuse when the request cannot be satisfied safely: empty audio, repeated validation failure, nonsense, or a slot the tool cannot represent. Do not call the tool with a best-effort fill.

build_reservation_agent() sets retries={"output": 2}. When that budget dies, handle_transcript catches UnexpectedModelBehavior and routes to clarify_or_refuse.

Low STT confidence never reaches the extractor. Empty speech refuses immediately.

LOW_CONFIDENCE = 0.5


async def clarify_or_refuse(transcript: Transcript) -> PipelineResult:
    text = transcript.text.strip()
    if not text:
        return PipelineResult(
            outcome="refuse",
            message="No speech to extract. Refusing — the tool will not be called.",
        )
    if transcript.confidence is not None and transcript.confidence < LOW_CONFIDENCE:
        return PipelineResult(
            outcome="clarify",
            message=(
                "STT confidence is too low to coerce into typed fields. "
                "Please repeat the date, time, party size, and seating."
            ),
        )
    return PipelineResult(
        outcome="clarify",
        message=(
            "Could not fill the reservation schema from this transcript. "
            "Please give a date, a time, and party size "
            "(and indoor or outdoor if you care)."
        ),
    )

handle_transcript never calls book_table on those paths.

Low STT confidence on an utterance is a clarify signal, not a reason to coerce into a typed field.

Log validation failures at the schema boundary: field paths (date, party_size), attempt count, and outcome. src/voice_structured/observability.py has a minimal log_boundary hook.

That is how you debug a bad booking in production instead of chasing a stack trace inside the reservation service.

Try it

Install from the repository: pip install -e ".[dev]", copy .env.example to .env, set OPENAI_API_KEY. Full setup is in the README. The --transcript flag skips capture.

Happy path: full utterance, validated booking:

python -m voice_structured --transcript "Book me a table for four tomorrow at seven, outdoor if you have it."

{
  "outcome": "booked",
  "message": "Reservation booked."
}
{
  "party_size": 4,
  "date": "2026-09-02",
  "time": "19:00:00",
  "seating": "outdoor"
}

Dates resolve against the day you run the command. The important part is outcome: booked with a validated ReservationRequest, the same object book_table receives.

Partial speech: output retries exhaust, the pipeline clarifies, and book_table is not called:

python -m voice_structured --transcript "Book a table for four."

{
  "outcome": "clarify",
  "message": "Could not fill the reservation schema from this transcript. Please give a date, a time, and party size (and indoor or outdoor if you care)."
}

No second JSON block means the tool was not called. That is the designed path for "Book a table for four." and similar partial utterances. Live models may still invent missing slots despite the system prompt. Treat that as prompt tuning, not a reason to drop the schema gate.

Low STT confidence: extraction never runs:

python -m voice_structured --transcript "Book me a table for four tomorrow at seven." --confidence 0.2

{
  "outcome": "clarify",
  "message": "STT confidence is too low to coerce into typed fields. Please repeat the date, time, party size, and seating."
}

Again, no reservation payload.

What this isn't

This is not an argument against Whisper, or against asking a model to extract fields. Extraction is how you get from a string to a candidate. The point is that extraction is not the door.

It is also not a voice-UX handbook. One clarify round, a confidence threshold, and a refuse path are enough to prove the design rule. They are not a dialogue manager.

Skip the conclusion that every voice app needs this exact five-stage graph. Dicts, transcripts, and JSON blobs are fine as owned data after a boundary has already checked the shape.

The problem is treating untrusted speech as if it were already a tool argument.

The rule, twice

Voice adds a door. It does not replace the one you already needed.

Map the pipeline so each stage owns one transformation. Keep transcription and extraction separate so you can re-run and debug. Define the tool contract in Pydantic and extract into it with PydanticAI.

When speech is partial, ambiguous, or invalid, retry inside the agent, clarify with the user, or refuse.

Never call the tool with a transcript, an untyped dict, or a payload that failed validation.

Validated JSON at the tool boundary, every time.

For HTTP and config boundaries, see Your Data Structure Is Too Flexible. For voice, add one more door, and keep the same rule: validate, then move on.

Follow me on Twitter: https://twitter.com/DevAsService

Follow me on Instagram: https://www.instagram.com/devasservice/

Follow me on TikTok: https://www.tiktok.com/@devasservice

Follow me on YouTube: https://www.youtube.com/@DevAsService

Photo by Namroud Gorguis / Unsplash

Your Data Structure Is Too Flexible

Developer Service — Mon, 24 Aug 2026 13:19:07 +0000

Python makes it easy to pass data around as dicts, nested JSON blobs, and “whatever the client sent”.

That flexibility is useful when you own the shape. It gets expensive when the data crosses a trust boundary - an HTTP body, a webhook, a file from another team - and you treat it like it already matches your assumptions.

You may already know Pydantic. This article is not a tour of fields and validators.

It is about a design rule that is easy to skip when the happy path works: validate at the boundary, then move on.

Keep flexibility only where you chose it.

Staging lied (inbound API JSON)

Imagine a small HTTP handler that accepts a JSON body as a Python dict and charges a user’s account. ledger below is a stand-in for whatever typed client or service call sits deeper in your stack.

In staging, every client you control sends the same clean shape, so the happy path is the only path you ever see.

def charge_user(payload: dict) -> None:
    user_id = payload["user"]["id"]
    amount = payload["amount"]
    # Somewhere deeper in the stack — expects an int user_id
    ledger.debit(user_id=user_id, amount=amount)

Staging payload:

{
    "user": {"id": 42, "email": "dev@example.com"},
    "amount": 19.99,
}

That works. user.id is an int, amount is a float, and ledger.debit never complains.

Then production gets a request from a mobile client, a partner integration, or a retry queue that re-serialized the body. The JSON is still “valid”.

The shape is almost right:

{
    "user": {"id": "42", "email": "buyer@example.com"},
    "amount": 19.99,
}

Nothing fails in the handler. It forwards user_id as whatever JSON gave it.

The failure shows up later - in ledger.debit, a SQL parameter binder, or a comparison that assumes an int. If that deeper call type-checks or binds user_id as int, a str raises there.

Worse: the wrong type slips into storage and you notice even later.

The stack trace points at infrastructure code. The bug was at the door: user.id crossed the trust boundary as a string, and a flexible dict let it through.

Staging did not lie about the feature. It lied about the contract.

The cost of flexible-by-default

A dict is patient. It will carry a string where you meant an int, a missing nest where you meant a required object, or a None where you meant a value.

And it will do it without complaining at the boundary.

Type annotations on the parameter help your editor and your teammates. They do not stop the wrong shape from entering the process.

“Almost the right shape” is worse than obviously wrong data. Staging taught everyone what the payload usually looks like.

Optional fields, defaulted nests, and defensive .get() calls then paper over the cases that don’t match. The contract becomes tribal knowledge instead of something the code can reject.

The consequence arrives later. You debug three layers down while the handler that accepted the body already returned.

Catching bad data at the door costs a validation error. Catching it downstream costs time, a misleading stack trace, and confidence in every other flexible boundary you still have.

The rule: strict at trust boundaries

ONE THING: Strict at edges; flexibility only where you chose it.

A trust boundary is any place data enters your process from something you do not fully control.

For this article, that means inbound API JSON. The same idea applies to config files, environment variables, and messages from another service - not shown here.

Inside the app, after the boundary has done its job, you can pass richer objects around, reshape data, or keep temporary dicts for a transform. That flexibility is earned. It is not the default for untrusted input.

If you already use Pydantic and it still feels like boilerplate before the “real” code, flip the framing.

The model is the contract.

The handler that runs after validation is the easy part, because the hard part already happened at the door.

One refactor

Same payload, different door. Instead of accepting a naked dict, define the contract once and validate before anything else runs.

from pydantic import BaseModel, ValidationError


class User(BaseModel):
    id: int
    email: str


class ChargeRequest(BaseModel):
    user: User
    amount: float


def charge_user(payload: dict) -> None:
    request = ChargeRequest.model_validate(payload)
    ledger.debit(user_id=request.user.id, amount=request.amount)

Feed it the production body where user.id was "42". In Pydantic v2’s default (lax) mode, that string is coerced to int at the boundary.

Strict mode would reject it instead - either way, the decision happens at the door. For this example: lax turns "42" into 42; a value like "not-an-id" fails in both modes:

try:
    ChargeRequest.model_validate(
        {
            "user": {"id": "not-an-id", "email": "buyer@example.com"},
            "amount": 19.99,
        }
    )
except ValidationError as e:
    print(e)

You get a ValidationError immediately, pointing at user.id, before ledger.debit is called. Trimmed, the message looks like this:

1 validation error for ChargeRequest
user.id
  Input should be a valid integer, unable to parse string as an integer
  [type=int_parsing, input_value='not-an-id', input_type=str]

By the time ledger.debit runs, request.user.id is already an int - the conversion happened at the boundary, not three layers down in a binder that was never meant to diagnose your API contract.

That is the whole point: fail at the trust boundary, with a message about the field that broke the contract. In an HTTP API, map that to a 400 or 422 with the validation details, not a 500 from deeper in the stack.

If you use FastAPI, the framework can wire the same models to the request body for you. The design rule does not change, only who calls model_validate.

What this isn't

Skip the conclusion that you should never use dicts. Dicts are fine for owned data, temporary transforms, and code that runs after a boundary has already checked the shape. The problem is treating untrusted JSON as if it were already trusted.

This article is also not a complete Pydantic course. You saw one nested model, model_validate, and a ValidationError at the door. That is enough to prove the design rule. It is not enough to cover custom validators, settings, serialization, or performance.

TypedDict and dataclasses are not the enemy either. Those tools document and structure data in useful ways. They do not, by themselves, give you the same runtime contract check at a trust boundary. That check is the point of this article.

Go deeper

If that design rule makes sense and you want the longer path - custom validators, nested models, settings, APIs, and how Pydantic compares to the alternatives - then read Practical Pydantic.

It is written for Python developers, FastAPI users, and anyone tired of bad data surviving staging and failing in production.

Keep the rule either way: validate at the trust boundary, then move on. Stay flexible only where you chose it.

Follow me on Twitter: https://twitter.com/DevAsService

Follow me on Instagram: https://www.instagram.com/devasservice/

Follow me on TikTok: https://www.tiktok.com/@devasservice

Follow me on YouTube: https://www.youtube.com/@DevAsService

Photo by Shubham Dhage / Unsplash

The Watchdog Pattern - From ESP32 to Python

Developer Service — Mon, 03 Aug 2026 07:48:44 +0000

The sensor node had been running for eleven days. The status LED blinked on schedule. ping returned in under a millisecond. But the last reading in the database was from Tuesday. The MQTT client had stopped publishing, and the main loop was stuck waiting on a mutex that would never be released. The device had wedged.

The fix: pull power, plug it back in. If you have spent time with embedded hardware on a bench or in the field, you have done this.

Name the problem before reaching for the solution. The device did not crash. It entered a state where it could not make progress, and nothing outside the firmware had authority to change that. Silent failure. No recovery path.

This is not an ESP32 problem. A Python worker that blocks on a dead database connection and never times out shows the same shape: the process exists, the supervisor thinks everything is fine, and the work stops. The MCU case is useful because the failure is physical, you can see the LED lie to you.

Something external must detect that failure and force recovery. On a microcontroller, that mechanism has a name you already know. What is the equivalent when your code runs on a server instead of a chip?

Inside the ESP32 Watchdog

The ESP32 watchdog is a hardware timer with a simple contract. You configure a timeout of say, five seconds. The timer counts down. Your firmware must reset it before the countdown reaches zero. If it does not, the hardware assumes the software has failed and resets the chip. No debugger required. No operator with a screwdriver.

The ESP32 has two watchdogs, and they watch different things. The Task Watchdog Timer (TWDT) monitors FreeRTOS tasks. If a task stops running blocked on a mutex, stuck in an infinite loop, or starved by a higher-priority task, the TWDT fires. The Interrupt Watchdog Timer (IWDT) monitors interrupt service routines. If an ISR runs too long or never returns, the IWDT fires. In MicroPython, machine.WDT wraps the task-level watchdog, you configure a timeout and call feed() to reset the timer. ISR hangs are still caught by the IWDT underneath.

Calling wdt.feed() is not a fix. It is proof that your code reached a known-good point and is still making progress. On a wedged device, the main loop never gets there, it spins on a mutex. The watchdog never gets fed. The countdown reaches zero. The chip resets.

That is the contract: something outside the running code must detect failure and force recovery. The watchdog is hardware enforcement of a rule your firmware cannot enforce on itself. A wedged task cannot un-wedge itself.

This contract is not unique to firmware.

Hope Is Not a Strategy

Hope is assuming your code keeps making progress because it has not crashed yet. The firmware is still running. The process still exists. The status light still blinks. Nothing has reported a failure, so you assume everything is fine.

That assumption fails the same way on a chip and on a server. Firmware can block on I/O, lose a task, or spin in a loop while looking healthy from the outside. A Python service can hang on a dead dependency while the supervisor still reports up. In both cases the system appears alive, and nothing external has authority to disagree. You are back to the power-cycle strategy, waiting for a human to notice.

Hope is not supervision. Supervision assumes failure will happen and plans for something outside the code to detect it and respond. Hope assumes failure will not happen, or that you will notice in time to fix it yourself.

Firmware cannot un-wedge itself. A hung worker cannot restart itself. The next section shows what external supervision looks like in code.

Build It: ESP32 and Python

The pattern is the same in both cases: monitor progress, enforce a timeout, recover when the timeout fires. Here are minimal pieces: one on the chip, the rest on the server.

On the ESP32:

from machine import WDT

wdt = WDT(timeout=5000)

while True:
    read_sensor()
    publish_reading()
    wdt.feed()

    # Uncomment to simulate a hang — the chip resets in ~5 seconds:
    # while True: pass

Start the watchdog. Do work. Feed it at the end of each loop iteration. If read_sensor() blocks forever or the hang line runs, wdt.feed() never runs. The hardware timer expires and the chip resets.

On a server:

import subprocess

TIMEOUT = 30

while True:
    proc = subprocess.Popen(["python", "worker.py"])
    try:
        proc.wait(timeout=TIMEOUT)
    except subprocess.TimeoutExpired:
        proc.kill()
        proc.wait()
        continue  # hung — kill and restart
    if proc.returncode != 0:
        continue  # bad exit — restart
    break  # clean exit — stop supervising

The outer loop is the supervisor. It starts the worker, waits up to 30 seconds, and kills it if the worker hangs. A worker that exits with a non-zero return code gets restarted too; a clean exit ends supervision.

The loop above shows the shape; in production you would probably use systemd:

In production (systemd):

[Unit]
Description=Sensor worker

[Service]
ExecStart=/usr/bin/python /opt/app/worker.py
Restart=on-failure
WatchdogSec=30
Type=notify

Restart=on-failure handles exits. WatchdogSec= handles hangs, but only if the worker sends WATCHDOG=1 to systemd via sd_notify on a schedule. No heartbeat, no hang detection.

Install systemd-python via pip, or use your distro's python3-systemd package:

import systemd.daemon

while True:
    do_work()
    systemd.daemon.notify("WATCHDOG=1")

For HTTP services (FastAPI):

@app.get("/health")
async def health():
    await db.execute("SELECT 1")  # prove the dependency is alive, not just the process
    return {"status": "ok"}

For Celery workers:

app.conf.task_time_limit = 30  # seconds — kill hung tasks

Inside the worker (asyncio):

import asyncio

async def main():
    while True:
        reading = await asyncio.wait_for(sensor.read(), timeout=5.0)
        await publish(reading)

asyncio.run(main())

asyncio.wait_for is an in-process timeout, it refuses to wait forever on a blocked call. It does not restart the process; it raises TimeoutError so your code can log, retry, or exit and let the supervisor restart.

External supervision and internal timeouts stack: the worker handles blocked calls, the supervisor handles a wedged worker.

Step	ESP32	Python
Hang timeout	`WDT(timeout=5000)`	`WatchdogSec=30` (production); subprocess loop shows the same shape
In-process timeout	N/A	`asyncio.wait_for(...)`
Timeout trigger	hardware countdown	systemd timer or supervisor kill
Recovery	chip reset	systemd restart or `proc.kill()` + restart

This is deliberately minimal. No logging, exponential backoff, or alerting. Those matter, but they are not the contract. The contract is that something outside the worker enforces recovery.

Working recovery is not the same as good recovery.

When Recovery Becomes the Problem

A watchdog that resets on schedule is not a healthy device. It is a device that fails on schedule.

The wedged mutex from the opening would have triggered a reset and then wedged again on the next boot, assuming the bug survived the restart. The ESP32 comes back, runs the same initialization, hits the same mutex, and stops feeding the watchdog again. Reset. Boot. Wedge. Reset. You have traded a silent failure for a noisy one, and the field log shows a device that reboots every five seconds without ever publishing a reading.

The same loop appears in Python. A systemd unit with Restart=always and a worker that crashes on startup will enter a crash loop, systemctl reports active (restarting), but no work completes. Celery eventually replaces the worker process, but if the database is still down, the new worker blocks in the same place. The supervisor did its job. The root cause did not move.

Supervision without observability makes this worse. If you do not log why recovery fired, the reset cause, a backtrace, or a metric, then you only know the device came back. You do not know why it reset.

A watchdog that fires too often becomes noise, operators tune it out. One that fires too late means damage is already done: a motor left running, a valve stuck open, a buffer overwritten before the timeout expired.

Recovery is necessary. It is not sufficient. You still need to fix the mutex, fix the connection string, add the timeout that prevents the hang in the first place. The watchdog buys you uptime. It does not buy you correctness.

One Pattern, Two Domains

Every supervision design answers three questions. What are you monitoring? What triggers recovery? What happens after the reset?

Question	ESP32 (MicroPython)	Python (production)
What to monitor?	Main loop progress: did `wdt.feed()` run on schedule?	Process health: `/health` endpoint, internal heartbeat (`sd_notify` / `WatchdogSec=`)
What triggers recovery?	Hardware timer expires without a feed	Supervisor timeout, failed health check, hung task limit
What happens after reset?	Chip reboots; check `machine.reset_cause()`, log to flash	Process restarts; capture exit code, log backtrace, alert if crash loop

Apply it to the wedged sensor node. A five-second WDT would have caught the hang, no operator, no power cycle. But recovery alone would have produced a reboot loop until someone fixed the mutex bug and added logging to read the reset cause on boot. The checklist covers detection and recovery. Your code still has to stop causing it.

If you build embedded systems, you already live this pattern. You feed watchdogs, read reset reasons, tune timeouts around your slowest legitimate operation. Python production is the same lesson in different clothes: external supervision, enforced timeouts, observable recovery. The platform changes. The contract does not.

Follow me on Twitter: https://twitter.com/DevAsService

Follow me on Instagram: https://www.instagram.com/devasservice/

Follow me on TikTok: https://www.tiktok.com/@devasservice

Follow me on YouTube: https://www.youtube.com/@DevAsService

Photo by HAYOUNG CHO / Unsplash

Your Next Breach Won't Be a Hacker. It'll Be an Agent

Developer Service — Mon, 27 Jul 2026 07:55:51 +0000

I opened a terminal, pointed an AI agent at a freshly deployed API, and gave it one instruction: "Find a way to access another user's data. You have no credentials and no documentation".

No exploit code. No CVE list. No hints. Just a goal, a target, and three tools: an HTTP client, a JWT signer, and a way to record confirmed findings. Ninety-one seconds later, it had done exactly what I asked.

This wasn't a red team engagement. There was no scoping call, no signed statement of work, no week of reconnaissance. Just a prompt and a target; and by the time I finished my coffee, the agent had access it was never supposed to have.

The Setup: A Real API, A Sandboxed Blast Radius

I didn't point the agent at a known security-training app. Juice Shop and crAPI are great for learning, but they invite an easy dismissal: "of course it found bugs, that app is built to have them". So I built something duller on purpose, a small backend that looks like the SaaS product you already run: users, accounts, billing, and orders.

Under the hood it's a FastAPI service backed by Postgres, probed by a PydanticAI agent running Claude Sonnet 4.6. It has no exposed OpenAPI or Swagger documentation (docs_url=None, redoc_url=None, openapi_url=None) so the agent had to discover the API's shape by probing it, the same way an attacker would approach a production system with no public docs.

The authorization bug looks like ordinary SaaS code. Authenticate the caller, load the row by ID, return it; and never ask whether that row belongs to them:

@router.get("/{user_id}", response_model=UserOut)
def read_user(
    user_id: int,
    current_user: CurrentUser = Depends(get_current_user),
    db: Session = Depends(get_db),
):
    user = db.query(User).filter(User.id == user_id).first()
    if user is None:
        raise HTTPException(status_code=404, detail="User not found")
    return user

There is no check that user_id == current_user.id. Any authenticated caller who guesses an ID gets the full row; including, via UserOut, the bcrypt password_hash.

The agent side is equally spare: one system prompt, Claude Sonnet 4.6 via PydanticAI, and three tools; an HTTP client locked to the target host, a JWT signer, and a finding reporter. The full prompt and tool implementations are in the demo repo.

I planted three common API flaws up front: Broken Object Level Authorization / BOLA (OWASP API1:2023), Broken Authentication (API2:2023), and Excessive Data Exposure (API3:2023). No zero-days, no custom exploit chains; just the mistakes real API teams make, in a system that otherwise looks unremarkable.

All of this ran inside an isolated Docker Compose network with no route to the public internet. Nothing here touched a production system, a third party, or a real person's data; every user, email, and card number in this API is fake, generated for the occasion.

Turning the Agent Loose

I didn't give the agent a playbook. I gave it a role, a goal, and those three tools, nothing else. Which endpoints exist, how authentication works, where the authorization checks are missing, that it had to work out on its own.

The first thing it did was reconnaissance, not exploitation. It fired parallel requests at /, /api, and /health. Only /health returned 200. Then it started guessing: /users, /auth, /login, /register; all 404 until it hit /auth/register and got a 422 back telling it the body needed email, name, and password. No documentation. Just an error message worth reading.

"Excellent! Found working endpoints! Registration needs email, name, and password. Login needs email and password. Let me register users and explore."

It registered a free account (user ID 11), then did something a scanner wouldn't think to do: it changed the user ID in the URL. With its own token, it requested /users/10 and got back a full profile including a bcrypt password_hash.

"Found a critical IDOR! The attacker (user 11) can access /users/10 (the victim's data) including their password_hash, email, and name. But let me dig deeper — there may be more data to access."

IDOR here is the same class of flaw as BOLA, broken object-level authorization. That last line matters more than the label. The agent didn't stop at the first win. It kept probing /users/1, /orders, /orders/1, until it had enough evidence to report.

The whole run took ninety-one seconds. The interesting part isn't the headline ("it found a BOLA"). It's that nobody told it to look for BOLA, or to try ID enumeration, or to register first. It reasoned its way there from a health check and a validation error.

What It Found

Let's see the actual traces that the agent found:

GET /users/10
Authorization: Bearer [token for user 11]

→ 200 {"id":10,"email":"victim@test.com","name":"Victim User",
       "password_hash":"$2b$12$slGb...[redacted]", ...}

No ownership check. Valid token; wrong user ID in the URL.

It then walked lower IDs. GET /users/1 returned a seeded account that predated the run, Alex Rivera (victim@example.com), the same flaw against a customer-shaped row from seed data, not the account the agent had just registered.

GET /orders/1
Authorization: Bearer [token for user 11]

→ 200 {"id":1,"item_description":"Annual plan upgrade", ...,
       "account":{"plan":"enterprise","card_brand":"amex",
       "card_last4":"5994","billing_address":"925 Reid Lake Suite 635, ...",
       "internal_notes":"[redacted]"}}

User 11's token. Order 1 belongs to user 1. Same missing ownership check, compounded by excessive data exposure: billing address, card brand, last four, plan, and internal notes, all fields no legitimate client needs, returned to whoever guesses the ID.

If you're a CTO, that isn't a bug ticket. Under GDPR or CCPA, unauthorized access to another customer's profile and financial PII is the kind of incident your legal team gets asked about. Iterate IDs 1, 2, 3… and you're harvesting every customer in the database, one request at a time.

I'd also planted a broken-authentication flaw: a JWT signing secret leaked through verbose error handling (API2). This agent run didn't find it. Two out of three in ninety-one seconds, with no hints about where to look. That's a very good scorecard.

Any authenticated user could read any other customer's profile, email, billing address, card details, and internal account notes by changing a number in the URL. No zero-day. No stolen credentials. Ninety-one seconds.

Why This Changes the Economics of Attack

Regulatory risk is the headline here. The cost curve is why this happens more often than your annual pentest can keep up with.

A skilled human pentester finding what this agent found is not just plausible, but expected; after all, that's their job.

This run took ninety-one seconds of wall-clock time from agent start to reported findings, and cost roughly fifty cents in Anthropic API usage for that transcript. A senior consultant doing the same work, blind recon, registering a test account, walking object IDs, documenting two critical findings with proof; is often a half-day minimum, and at boutique AppSec day rates we've all seen in the $2,000–$5,000 range, you're not comparing cents to cents. You're comparing a coffee break to a line item on a purchase order.

The cost of finding this in production is a breach notification. The cost of finding it in staging is fifty cents and ninety-one seconds.

And the agent doesn't get tired, doesn't bill hourly, and doesn't need a scoping call. Point it at your staging environment on every deploy, scoped hosts, read-only constraints, no destructive actions, and an NDA if someone outside your team is running it. Run ten of them in parallel against ten API versions. The marginal cost of the next test approaches zero.

That math doesn't just favor defenders who adopt this first. It favors attackers too. The same PydanticAI-and-Claude setup that ran against my demo API is available to anyone with an API key and a target URL. The capability that took ninety-one seconds in my sandbox doesn't require a security team, a budget approval, or a signed SOW. It requires patience and persistence, two things an autonomous agent has in unlimited supply.

What To Actually Do About It

This isn't a call to rip out your stack or hire an AI security team overnight. It's a call to change your testing cadence before someone else's agent changes it for you.

Put AI-agent testing in CI, on every meaningful deploy. That is the main move. Agentic red-teaming means pointing an autonomous AI agent at your API with a goal and a written scope, no scripted attack playbook, and letting it probe the way an attacker would.

Picture the pipeline step: a deploy to staging completes, the agent gets the base URL and scope, it runs for a few minutes, and any confirmed finding opens as a ticket, same cadence as your test suite, a different class of failure. A once-a-year pentest was already too slow for how fast APIs ship; against an attacker who can spin up an agent in minutes, you need something running between engagements.

Who owns it: security sets the goal and the rules; platform or DevOps wires it into CI. If those aren't two teams at your company, it's still two hats; don't leave it as an unspoken "someone should".

A written scope can be short. Something like: Target only https://staging.example.com. Register and authenticate if the API allows it. Do not call production, do not mutate or delete data, do not leave the staging host. Goal: find a way to read another user's data; report only with reproducible request/response proof.

You don't need a vendor to start Monday:

Audit every GET /…/{id} (and the POST/PATCH equivalents) for an ownership check before the row goes out and strip response fields your UI never shows.
Clone the demo agent, point it at staging under a scope like the one above, and treat the first confirmed finding as a real ticket.

The rest is hygiene that makes that pipeline worth running: BOLA and excessive data exposure show up constantly in API assessments, including this one, and they're usually cheap to fix once you look. Budget for continuous testing the way you budget for monitoring and backups.

If you wouldn't bet your customer data on "nobody will think to enumerate IDs", don't bet it on "nobody has an API key".

Bottom Line

I ran this exact experiment against a custom demo API in ninety-one seconds of wall-clock time. It cost roughly fifty cents in model API fees for that run. It found two critical authorization flaws that would have triggered a breach notification if this had been production.

Your API is not a training app. But the agent doesn't know the difference and neither does an attacker.

Book a free AI-agent security assessment →

I point the same technique at your staging or pre-production environment, safely, under NDA, with scoped hosts and no destructive actions and a full transcript of what the agent tried and what it found. You get a prioritized findings report. You get proof, not a slide deck.

Want the full agent transcript and demo code? See the experiment on GitHub →.

Follow me on Twitter: https://twitter.com/DevAsService

Follow me on Instagram: https://www.instagram.com/devasservice/

Follow me on TikTok: https://www.tiktok.com/@devasservice

Follow me on YouTube: https://www.youtube.com/@DevAsService

Photo by Kevin Horvat / Unsplash

Building a Weather Home Screen on a Reflective LCD (MicroPython, ESP32-S3)

Developer Service — Wed, 15 Jul 2026 08:39:39 +0000

You know the moment. The board arrives. You flash MicroPython. You run print("hello") and the serial console cheers. Then try the display, the whole reason you bought this thing, and… nothing. Or worse: a white rectangle that flickers once and dies.

I've been there more times than I'd like to admit. The Waveshare ESP32-S3 4.2" Reflective LCD is a gorgeous panel for a weather station: sunlight-readable, low power, no backlight glow at 2 a.m. But it's also a niche combo, ESP32-S3 plus an ST7305 controller on SPI, and the ecosystem doesn't hand you an import display and walk away.

This post is a weekend walkthrough to get the ST7305 driver working, connect WiFi, pull live weather from Open-Meteo, and have something useful on the display by Sunday.

All the source code is in this repository, flat Python files, no framework. Copy them to the board, add your WiFi credentials, reset.

Just want it working? Quiet Forecast is ready-to-flash multi-screen firmware for this board. Everything below is the free weekend walk-through if you'd rather build it yourself.

Friday night: pixels

Clone the repo, you don't type st7305.py from scratch unless you want to. The sections below explain how the driver works;

Why this board

Reflective LCDs are not TFTs. Everything is monochrome, so design for contrast and big type, not gradients. That constraint is a feature for a wall or desk display: readable in daylight, pleasant at night, low power.

Most tutorials for this kit stop at Arduino examples. There is no official MicroPython driver for the ST7305 on this board.

Building `st7305.py`

So st7305.py is a port. The init sequence, register values, delays, SPI command/data toggling, comes straight from the vendor BSP (Board Support Package). The pixel format does not.

Reflective panels pack pixels in a peculiar layout: four vertical rows are interleaved in each byte column. Getting (x, y) wrong doesn't produce a shifted image; it produces static. The core function is _rlcd_pixel(), which maps logical coordinates into that buffer layout, with Y inverted to match the panel's scanning direction:

def _rlcd_pixel(buf, h4, height, width, x, y, color):
    inv_y = height - 1 - y
    block_y = inv_y >> 2
    local_y = inv_y & 3
    byte_x = x >> 1
    local_x = x & 1
    idx = byte_x * h4 + block_y
    mask = 1 << (7 - ((local_y << 1) | local_x))
    if color:
        buf[idx] = buf[idx] | mask
    else:
        buf[idx] = buf[idx] & ~mask

On top of that buffer we implement the primitives every UI needs:

fill, pixel, hline, rect, circle
text via framebuf 8×8 glyphs, scaled to 8 / 16 / 24 / 32 px heights
refresh(), which push the full buffer over SPI in chunks (_SPI_CHUNK = 4092) to avoid allocation spikes

display.py wraps this in a small Display class: margins, header/footer heights, and content_area() so pages never hard-code like y = 48. The ST7305 import is deferred inside _create_panel() to keep boot-time RAM lower:

@staticmethod
def _create_panel():
    spi = SPI(pins.LCD_SPI_ID, baudrate=1_000_000, ...)
    from st7305 import ST7305  # defer heavy import
    return ST7305(spi, cs, dc, rst)

def content_area(self):
    y = self.MARGIN + self.HEADER_H + self.CONTENT_GAP
    bottom = self.HEIGHT - self.MARGIN - self.FOOTER_H - self.CONTENT_GAP
    return self.MARGIN, y, self.WIDTH - 2 * self.MARGIN, bottom - y

If you make it through Friday with a filled rectangle and readable text on the display, you've cleared the hardest hurdle.

Saturday morning: structure

Pixels work. Now make the code survivable.

Big firmware projects tend to grow into services/, drivers/, pages/, schedulers, event buses. That's the right shape when you have twelve screens and three I2C sensors.

For a first weekend with a new board, it's overkill.

This project uses a flat layout with just enough separation to read top-to-bottom:

boot.py          → create /data/cache on import
main.py          → entry point, WiFi, fetch loop, build state dict
home.py          → home screen layout (draw only)
chrome.py        → header + footer
wifi.py          → STA connection
weather.py       → Open-Meteo current conditions
wmo.py           → WMO weather code labels
http_client.py   → HTTP client (not network.py — clashes with built-in)
config.py        → load config.json
cache.py         → offline weather cache
clock.py         → NTP sync + date/time labels
battery.py       → ADC battery level
weather_icons.py → WMO weather bitmaps
display.py       → SPI setup + drawing facade
st7305.py        → panel driver (the hard part)
pins.py          → board pin map
text_sizes.py    → font size tokens
sync_device.py   → host-side deploy script (not copied to the board)

The design rules are simple:

main.py owns the loop - connect WiFi, fetch weather, when to redraw.
home.py only draws - it receives a plain dict and never touches SPI or HTTP.
display.py hides hardware - pages call disp.text(), not spi.write().
weather.py / wifi.py fetch - network code never imports drawing modules.

Data flows in one direction: config → fetch → build_state() → render_home() → refresh().

The home page layout

The home screen answers one question: what's it like outside right now?

┌─────────────────────────────────────────┐
│  Header: title · time · wifi · battery  │
├─────────────────────────────────────────┤
│                                         │
│         Home page content               │
│      (icon + location + readings)       │
│                                         │
├─────────────────────────────────────────┤
│  Footer: page · date · last updated     │
└─────────────────────────────────────────┘

Layout is a two-column block centered in the content area:

Left	Right
Weather icon (32×32, scaled 4×)	Location name
	Large temperature
	Humidity %
	Condition label
	Feels-like (optional)
	UV index (optional)

home.py computes the vertical stack height from whichever fields are present, centers the block between header and footer, and truncates long location names with ....

Weather icons use WMO codes from the API. Temperature is drawn as digits, a small filled circle for the degree sign, then C. The built-in 8×8 font has no ° glyph. Whole numbers look like 23; one decimal otherwise (22.5). Missing data renders as --.

def render_home(disp, state):
    disp.fill(0)
    draw_header(disp, "Home", state.get("time", "--:--"),
                state.get("battery"), state.get("wifi"))

    cx, cy, cw, ch = disp.content_area()
    # ... layout math: icon_x, right_x, y ...
    draw_weather_icon(disp, state.get("weather_code", 3), icon_x, icon_y, scale=4)
    _col_text(disp, state.get("location", ""), right_x, right_w, y, disp.SIZE_MEDIUM)
    _col_temp(disp, state.get("temperature"), right_x, right_w, y, disp.SIZE_XL)
    # ... humidity, condition, feels-like, UV ...

    draw_footer(disp, state.get("page_index", 0), state.get("page_count", 1),
                state.get("date_label", ""), state.get("updated_label", ""))

A dict in, pixels out. chrome.py draws the header (title, clock, WiFi/battery icons, divider) and footer (page counter, date, "Xm ago" label). icons.py supplies segment-style WiFi and battery glyphs plus _blit_mono() for weather icons.

Saturday afternoon: live data

Networking

Config

Copy config.example.json to config.json on the device:

{
  "wifi": { "ssid": "", "password": "" },
  "location": {
    "latitude": 52.0907,
    "longitude": 5.1214,
    "name": "Utrecht"
  },
  "utc_offset_hours": 2,
  "intervals": { "weather_minutes": 10 }
}

Fill in your WiFi credentials before copying to the device.

config.py merges this with defaults. Location name is display-only; lat/lon drive the API URL.

WiFi

wifi.py brings up STA mode (station mode, the board joins your home network as a client, not as an access point), connects with backoff on failure, and exposes status() as {"connected": bool, "level": 1–4} from RSSI, exactly what the header icons expect:

def status(self):
    if not self.is_connected():
        return {"connected": False, "level": 0}
    level = 3
    rssi = self.wlan.status("rssi")
    if rssi >= -55:
        level = 4
    elif rssi >= -65:
        level = 3
    elif rssi >= -75:
        level = 2
    else:
        level = 1
    return {"connected": True, "level": level}

Weather

weather.py calls Open-Meteo with only the current= fields the home page needs:

temperature_2m, relative_humidity_2m, apparent_temperature, weather_code, uv_index

The WMO weather_code maps to a human label via wmo.py and to a bitmap via weather_icons.py. Successful fetches are saved to /data/cache/weather_current.json so a failed refetch still shows the last good reading.

def _url(self):
    lat, lon = self._loc()
    params = (
        "latitude={}&longitude={}"
        "&current=temperature_2m,relative_humidity_2m,apparent_temperature,"
        "weather_code,uv_index"
        "&timezone=auto"
    ).format(lat, lon)
    return _BASE + "?" + params

def fetch(self):
    raw = http_client.get_json(self._url())
    self._current = self._parse(raw)
    cache.save("weather_current", self._current)

http_client.py handles the HTTP side, retries, gc.collect(), HTTPS→HTTP fallback, and always closes the response:

def _attempt_get(url, timeout):
    resp = requests.get(url, timeout=timeout)
    try:
        if resp.status_code != 200:
            raise OSError("HTTP " + str(resp.status_code))
        return resp.json()
    finally:
        resp.close()  # urequests leaks RAM if you skip this

Main loop

main.py does the following on startup:

Load config
Init display
Connect WiFi
NTP sync (if online)
Fetch weather
Enter loop: refetch every 10 minutes, redraw every minute (clock + battery), reconnect WiFi if dropped

build_state() assembles the dict home.py expects, no global variables, no context object:

def build_state(config, clock, wifi, weather, battery):
    cur = weather.current()
    loc = config.get("location", {})
    bat = battery.status()
    return {
        "location": loc.get("name", ""),
        "temperature": cur.get("temperature"),
        "humidity": cur.get("humidity"),
        "feels_like": cur.get("feels_like"),
        "uv_index": cur.get("uv_index"),
        "weather_code": cur.get("weather_code", 3),
        "condition": cur.get("condition", "Waiting"),
        "time": clock.format_time(),
        "wifi": wifi.status(),
        "battery": bat,
        "page_index": 0,
        "page_count": 1,
        "date_label": clock.format_date(),
        "updated_label": weather.updated_label(),
    }

The main loop only redraws when something changed, weather fetch, a new minute (t[4] from clock.now_local()), or reconnect. So you're not hammering SPI every second:

while True:
    now = time.time()
    t = clock.now_local()

    if not wifi.is_connected():
        wifi.connect()

    if wifi.is_connected() and now - last_weather >= interval:
        if weather.fetch():
            dirty = True
        last_weather = now

    if t[4] != last_minute:      # t[4] = minute from clock.now_local()
        last_minute = t[4]
        battery.poll()
        dirty = True

    if dirty:
        render_home(disp, build_state(config, clock, wifi, weather, battery))
        disp.refresh()
        dirty = False

    time.sleep(1)

Supporting modules

weather_icons.py - 32×32 bitmaps per WMO code. draw_weather_icon(disp, code, x, y, scale=4).

battery.py - Reads GPIO4 ADC (3× divider per Waveshare docs) and maps voltage to a percent estimate.

clock.py - ntptime sync, format_time() for the header, format_date() like "Mon 6 Jul" for the footer.

cache.py - Tiny JSON store with fetched_at timestamps for offline display and the footer age label.

Sunday: ship it

Flash MicroPython for ESP32-S3 on the Waveshare board. I use Thonny (Run → Install MicroPython, pick the ESP32-S3 port) or esptool.py from the command line.
Install mpremote on your PC: python -m pip install mpremote.
Copy config.example.json → config.json on the device and set your WiFi + coordinates.
Deploy from the repo folder:

python sync_device.py --port /dev/ttyUSB0

Use COMx on Windows (with x being your port number). After a fresh MicroPython flash, run once with --force. Add --with-config to push the example config to config.json.

sync_device.py uploads every root .py except itself, then hard-resets the board. boot.py creates /data/cache before main.py runs.

Manual copy works too, just copy every .py except sync_device.py to the device root with Thonny or mpremote.

Hardware reset (not IDE soft reboot). I mean it, a soft reboot from Thonny often leaves SPI peripherals in a weird state.
Serial console should show WiFi OK, NTP synced, Weather updated.
Panel shows live conditions; clock ticks each minute; footer shows cache age.

If WiFi fails, you'll still see cached data from a previous session or placeholders (Waiting, No data) on first boot.

What it looks like

Mine, running in Utrecht:

If you get it running, share a photo of your screen in the comments showing weather where you are. I'd love to see these panels out in the wild.

What you built and what comes next

You've got a flat MicroPython stack, one home screen, and live weather on the panel. All built from modules that don't need a rewrite when you add pages.

This repo stops at one page, but the split is deliberate: home.py and chrome.py stay put when you add forecast pages, indoor sensors, or more screens. For the full multi-screen build, see Quiet Forecast.

Start with pixels. Then WiFi. Then one API call. Mount it when the panel shows your actual weather.

Happy making.

Follow me on Twitter: https://twitter.com/DevAsService

Follow me on Instagram: https://www.instagram.com/devasservice/

Follow me on TikTok: https://www.tiktok.com/@devasservice

Follow me on YouTube: https://www.youtube.com/@DevAsService

Tracking YouTube Video Watch Progress with Django

Developer Service — Fri, 03 Jul 2026 10:57:53 +0000

Your Django app serves a YouTube video. The page loads, the player appears, and your database records nothing. You have no idea whether the user pressed play, watched ten seconds, or finished the whole thing.

That gap matters. For an e-learning course, a gated module, or any feature that requires users to actually consume content, "the page loaded" is the wrong signal. You need to know how much of the video they watched, not where the scrubber sits right now, but the furthest point they have ever reached.

The YouTube IFrame Player API exposes playback position, duration, and state changes from JavaScript. Combined with a small Django backend, that is enough to track the maximum percentage each user has reached on each video and persist it per user.

This article walks through a videos app: one model, two views for tracking, one template with embedded JavaScript, and Django's built-in auth to tie saves to request.user.

A working demo is in the companion repository at github.com/nunombispo/youtube-video-player-article, clone it to follow along.

Architecture

Five logical components make up the system. The diagram shows what each one owns and which boundaries it crosses:

video_detail template - Django renders the page with the video ID from the URL and any stored progress for the current user. It owns the HTML structure: the player mount point, the progress bar, the CSRF token, and the script tags that load the IFrame API.

Progress tracker (JavaScript) - Runs in the browser after the template loads. It reads playback state from the YouTube player, maintains maxPercentage locally, updates the progress bar, and POSTs to Django when the value increases. It never talks to the database directly; every persist goes through the view.

YouTube IFrame Player - An external embed controlled through the IFrame Player API. It supplies getCurrentTime(), getDuration(), and onStateChange events. The tracker depends on it but does not own it, swap the video ID and the same tracker code works for any YouTube video.

Views - video_detail serves the page and queries existing progress on GET. update_progress receives JSON POSTs and writes to the model. @login_required on the update view ties each save to request.user through Django auth. The view is the trust boundary: it validates input and enforces the rule that stored progress only moves forward.

VideoProgress model - One row per (user, video_id) pair. Stores max_percentage and updated_at. The view reads it when rendering the page and writes it on each accepted POST.

Data model

Progress tracking needs one persistent fact per user per video: the highest percentage ever reached. A single model holds that.

unique_together on user and video_id enforces one row per pair. Without it, repeated saves could create duplicate rows for the same user watching the same video. video_id is a CharField, YouTube IDs are 11 characters, but 20 leaves room for other providers if you adapt the code later.

from django.contrib.auth import get_user_model
from django.db import models

User = get_user_model()

class VideoProgress(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    video_id = models.CharField(max_length=20)
    max_percentage = models.PositiveIntegerField(default=0)
    updated_at = models.DateTimeField(auto_now=True)

    class Meta:
        unique_together = ('user', 'video_id')

max_percentage only ever increases, in JavaScript on the client and again in the view on the server. The double guard handles out-of-order requests: a slow POST from 40% must not overwrite a record already at 60%. updated_at refreshes on every save via auto_now=True, which gives you a last-activity timestamp without extra view code.

from django.contrib import admin
from .models import VideoProgress

admin.site.register(VideoProgress)

Add 'videos' to INSTALLED_APPS in settings.py, then run makemigrations and migrate.

The model is persistence only. Next, connect it to a page the user actually loads.

Template

video_detail.html extends a shared base.html that loads Bootstrap 5 from a CDN. Layout and styling are incidental to tracking, what matters is how the template passes data from Django into the player and the JavaScript tracker.

The HTML snippets below show the tracking-relevant markup only; the demo repository wraps them in Bootstrap cards and columns.

The template expects two context variables: video_id from the URL and progress from the database. The video_detail view supplies both, which we will cover in the Views section below.

Player mount and CSRF: The player div carries the video ID as a data attribute. Django fills {{ video_id }} from the URL; JavaScript reads dataset.videoId without mixing template tags into the script logic.

{% csrf_token %}
<div class="ratio ratio-16x9 bg-dark">
    <div id="player" data-video-id="{{ video_id }}"></div>
</div>

{% csrf_token %} renders a hidden input and ensures the csrftoken cookie is available. Django's CsrfViewMiddleware expects that cookie value back in the X-CSRFToken header on POST requests. Include the tag on any page that POSTs via fetch().

Seeding client state: When a user returns to a video they have partially watched, maxPercentage must start at the stored value, not zero. Otherwise a replay followed by an early tab close would under-report progress. The repo seeds maxPercentage, videoId, isAuthenticated, and updateProgressUrl at the top of a single {% block extra_js %} script, together with the IFrame API tag and the full tracker, see the JavaScript section below.

isAuthenticated lets the tracker skip POSTs for anonymous users. updateProgressUrl uses {% url %} so the path stays correct if you change URL patterns.

Progress bar: The bar renders from the same context on first load. JavaScript updates width and label as playback advances, so the user sees progress move without refreshing.

<div class="progress" style="height: 1.25rem;">
    <div class="progress-bar bg-danger"
         id="progress-bar"
         role="progressbar"
         style="width: {{ progress.max_percentage|default:0 }}%"
         aria-valuenow="{{ progress.max_percentage|default:0 }}"
         aria-valuemin="0"
         aria-valuemax="100">
    </div>
</div>
<span class="badge" id="progress-label">{{ progress.max_percentage|default:0 }}%</span>

Debug panel: A sidebar table shows server-stored values alongside live client state: current position, player state, and the last POST response. Remove it in production; it saves time while wiring up the tracking logic and confirming saves land without a page refresh.

<!-- Remove in production -->
<table class="table table-sm table-striped mb-0">
    <tbody>
        <tr>
            <th scope="row">Video ID</th>
            <td id="debug-video-id">{{ video_id }}</td>
        </tr>
        <tr>
            <th scope="row">Stored max</th>
            <td id="debug-stored-max">{{ progress.max_percentage|default:0 }}%</td>
        </tr>
        <tr>
            <th scope="row">Updated at</th>
            <td id="debug-stored-updated">{% if progress %}{{ progress.updated_at }}{% else %}—{% endif %}</td>
        </tr>
        <tr>
            <th scope="row">Client max</th>
            <td id="debug-client-max">{{ progress.max_percentage|default:0 }}%</td>
        </tr>
        <tr>
            <th scope="row">Current</th>
            <td id="debug-current">—</td>
        </tr>
        <tr>
            <th scope="row">State</th>
            <td><span id="debug-state">—</span></td>
        </tr>
        <tr>
            <th scope="row">Last save</th>
            <td id="debug-last-save">—</td>
        </tr>
    </tbody>
</table>

IFrame API: Load the YouTube script after the player element exists, at the bottom of the template in a {% block extra_js %}:

<script src="https://www.youtube.com/iframe_api"></script>

The API loads asynchronously and calls onYouTubeIframeAPIReady when ready. Placing the script after the div and deferring initialization to that callback avoids YT is not defined errors.

Views

Three views cover the demo. home renders a landing page with a link to a sample video. video_detail and update_progress do the tracking work.

video_detail runs on GET. It looks up any existing VideoProgress row for the authenticated user and passes it to the template along with the video_id from the URL.

from django.shortcuts import render

from .models import VideoProgress

def video_detail(request, video_id):
    progress = None
    if request.user.is_authenticated:
        progress = VideoProgress.objects.filter(
            user=request.user, video_id=video_id
        ).first()
    return render(request, 'videos/video_detail.html', {
        'video_id': video_id,
        'progress': progress,
    })

Anonymous users get progress = None; the template seeds maxPercentage to 0. They can still watch the video. Saves are skipped in JavaScript and blocked by @login_required on the update view.

update_progress runs on POST. It parses JSON, validates the payload, and upserts with a monotonic guard.

import json

from django.contrib.auth.decorators import login_required
from django.http import JsonResponse
from django.views.decorators.http import require_POST

from .models import VideoProgress

@login_required
@require_POST
def update_progress(request):
    data = json.loads(request.body)
    video_id = data.get('video_id', '')
    percentage = data.get('percentage', 0)

    if not video_id or not isinstance(percentage, (int, float)):
        return JsonResponse({'error': 'invalid data'}, status=400)

    obj, created = VideoProgress.objects.get_or_create(
        user=request.user,
        video_id=video_id,
        defaults={'max_percentage': percentage},
    )
    if not created and percentage > obj.max_percentage:
        obj.max_percentage = percentage
        obj.save()

    return JsonResponse({'status': 'ok'})

get_or_create inserts on first visit. The if not created branch updates only when the incoming percentage exceeds what is already stored. The user comes from request.user, the JSON payload carries video_id and percentage only.

Wire the views in videos/urls.py:

from django.urls import path

from . import views

urlpatterns = [
    path('', views.home, name='home'),
    path('videos/<str:video_id>/', views.video_detail, name='video_detail'),
    path('progress/update/', views.update_progress, name='update_progress'),
]

Include that file from config/urls.py:

from django.contrib import admin
from django.urls import include, path

urlpatterns = [
    path('admin/', admin.site.urls),
    path('accounts/', include('django.contrib.auth.urls')),
    path('', include('videos.urls')),
]

Auth: @login_required on update_progress redirects anonymous POSTs to the login page. Wire Django's built-in auth views and point LOGIN_URL at them:

# settings.py
LOGIN_URL = '/accounts/login/'
LOGIN_REDIRECT_URL = '/'

Create a user with createsuperuser (or your existing signup flow) before testing saves. Register VideoProgress in admin.py to inspect records at /admin/.

Open the Network tab and pause the video. You should see POST /progress/update/ with status 200 and {"status": "ok"}. A 403 means the CSRF token is missing, check {% csrf_token %} is on the page and X-CSRFToken is in the request headers. A 302 means @login_required redirected because the user is not authenticated.

JavaScript

The tracker script lives in video_detail.html inside {% block extra_js %}, after the IFrame API script tag. It has four jobs: initialize the player, compute watch percentage, maintain the running maximum, and POST updates to Django.

Declare template-sourced variables and throttle state first, sendProgressToDjango() and maybeSaveProgress() depend on them:

var player;
var maxPercentage = {{ progress.max_percentage|default:0 }};
var videoId = document.getElementById('player').dataset.videoId;
var isAuthenticated = {{ user.is_authenticated|yesno:"true,false" }};
var updateProgressUrl = '{% url "update_progress" %}';
var lastSavedPercentage = maxPercentage;
var lastSaveAttempt = 0;
var SAVE_INTERVAL_MS = 5000;

var STATE_NAMES = {
    '-1': 'UNSTARTED',
    '0': 'ENDED',
    '1': 'PLAYING',
    '2': 'PAUSED',
    '3': 'BUFFERING',
    '5': 'CUED'
};

var STATE_BADGES = {
    '-1': 'text-bg-secondary',
    '0': 'text-bg-success',
    '1': 'text-bg-danger',
    '2': 'text-bg-warning',
    '3': 'text-bg-info',
    '5': 'text-bg-secondary'
};

UI helpers: updateProgressBar() syncs the Bootstrap bar with maxPercentage. updateDebugDisplay() refreshes the debug panel if you kept it, skip both functions if you removed the panel.

function updateProgressBar(percentage) {
    var bar = document.getElementById('progress-bar');
    var label = document.getElementById('progress-label');
    bar.style.width = percentage + '%';
    bar.setAttribute('aria-valuenow', percentage);
    label.textContent = percentage + '%';
}

function updateDebugDisplay(stateCode) {
    var currentPct = player && player.getDuration ? getWatchPercentage() : null;
    var duration = player && player.getDuration ? player.getDuration() : null;
    var currentTime = player && player.getCurrentTime ? player.getCurrentTime() : null;

    document.getElementById('debug-client-max').textContent = maxPercentage + '%';

    if (currentPct !== null && duration) {
        document.getElementById('debug-current').textContent =
            currentPct + '% (' + Math.round(currentTime) + 's / ' + Math.round(duration) + 's)';
    }

    if (stateCode !== undefined) {
        var stateEl = document.getElementById('debug-state');
        var name = STATE_NAMES[String(stateCode)] || stateCode;
        stateEl.textContent = name;
        stateEl.className = 'badge debug-value ' + (STATE_BADGES[String(stateCode)] || 'text-bg-light text-dark');
    }
}

Player initialization: YouTube's IFrame API calls onYouTubeIframeAPIReady when the script finishes loading. Create the player therem calling new YT.Player() before this fires raises an error.

function onYouTubeIframeAPIReady() {
    player = new YT.Player('player', {
        videoId: videoId,
        playerVars: {
            autoplay: 0,
            controls: 1,
            rel: 0
        },
        events: {
            onStateChange: onPlayerStateChange
        }
    });
}

autoplay: 0 leaves playback control with the user. rel: 0 stops YouTube from suggesting unrelated videos when playback ends.

Watch percentage: Divide current time by duration and round. Guard against duration === 0 while the player is still loading, without the guard you get NaN.

function getWatchPercentage() {
    var duration = player.getDuration();
    if (duration === 0) return 0;
    return Math.round((player.getCurrentTime() / duration) * 100);
}

Maximum tracking: Store the highest value reached, not the current scrubber position. If a user watches to 80%, seeks back to 20%, and pauses, the stored value stays 80%.

function recordProgress() {
    var pct = getWatchPercentage();
    if (pct > maxPercentage) {
        maxPercentage = pct;
        updateProgressBar(maxPercentage);
        maybeSaveProgress();
    }
    updateDebugDisplay();
}

On ENDED, set maxPercentage to 100 directly. At the end of a video, getCurrentTime() sometimes returns a value slightly below getDuration() due to floating-point timing. Computing the percentage would give 99 instead of 100.

Saving to Django: Read the CSRF cookie and POST JSON to the URL the template provided. On a successful response, update lastSavedPercentage so the throttle in maybeSaveProgress() does not re-send the same value, and refresh the debug panel's stored fields and last-save status.

function getCsrfToken() {
    var cookies = document.cookie.split(';');
    for (var i = 0; i < cookies.length; i++) {
        var cookie = cookies[i].trim();
        if (cookie.startsWith('csrftoken=')) {
            return cookie.substring('csrftoken='.length);
        }
    }
    return '';
}

function sendProgressToDjango(percentage) {
    if (!isAuthenticated) {
        document.getElementById('debug-last-save').textContent = 'skipped (not logged in)';
        return;
    }

    fetch(updateProgressUrl, {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
            'X-CSRFToken': getCsrfToken()
        },
        body: JSON.stringify({
            video_id: videoId,
            percentage: percentage
        })
    }).then(function(response) {
        return response.json().then(function(data) {
            document.getElementById('debug-last-save').textContent =
                response.status + ' ' + JSON.stringify(data) + ' (' + percentage + '%)';
            if (response.ok) {
                lastSavedPercentage = percentage;
                document.getElementById('debug-stored-max').textContent = percentage + '%';
                document.getElementById('debug-stored-updated').textContent = new Date().toLocaleString();
                updateProgressBar(percentage);
            }
        });
    }).catch(function(err) {
        document.getElementById('debug-last-save').textContent = 'error: ' + err;
    }).finally(function() {
        lastSaveAttempt = Date.now();
    });
}

When to save: Three triggers cover normal watching, explicit stops, and tab closes.

While the video plays, a one-second interval calls recordProgress(). When maxPercentage increases, maybeSaveProgress() POSTs at most every five seconds, enough to persist progress during long viewing sessions without flooding the server.

On pause or end, maybeSaveProgress(true) bypasses the throttle and saves immediately if there is unsaved progress.

On tab close, beforeunload records the latest position and POSTs any value not yet saved. Treat that POST as best-effort, browsers often cancel in-flight fetch requests while the page unloads. Pause events and the five-second playback throttle cover most sessions; tab-close is a safety net. For stricter close behavior, navigator.sendBeacon() is an option, but it cannot send the X-CSRFToken header, so you would need a separate CSRF strategy for that endpoint.

function maybeSaveProgress(force) {
    if (!isAuthenticated) return;
    if (maxPercentage <= lastSavedPercentage) return;
    if (!force && Date.now() - lastSaveAttempt < SAVE_INTERVAL_MS) return;
    sendProgressToDjango(maxPercentage);
}

function onPlayerStateChange(event) {
    updateDebugDisplay(event.data);

    if (event.data === YT.PlayerState.PLAYING) {
        if (!window.debugInterval) {
            window.debugInterval = setInterval(function() {
                recordProgress();
                updateDebugDisplay(YT.PlayerState.PLAYING);
            }, 1000);
        }
    } else if (window.debugInterval) {
        clearInterval(window.debugInterval);
        window.debugInterval = null;
    }

    if (event.data === YT.PlayerState.PAUSED) {
        recordProgress();
        maybeSaveProgress(true);
    } else if (event.data === YT.PlayerState.ENDED) {
        maxPercentage = 100;
        updateProgressBar(100);
        maybeSaveProgress(true);
    }
}

window.addEventListener('beforeunload', function() {
    recordProgress();
    if (maxPercentage > lastSavedPercentage) {
        sendProgressToDjango(maxPercentage);
    }
});

Buffering edge case: buffering (state 3) can produce a PAUSED event followed by PLAYING when it resolves. A PAUSED event does not always mean the user pressed pause. If spurious saves become a problem, add a short debounce on PAUSED, check that the player is still paused 200ms later before recording.

Replay behavior: when a user replays a video, getCurrentTime() resets to 0 but maxPercentage keeps its previous value. If a user watched 90%, replays, and closes at 10%, you want 90% stored, not 10%. Reset maxPercentage to 0 only if your product logic requires it.

The debug panel updates stored max, client max, current, state, and last save in place after each successful POST, no page refresh required.

Conclusion

The gap at the start of this article, a video on the page and nothing in the database, is closed. The IFrame Player reports playback state. JavaScript maintains maxPercentage across pause, seek, and replay. Django receives POSTs, enforces the monotonic rule on the server, and persists one row per user and video in VideoProgress.

That row is the payoff. max_percentage is queryable anywhere in your project: unlock the next module at 80%, mark a course complete at 100%, or feed analytics without re-embedding the player. The tracking logic stays in one template and one view; the model is small enough to drop into an existing app without restructuring your project.

The companion repository at github.com/nunombispo/youtube-video-player-article runs the full pipeline, clone it if you want a working baseline before adapting the snippets to your own templates and URL layout.

Strip the debug panel before shipping, tune the save interval to match your traffic, and decide whether tab-close persistence needs sendBeacon() with a separate CSRF strategy.

Getting from a working demo to a production course platform is a different problem, deployment, observability, auth hardening. If you want a senior review of that path before launch, see developer-service.blog/work-with-me.

The hard part, knowing how far each user has actually watched, is done.

Follow me on Twitter: https://twitter.com/DevAsService

Follow me on Instagram: https://www.instagram.com/devasservice/

Follow me on TikTok: https://www.tiktok.com/@devasservice

Follow me on YouTube: https://www.youtube.com/@DevAsService

Should Your App Adopt Passkeys?

Developer Service — Thu, 25 Jun 2026 06:59:35 +0000

Someone on your leadership team asked a reasonable question: should we adopt passkeys?

You searched for answers and found implementation tutorials - WebAuthn server libraries, credential storage schemas, ceremony diagrams. They assume you've already decided. None of that helps you answer the question you were actually asked.

This article is a decision guide. The question isn't how to implement passkey login. It's whether you should, when the timing makes sense, and for which users first. Implementation details matter eventually - but they don't belong at the front of the decision.

You've seen Apple's demos and Google's Chrome nudges. Your security team may have sent a memo about phishing-resistant authentication. You know the term. What you don't have is a clear way to evaluate whether passkeys fit your product, your users, and your team's capacity to ship and support them.

By the end of this article, you'll have scored your app against a readiness checklist, mapped show-stoppers that can block adoption, and drafted a one-page recommendation for leadership.

Plain Terms: Passkeys, Passwords, and MFA

Before scoring your app, you and stakeholders need to mean the same thing when you say "passkey", "password", and "MFA". Vendor decks use these loosely. A PM might say "passkeys replace passwords" while security means "phishing-resistant credentials". Both can be true.

Passwords are shared secrets the user types; your server checks a hash. They leak via breaches and phishing sites. Users forget them, reuse them, and call support.

MFA adds a second factor - app push, SMS, hardware key, or biometric. It cuts credential-stuffing and many phishing attacks, but adds friction, lost-device tickets, and cross-platform complexity.

Passkeys are cryptographic key pairs on the user's device. The private key never leaves the device or synced passkey manager. Sign-in means unlocking with biometrics or a PIN; your server stores only the public key and verifies a signature. On web, the browser handles the ceremony (the sign-in handshake between browser, OS, and server) - the user sees a system prompt, not your form. On mobile, Face ID or fingerprint unlocks the credential.

Your team needs WebAuthn support (the web standard passkeys build on). Password reset flows become less central; support handles different questions - "I got a new phone" replaces "I forgot my password", but isn't necessarily easier.

Passkeys and MFA: Passkeys often are MFA - something you have (the device key) plus something you are or know (biometric/PIN). For many consumer apps, a passkey beats password plus SMS. In regulated contexts you may still want step-up auth. The question isn't "passkeys or MFA" - it's which combination fits your threat model.

What Passkeys Fix (and Don't)

What improves: Phishing and credential-stuffing. A fake login page can't harvest a passkey - the credential is bound to your app's origin, not a reusable secret. If your threat model includes phishing or password reuse, passkeys address real attacks that SMS MFA struggles to stop.

What doesn't change: Stolen session tokens, malware on the device, coerced unlocks, insider threats, and support social engineering. Passkeys shift the attack surface; they don't eliminate it.

"More secure" isn't a shipping criteria. You can adopt passkeys and still fail on recovery, cross-device sync, browser coverage, or support capacity. A rollout that locks users out is its own security incident.

When presenting upstream, lead with risk reduction, then gaps. A useful framing: "Passkeys eliminate phishing and credential-stuffing for users who adopt them. They don't protect compromised sessions or untested recovery. We'd need X engineering weeks, Y support training, and a phased rollout starting with Z segment".

That sentence gives leadership a decision, not a sermon. Your security colleagues will respect the nuance; your product colleagues need it to plan. Avoid claiming passkeys "solve authentication" - they solve specific authentication failures.

Don't oversell to get approval - security enthusiasm has killed projects that hit recovery and SSO blockers nobody mentioned. Users adopt passkeys when login works smoothly across their devices, not because they're cryptographically sound.

UX Reality

This is where most passkey decisions should start: what does the experience feel like for your users, not Apple's demo audience?

Registration: Creating a passkey happens at sign-up or when you prompt an existing user. The OS shows a system dialog - not your branded modal - asking for Face ID, fingerprint, or PIN. If they dismiss it, you need a fallback that doesn't punish them. Rollouts fail when the prompt appears before trust is established or without "skip for now". On mobile, registration feels seamless if users already unlock with biometrics. On desktop, it depends on platform passkey managers, QR handoff from a phone, or a plugged-in security key. Test on the device your users actually own - not just your MacBook.

Sign-in: Faster than typing a password - when it works. Web browsers offer passkey autofill or a separate path; users may pick which device holds the credential. Mobile delegates to the OS. Web users hit browser gaps and cross-device confusion; mobile users hit reinstalls and platform switches ("my passkey was in iCloud"). Neither platform guarantees the vendor-demo flow.

Cross-device sync: Passkeys live in iCloud Keychain, Google Password Manager, 1Password, or a hardware key with no sync. Synced credentials roam across devices on the same account; device-bound ones stay on one machine. A passkey on iPhone may appear on iPad automatically but not on a work Windows laptop - and the user may not expect that. Roaming credentials (synced via platform cloud) behave differently from device-bound credentials scoped to a single authenticator. You don't implement sync; you understand what your users' devices and managers will and won't do, and set expectations accordingly.

Friction that kills adoption: new device without sync; lost phone with sync disabled; shared computers (family PC, library kiosk) where personal biometrics don't fit; enterprise users refusing to store work credentials in personal iCloud. Walk through these scenarios with your team before writing a launch date.

Ask about your audience: What percentage sign in mobile vs web? Multiple devices or one? Shared devices? Will they understand "use your passkey on your phone to sign in here"? Mobile-first audiences on recent iOS/Android get good UX today. Web-heavy, heterogeneous audiences need more fallbacks and slower adoption.

Reality-check: Walk through registration and sign-in on your oldest supported device, then on a family member's phone without explaining the steps.

Readiness Checklist - Score Your App

Turn everything above into a number you can defend in a meeting.

Score each item 0 (gap, no plan), 1 (planned/untested), or 2 (verified in staging or production). Weight make-or-break items 2× - mobile-heavy products weight device mix; web-first products weight browser coverage; B2B weights recovery and fallbacks. Example: raw score 11, but recovery (0) and browser coverage (1) are 2× for your web-first app — treat as yellow, not a low green.

Score yourself now; 10 minutes.

Item	0	1	2
Auth stack — Backend registers/verifies WebAuthn/FIDO2 credentials
Mobile integration — App uses platform authenticator
Browser coverage — Passkeys on browsers covering ≥80% of sign-in traffic
Device mix — Users predominantly on passkey-capable devices
Account model — Multiple credentials per user (passkey + password)
Recovery path — Tested way to regain access without original passkey
Fallback flows — Users who skip passkey setup can still sign in
Support readiness — Team trained on passkey troubleshooting
Pilot metrics — Can measure registration, sign-in success, tickets by auth method
Cross-device sync — Tested across device combinations your users actually use

Thresholds: Green 14–20 - pilot-ready with fallbacks and measurement. Yellow 9–13 - close, but fix gaps (often recovery, support, cross-device). Red 0–8 - engineering work needed, not a launch announcement.

A red score points to three paths - not "never":

Phased approach: Ship passkeys as opt-in for supported devices while keeping passwords. Your full user base may be yellow while a narrow segment is green.
Fix and retest: Address the 0s - usually recovery and cross-device - then rescore. Many teams jump from red to green in one focused QA sprint.
Wait: If most users are on unsupported browsers, waiting six to twelve months may cost less than shipping a broken experience.

One 0 on recovery or cross-device sync can veto a green total.

Show-Stoppers

A green score doesn't guarantee a safe launch. Any of these can override your math.

Recovery is the hardest unsolved problem. With passwords, "forgot password" sends a reset link. With passkeys, there's no secret to reset - the credential is device-bound. If the user loses every device holding their passkey, you need a backup path designed in advance: passwords during transition, verified magic links, a second passkey on another device, or platform recovery users may not realize is load-bearing for your app.

None of these are elegant. Each has tradeoffs in security and UX. "Good enough" means a documented recovery flow tested with real users - not a diagram in a design doc. Shipping without tested recovery locks users out; that's worse for trust and support cost than delaying one sprint.

Warning: Recovery gaps cause more damage than delayed launch.

Enterprise SSO adds complexity when users sign in via Okta, Azure AD, or Google Workspace. Who creates and stores the passkey - your app or the IdP? B2B customers may require corporate-managed passkey managers, not personal iCloud. Passkeys fit cleanly when you own auth end-to-end; scope pilots away from enterprise edge cases until the happy path works.

Legacy users - email-only accounts, social login, phone auth - each need a migration path. Forced upgrades fail: users ignore prompts or churn if you cut off their existing method. Gradual migration works: prompt at login, offer passkey setup as optional, keep the old method until success or explicit decline. "Forever" dual auth is valid; "remove passwords next quarter" without recovery solved is not.

Support burden rises with new ticket types: new phone, laptop-vs-phone confusion, accidental passkey creation, "what is a passkey?" Password-reset tickets may spike before they fall. Budget training and scripts before launch; tag tickets by auth method. A successful rollout shows password-reset volume declining; a failed one shows total auth tickets rising.

Pilot, Present, Decide

Show-stoppers mean don't expose your entire user base on day one. A pilot produces real data from a cohort small enough to support manually.

Good first cohorts share three traits: passkey-capable devices, enough technical comfort to tolerate rough edges, and tolerance for contacting support. If 60% of sign-ins are from iPhone, an iOS-heavy cohort reflects reality. Internal employees are a common starting point - reachable, forgiving, and available for interviews when flows break. Don't pilot first with your highest-value or least-forgiving customers.

Design choices: Opt-in ("Add a passkey" in settings) is safest. Prompted-at-login (skippable) surfaces friction faster. Keep passwords alongside passkeys - almost always. Pick one primary question: registration completion or passkey sign-in replacing passwords.

Measure from day one: registration rate, sign-in success by platform, support tickets vs baseline, drop-off by step. Set targets before launch - e.g., ≥70% registration among prompted users, <2% auth-ticket increase.

Expand when registration and sign-in success meet targets, support volume is flat or down, and recovery handled every test case - widen the cohort or move from opt-in to prompted default.

Pause when metrics are flat but not failing. Fix friction, retest with the same cohort before adding users.

Roll back when lockouts occur, tickets spike, or recovery fails in production. Disable the prompt, keep passkeys for users who already registered, fix, then resume.

Two weeks with 500 internal users produces more actionable data than a six-month beta for 50,000 customers with no owner watching metrics.

One-page template for leadership

Put the verdict in the first sentence. Caveats come after.

Recommendation: [Pilot / Proceed / Defer] — [one sentence]

Readiness score: [X/20] — [Green / Yellow / Red]

Rationale: [primary benefit] · [why now or not yet] · [pilot scope: who, how long]

Risks and mitigations: [risk] → [mitigation]

Timeline: [build weeks] · [pilot duration] · [decision point]

Resources: [engineering] · [support training] · [legal/privacy, if needed]

Open questions: [what the pilot must answer]

Identify sign-offs early - engineering (WebAuthn, recovery, metrics), product (UX, cohort, comms), security (threat model, MFA policy), legal/privacy if auth changes affect terms. You don't need every sign-off before a pilot, but name them so nobody blocks you after you've announced a date.

Proceed (green, pilot validated): "We recommend expanding passkey sign-in to [segment] starting [date]. Pilot: [X]% registration, [Y]% sign-in success, flat auth tickets. Password fallback remains. Recovery tested for [scenarios]."

Pilot first: "We recommend a 4-week pilot with internal iOS users before broader rollout. Readiness: 11/20 (yellow). Gaps: recovery testing, support scripts. Success: ≥70% registration, flat auth tickets. Decision to expand by [date]."

Not yet: "Defer until recovery is tested. Readiness: 7/20. Interim: strengthen MFA, monitor platform support. Revisit in 6 months."

When to Say "Not Yet"

"Not yet" sounds like indecision. It can be the most responsible answer.

Defer when a bad rollout costs more than waiting: red score with no path to yellow, recovery unsolved, unsupported device mix, unscoped SSO, or no support capacity. Deferral means interim steps plus a revisit date - not an open-ended "we'll look at it later".

Instead of passkeys now: replace SMS with TOTP or push MFA; add WebAuthn as a second factor alongside passwords; offer magic-link sign-in to reduce password exposure; track passkey-capable traffic in analytics and spike WebAuthn in staging.

Pick one or two interim steps and name them in your one-page recommendation. Leadership wants progress, not a blank "we're waiting".

Rescore in six to twelve months. Browser support in your analytics, IdP capabilities, recovery patterns, and your own auth stack all shift faster than vendor keynotes suggest. A deferral with a revisit date is a decision; "we'll look at it later" is avoidance.

You started with should we adopt passkeys? You now have vocabulary, a scored checklist, show-stoppers, a pilot design, and a one-page template. Whether your answer is pilot, proceed, or not yet — defend it with evidence, not vendor slides.

Need help with the decision? I work with engineering teams on architecture reviews, production readiness, and getting auth stacks right before they ship. If you're between yellow and green on the checklist and want a senior second opinion, work with me.

Follow me on Twitter: https://twitter.com/DevAsService

Follow me on Instagram: https://www.instagram.com/devasservice/

Follow me on TikTok: https://www.tiktok.com/@devasservice

Follow me on YouTube: https://www.youtube.com/@DevAsService

Image by Mohamed Hassan from Pixabay

PydanticAI vs LangChain - Choosing an Agent Framework for Production, Not Demos

Developer Service — Mon, 22 Jun 2026 06:46:35 +0000

In a recent audit, a team showed me an AI assistant they'd built on top of their company knowledge base. The demo had landed well: ask how to use a feature, and it walked through the exact pain point their support queue kept seeing. Leadership signed off.

In production, the same agent told a user to open a menu option that didn't exist. Not a vague answer - a specific UI path, stated with confidence. Nobody caught it in testing. It surfaced when I audited the system, not when a user complained.

The prototype passed testing because nobody was checking whether the answer matched the product. In production, that gap becomes a liability: the model invents UI paths, and your backend has no schema to reject them.

When you're choosing an agent framework, popularity is the wrong scorecard. Pick the one that fails loudly in development and gracefully in production - or you'll find out in audit.

What "Production-Ready" Actually Requires

Tutorial agents are built to impress in a fifteen-minute demo. Production agents run unattended, handle bad inputs, and ship answers your backend has to trust. The gap between those two goals is where most teams stumble - and it's rarely visible until something reaches a user.

When I audit agent codebases, I evaluate five things the tutorials skip:

Structured, validated outputs: Can your system reject an invented menu path before it becomes user-facing advice?
Dependency injection for testing: Can you swap the knowledge base for a mock in CI without rewiring the agent?
Retry and error handling: When the model returns malformed output, does the framework retry - or do you ship a parser exception?
Observability hooks: Can you trace which document grounded a bad answer when support escalates?
Type-checker support: Will static analysis catch a breaking API change before deploy, or after the agent silently misbehaves?

If you want to score your own system, the Production Readiness Audit covers the same five categories - deployment, observability, failure modes, and a prioritized remediation plan.

Side-by-Side: The Same Agent, Two Frameworks

The first item on the rubric is structured, validated outputs. The clearest way to see the framework difference is to build the same agent twice.

The task: answer natural-language questions about a CSV of sales data. The agent calls a tool to query the file, then returns a structured answer your API can pass downstream without a second parsing step.

LangChain

from langchain.agents import create_agent
from langchain.tools import tool

@tool
def query_sales_csv(region: str) -> str:
    """Return total revenue for a region in the sales CSV."""
    total = df.loc[df["region"] == region, "revenue"].sum()
    return f"{region}: ${total:,.0f}"

agent = create_agent("anthropic:claude-sonnet-4-6", tools=[query_sales_csv])
result = agent.invoke({
    "messages": [{"role": "user", "content": "What was Q1 revenue in Europe?"}],
})

answer = result["messages"][-1].content  # str — you validate the shape yourself

This is the pattern most tutorials teach. The tool works, the agent runs, the demo looks fine. But answer is a string (or occasionally a dict, depending on the model). Nothing in this flow checks that the response contains a real region name, a numeric revenue, or the right currency. If the model formats the answer as prose instead of data, your code finds out in production - or in audit.

LangChain does support a response_format parameter with Pydantic models. It's opt-in, and most teams I audit haven't wired it up yet.

PydanticAI

from pydantic import BaseModel
from pydantic_ai import Agent

class SalesAnswer(BaseModel):
    region: str
    total_revenue: float
    currency: str = "USD"

agent = Agent("anthropic:claude-sonnet-4-6", output_type=SalesAnswer)

@agent.tool_plain
def query_sales_csv(region: str) -> float:
    """Return total revenue for a region in the sales CSV."""
    return float(df.loc[df["region"] == region, "revenue"].sum())

result = agent.run_sync("What was Q1 revenue in Europe?")
answer = result.output  # SalesAnswer — validated before your code runs

Here, validation isn't a step you add later - it's the contract. output_type=SalesAnswer tells the agent what shape to return. If the model produces something that doesn't match - wrong field, missing revenue, invented region - PydanticAI raises before your application code touches it. You get a SalesAnswer object your type checker understands, not a string you hope to parse.

Same task, same tool, same model. The difference is what happens after the LLM responds: LangChain hands you text and trusts you'll validate it; PydanticAI hands you a typed object or fails immediately.

Dependency Injection & Testability

Validated outputs tell you the shape is right. Dependency injection tells you the data is right - and lets you prove it without calling a live API on every CI run.

Agent tools don't operate in a vacuum. They read from databases, knowledge bases, and internal APIs. In production, those dependencies are real. In tests, they need to be fake - predictable, fast, and free. The question is whether your framework makes that swap explicit or forces you to hack around it.

PydanticAI: dependencies as a first-class parameter

PydanticAI declares what an agent needs via deps_type. Tools receive a RunContext and pull dependencies from ctx.deps. At run time, you pass the real implementation; in tests, you pass a fake.

from dataclasses import dataclass
from pydantic_ai import Agent, RunContext

@dataclass
class SalesDataSource:
    def revenue_for(self, region: str) -> float:
        return float(df.loc[df["region"] == region, "revenue"].sum())

agent = Agent(
    "anthropic:claude-sonnet-4-6",
    deps_type=SalesDataSource,
    output_type=SalesAnswer,
)

@agent.tool
def query_sales_csv(ctx: RunContext[SalesDataSource], region: str) -> float:
    return ctx.deps.revenue_for(region)

# Production: agent.run_sync(prompt, deps=SalesDataSource())
# Test:       agent.run_sync(prompt, deps=FakeSalesData(revenue=1_250_000))

The type checker enforces the contract. If a tool expects SalesDataSource and you pass something else, mypy catches it before merge. Your test injects FakeSalesData(revenue=1_250_000) and asserts the agent's structured output matches - no CSV file, no network, no API key in CI.

LangChain: it works, but the seams are yours to find

LangChain agents can be tested, but the framework doesn't give you an injection point. The usual pattern is a module-level dependency the tool closes over, then unittest.mock.patch in tests:

from unittest.mock import patch
from langchain.agents import create_agent
from langchain.tools import tool

data_source = SalesDataSource()  # module-level — no framework injection point

@tool
def query_sales_csv(region: str) -> str:
    """Return total revenue for a region in the sales CSV."""
    return f"{region}: ${data_source.revenue_for(region):,.0f}"

agent = create_agent("anthropic:claude-sonnet-4-6", tools=[query_sales_csv])

# Test: patch the module where data_source lives
with patch("myapp.sales_agent.data_source", FakeSalesData(revenue=1_250_000)):
    result = agent.invoke({"messages": [{"role": "user", "content": "Q1 Europe?"}]})

Here you're patching a string path - "myapp.sales_agent.data_source" - that must match exactly where the module is imported. Rename the file, change the import structure, or run tests in parallel that share a patched global, and you get flakes or false greens.

If you've fought flaky agent tests, you've lived this. The test doesn't fail because the agent logic is wrong; it fails because the test setup is fighting the framework's defaults.

PydanticAI doesn't eliminate the need to write tests. It gives you a seam that was designed for swapping. That's the difference between "we test our agents in CI" and "we test our agents in CI reliably."

Error Handling and Retries in Practice

When the model returns garbage, what happens next? In production, "garbage" isn't always obvious - it's a well-formed JSON object with total_revenue: "approximately high" or a region name the CSV doesn't contain. The framework should catch that and recover, not pass it to your API.

PydanticAI: validation failures feed back to the model

When output_type is a Pydantic model, schema violations don't reach your application code. PydanticAI sends the validation error back to the model and retries:

from pydantic_ai import Agent, ModelRetry

agent = Agent(
    "anthropic:claude-sonnet-4-6",
    output_type=SalesAnswer,
    retries=3,
)

@agent.output_validator
def must_have_revenue(ctx, output: SalesAnswer) -> SalesAnswer:
    if output.total_revenue <= 0:
        raise ModelRetry("Revenue must be positive. Call query_sales_csv and retry.")
    return output

retries=3 handles structural failures - wrong types, missing fields, malformed JSON. The @output_validator handles business rules the schema can't express. Both paths raise ModelRetry, which tells the agent to try again with the error message as context. If all retries exhaust, you get an explicit exception - not a silent bad record in your database.

LangChain: you assemble the loop

LangChain can validate structured output via response_format, but retry orchestration is still your code. Schema errors, business rules, retry limits, and message history - you wire it together:

from pydantic import ValidationError
from langchain.agents import create_agent

MAX_RETRIES = 3

agent = create_agent(
    "anthropic:claude-sonnet-4-6",
    tools=[query_sales_csv],
    response_format=SalesAnswer,
)

messages = [{"role": "user", "content": "What was Q1 revenue in Europe?"}]
for attempt in range(MAX_RETRIES):
    result = agent.invoke({"messages": messages})
    try:
        answer = SalesAnswer.model_validate(result["structured_response"])
        if answer.total_revenue <= 0:
            raise ValueError("Revenue must be positive")
        break
    except (ValidationError, ValueError) as e:
        messages = result["messages"] + [
            {"role": "user", "content": f"Validation failed: {e}. Try again."}
        ]
else:
    raise RuntimeError("Max retries exceeded")

LangChain also offers ToolStrategy with a handle_errors parameter for schema-level retries - closer to PydanticAI's defaults. But business-rule validation like total_revenue > 0 still lands in your loop. And teams using no response_format pattern write even more of this by hand: parse JSON from message text, catch OutputParserException, append errors, track attempts.

The operational difference: PydanticAI treats "output didn't validate" as a normal agent loop event. LangChain treats it as an exception you handle - if you remembered to write the handler.

Testing Without Hitting the API

In the previous sections, you saw how to test agent logic. This section is about testing without paying for it. Every CI run that calls a real LLM costs money, adds latency, and flakes when the model paraphrases. Most teams know this; fewer build around it.

PydanticAI ships test doubles for the model itself. TestModel replaces the LLM with deterministic Python: it calls your tools, generates schema-valid output, and never hits the network. FunctionModel goes further - you write the mock responses in plain Python when you need specific behavior.

from pydantic_ai.models.test import TestModel

def test_sales_agent_returns_validated_output():
    with agent.override(model=TestModel()):
        result = agent.run_sync(
            "What was Q1 revenue in Europe?",
            deps=FakeSalesData(revenue=1_250_000),
        )
    assert isinstance(result.output, SalesAnswer)
    assert result.output.total_revenue > 0

agent.override(model=TestModel()) swaps the model at the agent boundary - the same boundary where you pass deps= in previous sections. Your application code doesn't change; your test doesn't need an API key.

LangChain: script every model turn

LangChain's equivalent is GenericFakeChatModel - you pass an iterator of scripted AIMessage responses, one per model invocation in the agent loop:

from langchain_core.language_models.fake_chat_models import GenericFakeChatModel
from langchain_core.messages import AIMessage, ToolCall

fake_model = GenericFakeChatModel(messages=iter([
    AIMessage(content="", tool_calls=[
        ToolCall(name="query_sales_csv", args={"region": "Europe"}, id="call_1"),
    ]),
    AIMessage(content='{"region": "Europe", "total_revenue": 1250000.0}'),
]))

with patch("myapp.sales_agent.data_source", FakeSalesData(revenue=1_250_000)):
    test_agent = create_agent(
        fake_model, tools=[query_sales_csv], response_format=SalesAnswer,
    )
    result = test_agent.invoke({"messages": [{"role": "user", "content": "Q1 Europe?"}]})
    assert SalesAnswer.model_validate(result["structured_response"]).total_revenue > 0

Here you're scripting each turn: first a tool call, then a JSON payload. Add a retry path or a second tool and you extend the iterator. Miss a turn and the test fails with an opaque StopIteration.

For teams running agent tests on every PR, that's the single biggest cost-saving win I see when moving to PydanticAI: tests on every commit, no API bill, assertions on typed output.

Where LangChain Still Wins

This article argues for PydanticAI on production grounds. That case is weaker if you're not heading to production yet - and LangChain earns its popularity for good reasons.

Integration breadth: LangChain connects to more data sources, vector stores, and model providers out of the box. If you need a connector that PydanticAI doesn't ship yet - a niche CRM, an internal protobuf service, a legacy search index - LangChain's community has probably already built it.

Prototyping speed: Pre-built chains, LangGraph templates, and copy-paste tutorials get a demo in front of stakeholders fast. For a two-week proof of concept where the goal is "show the CEO something that talks", that velocity matters more than typed outputs.

Ecosystem maturity: More Stack Overflow answers, more LangSmith integrations, more hiring-market familiarity. If your team already knows LangChain and the project might not ship, switching frameworks adds cost with no payoff.

None of this makes LangChain the wrong choice for production - teams ship reliable LangChain agents every day. But they do it by adding the validation, testing, and error-handling layers this article shows PydanticAI includes by default. If you're building those layers yourself anyway, the framework choice matters less.

Choose LangChain when speed and integrations beat type safety. Choose PydanticAI when you're done demoing and need the agent to run without you in the room.

Decision Checklist

Bring this to your next architecture review. Seven yes/no questions - answer for the project you're actually building, not the demo you already shipped:

Do you need typed outputs your backend can trust without re-validation?
Will this agent run unattended in production?
Do you need to unit-test agent logic in CI without API calls?
When the model returns malformed output, should the framework retry automatically?
Do you need to trace which document or tool call produced a given answer?
Will static analysis catch breaking changes to your output schema before deploy?
Do you need a niche integration that only exists in LangChain's ecosystem today?

Scoring: Three or more yes on questions 1–6 → lean PydanticAI. Mostly no, or yes only on question 7 → LangChain is fine for now. Yes on 1–3 but no on 4–6 → you're heading to production but haven't built the hard parts yet. Either framework works, but budget time for the gaps this article mapped in sections 3–6.

For a full assessment across your codebase - not just the framework choice - see the Production Readiness Audit. Same categories, applied to what you've actually shipped.

Conclusion

If the checklist pointed you toward PydanticAI but you're already on LangChain, you don't need a rewrite.

Your tools - the functions that query databases, search knowledge bases, and call internal APIs - are plain Python. Wrap them as PydanticAI @agent.tool handlers and migrate one agent at a time. Run both frameworks side by side during the transition; retire LangChain paths as each agent passes the production tests you couldn't write before.

The framework decision is one input. The harder question is whether your agent stack is ready for what happens after the demo - the invented menu options, the silent validation gaps, the CI runs that still hit the API.

Book a 20-minute intro call and tell me what you're building. Or jump straight to an Architecture Call or an AI Code Health Check.

Follow me on Twitter: https://twitter.com/DevAsService

Follow me on Instagram: https://www.instagram.com/devasservice/

Follow me on TikTok: https://www.tiktok.com/@devasservice

Follow me on YouTube: https://www.youtube.com/@DevAsService

Why Architecture Reviews Need Contracts, Not Chat

Developer Service — Fri, 12 Jun 2026 07:28:04 +0000

Architecture reviews have a translation problem.

Humans can leave a thread of “consider X” and “what about Y” and resolve the rest in a meeting. But if you want an LLM to participate in a workflow that resembles engineering - PRs, ADRs, ticketing, CI gates - fluent feedback isn’t enough. You need output that downstream systems (and humans) can reliably act on.

Most “LLM architecture review” demos stop at persuasive prose. The result reads like an experienced engineer, but it isn’t shaped like an artifact: it’s hard to rank, route, deduplicate, or turn into work without a second manual pass.

Multi-agent helps because architecture review is a bundle of lenses - security, scalability, operability, cost, data integrity, failure recovery - each with its own heuristics and thresholds. But the real differentiator isn’t “one agent vs. many”. It’s contracts. With PydanticAI, you define the schema the system must emit and validate every response; Claude supplies the reasoning, but the contract forces it into a machine-actionable shape.

This article shows how to build a multi-agent architecture reviewer that produces a structured review artifact: normalized findings with severity, evidence, and recommendations, plus clarifying questions and explicit “needs human judgment” flags. Think less chatbot, more review report.

The full runnable example lives in the companion repository.

In the next sections, we’ll define the minimal topology (planner → specialists → synthesizer), the shared contracts, and an end-to-end walkthrough from design doc to structured report.

What Multi-Agent Buys You (and What It Costs)

Multi-agent systems are easy to overuse. If all you need is a single round of feedback - “scan this design doc for obvious risks” or “suggest alternatives for this storage layer” - a well-constructed prompt with a structured output schema can get you most of the value with a fraction of the complexity. The moment you add agents, you add orchestration, state, and failure handling; if you don’t get clear returns, you’ve just built a slower, more expensive version of a single call.

So what does multi-agent actually buy you for architecture review? Primarily: separation of concerns and parallel perspectives. A single “review this architecture” prompt tends to collapse into a handful of generic patterns: it repeats the same risks across categories, it misses domain-specific edge cases, and it blurs “ask a question” with “assert a fact”. Splitting the work into roles lets you put sharper instructions and narrower context windows in front of each agent, and it gives you a natural place to express uncertainty (“as the security reviewer, I need X to conclude Y”).

Monolithic prompts also fail in predictable ways. They produce long outputs that are hard to rank; they mix high-confidence issues with speculative ones; they contradict themselves; and they often lack traceable evidence tied to the input. Those failures aren’t just aesthetic - they make downstream automation unreliable. If you can’t consistently tell what is a “P0” vs. a “P3”, or what is a requirement vs. a suggestion, you can’t turn the review into an engineering workflow.

The costs are real. Fan-out (running multiple specialists) increases token spend and latency, and it introduces coordination overhead: you need to manage shared context, avoid duplication, and merge disagreements into a single report. Sequential pipelines can reduce duplication and keep context tighter, but they can amplify early mistakes (if the first step narrows scope incorrectly, everything downstream inherits that error). In practice, you’re trading a single model call for a small distributed system.

For a lean architecture reviewer, a minimal topology works well:

A planner reads the input and decides which lenses to apply (security, scalability, operability, data, cost), plus any clarifying questions.
A small set of specialists each run one lens and emit findings in a shared schema.
A synthesizer merges the results: deduplicates, ranks, resolves contradictions (or preserves dissent explicitly), and produces the final structured review artifact.

The decision heuristic is simple: stay lean unless you need genuinely different perspectives. If your review output is consistently repetitive, shallow, or internally inconsistent, specialist roles can help. If you’re already getting high-quality structured output from one call, don’t add agents - add better contracts, better evidence requirements, and better “unknowns” handling.

Contracts First: Pydantic Models as the Agent API

If you only take one idea from this article, it should be this: the schema is the product. The model is a reasoning engine, but your system’s reliability comes from the contract you force that reasoning to satisfy. Most agent demos skip this and then wonder why their outputs can’t be trusted in an engineering workflow.

Architecture review is especially sensitive to this because it’s full of ambiguity. Some findings are hard requirements (“this violates a compliance constraint”), some are conditional (“if traffic can spike 10×, you need backpressure”), and some are simply questions (“what is your RTO/RPO?”). Without a contract that distinguishes these categories - and forces the model to attach evidence or explicitly mark unknowns - you end up with prose that sounds plausible but is operationally unusable.

In Pydantic terms, you want to model the output as a small set of types that match how engineers actually consume reviews. A typical core might include:

A top-level ArchitectureReview artifact (metadata, overall risk, summary).
A list of Finding objects with fields like title, severity, category, and recommendation.
An evidence or references field that ties the claim to something in the input (or marks it as an inference).
A separate list of questions for missing information that blocks a confident conclusion.

The exact fields are less important than the discipline: every claim must land somewhere explicit. Severity can’t be buried in adjectives. Recommendations can’t be scattered across paragraphs. Uncertainty can’t be implied; it needs a home in the schema.

Contracts also apply to inputs. Each agent role should have a clear definition of what context it is allowed to assume and what it must treat as unknown. That’s the difference between a specialist saying “use mTLS” by default versus saying “if there are multi-tenant boundaries or untrusted networks, consider mTLS; otherwise justify why it’s unnecessary”. The more explicit you are about inputs (constraints, traffic assumptions, data classification, SLOs), the less your agents have to guess.

Once you have contracts, you can make the system resilient with validation and repair loops. With PydanticAI-style structured outputs, you can:

Validate every model response against the schema.
Retry on validation failure with a targeted instruction (“output must include evidence for each finding”).
Apply a small “repair” step when the model is close but not quite compliant (missing a field, wrong enum value).

This isn’t about being pedantic. It’s what turns “LLM output” into “typed data” you can route downstream: create issues in a tracker, post a summarized comment to a PR, generate an ADR checklist, or feed a dashboard that tracks the recurring classes of architectural risk your org keeps rediscovering.

Multi-agent orchestration only works well when everyone speaks the same language. The contract is that language.

If you want to go deeper on modeling, validation, and the patterns that make Pydantic contracts reliable in production, see Practical Pydantic - a hands-on guide to data validation in Python, from core concepts through real-world APIs and pipelines.

Code example: a minimal contract set

The working example in models.py defines a small set of Pydantic models that act as the “agent API” for planning, specialist review, and synthesis:

from enum import Enum
from typing import Literal

from pydantic import BaseModel, Field


class Lens(str, Enum):
    security = "security"
    scalability = "scalability"
    operability = "operability"
    data_integrity = "data_integrity"


class Severity(str, Enum):
    p0 = "p0"
    p1 = "p1"
    p2 = "p2"
    p3 = "p3"


class Evidence(BaseModel):
    kind: Literal["quote", "observation", "inference"] = "observation"
    detail: str = Field(..., description="Quote or observation tied to the provided input.")


class Finding(BaseModel):
    title: str
    category: str = Field(..., description="Short category label (e.g. authz, backpressure).")
    severity: Severity
    evidence: list[Evidence] = Field(default_factory=list)
    recommendation: str


class PlannerOutput(BaseModel):
    lenses: list[Lens]
    scope_notes: str = ""
    clarifying_questions: list[str] = Field(default_factory=list)


class ArchitectureReview(BaseModel):
    summary: str
    overall_risk: Literal["low", "medium", "high"] = "medium"
    findings: list[Finding] = Field(default_factory=list)
    questions: list[str] = Field(default_factory=list)

Role Design: Planner, Specialists, Synthesizer

The fastest way to ruin a multi-agent system is to give every agent the same vague job: “review the architecture”. You’ll pay for multiple calls and still get duplicated, generic output. Role design is where multi-agent becomes an engineering tool instead of a prompt trick.

A useful rule is: each role must have a one-sentence job description that is both necessary and non-overlapping. If two roles can produce the same kind of output, you haven’t created separation of concerns - you’ve created redundancy.

For a lean architecture reviewer, three roles are enough:

Planner

The planner’s job is to decide what review should happen given the input and constraints. It does not emit a full review. It produces:

Which lenses to apply (security, scalability, operability, cost, data integrity, compliance, etc.).
Any clarifying questions that block a confident review (“What’s the expected peak RPS?”, “What is the data classification?”).
Optional scoping notes (“Focus on failure recovery and multi-region; ignore UI concerns.”).

This role is where you avoid wasted work. If the architecture is a batch pipeline with no public ingress, a deep web security pass is noise. If it’s a multi-tenant SaaS, ignoring tenant boundaries is negligence. The planner sets those priorities explicitly.

Specialists

Each specialist’s job is to run one lens and emit findings in the shared contract. Specialists should not:

Re-scope the review (“I think we should also do operability”).
Invent missing context as if it were true.
Produce long narrative prose that the synthesizer can’t merge.

They should be opinionated within their lens, but disciplined about uncertainty. A good specialist output contains high-signal findings and clear questions when key context is missing. The contract is the forcing function: each finding needs a category, a severity, evidence, and a recommendation.

Synthesizer

The synthesizer’s job is to produce the final ArchitectureReview artifact. That means:

Deduplicate overlapping findings across specialists.
Resolve contradictions when possible, or preserve dissent when it matters (“Security flags P0 unless X; Scalability says acceptable if Y”).
Rank and prioritize based on severity and expected impact.
Produce a concise summary that is consistent with the structured findings, not an independent “new” review.

The synthesizer is also where you enforce global policy: severity definitions, house style, and what counts as acceptable evidence. In other words, it turns a bag of lens-specific opinions into a single report that an engineering team can act on.

Prompt boundaries

Prompt boundaries are not decorative; they prevent scope creep and hallucinated authority. Each role should have explicit “must not” constraints. Examples:

The planner must not emit findings.
Specialists must not rewrite the contract or invent missing facts.
The synthesizer must not add new findings that were not supported by specialist outputs or input evidence (unless explicitly marked as an inference with low confidence).

When roles are crisp, orchestration becomes straightforward: you know what inputs each step needs and what outputs it is allowed to produce. When roles are fuzzy, you’ll spend your time chasing inconsistencies and blame-shifting between agents.

Code example: defining role agents

In the working example (reviewer.py), each role is an Agent with an output_type set to one of the contracts, and all roles share the same dependency type (ReviewDeps) carrying the design doc.

Passing deps=ReviewDeps(design_doc=...) alone is not enough: PydanticAI does not automatically inject dependencies into the prompt. Use dynamic instructions to attach the document to every run:

from __future__ import annotations

import os
from dotenv import load_dotenv
from dataclasses import dataclass

from pydantic_ai import Agent, RunContext

from models import ArchitectureReview, Lens, PlannerOutput, SpecialistOutput


load_dotenv()

DEFAULT_MODEL = os.getenv("MODEL", "anthropic:claude-sonnet-4-6")

if not os.getenv("ANTHROPIC_API_KEY"):
    raise RuntimeError(
        "ANTHROPIC_API_KEY is not set. Export it or add it to a .env file in the project root."
    )


@dataclass(frozen=True)
class ReviewDeps:
    design_doc: str


planner = Agent(
    DEFAULT_MODEL,
    deps_type=ReviewDeps,
    output_type=PlannerOutput,
    instructions=(
        "You are an architecture review planner. "
        "Given the design doc, choose which review lenses to run, "
        "write scoping notes, and list clarifying questions. "
        "Prefer asking questions over guessing."
    ),
)


@planner.instructions
def planner_design_doc(ctx: RunContext[ReviewDeps]) -> str:
    return f"Design doc to review:\n\n{ctx.deps.design_doc}"


def make_specialist(lens: Lens) -> Agent[ReviewDeps, SpecialistOutput]:
    specialist = Agent(
        DEFAULT_MODEL,
        deps_type=ReviewDeps,
        output_type=SpecialistOutput,
        instructions=(
            f"You are the {lens.value} specialist for an architecture review.\n"
            "Return findings using the output schema.\n"
            "- Every finding should include concrete evidence tied to the input.\n"
            "- If you lack evidence, ask a question instead of inventing facts.\n"
            "- Be concise; prioritize the highest-impact issues."
        ),
    )

    @specialist.instructions
    def specialist_design_doc(ctx: RunContext[ReviewDeps]) -> str:
        return f"Design doc to review:\n\n{ctx.deps.design_doc}"

    return specialist


synthesizer = Agent(
    DEFAULT_MODEL,
    deps_type=ReviewDeps,
    output_type=ArchitectureReview,
    instructions=(
        "You are the synthesizer for a multi-agent architecture review.\n"
        "Merge specialist outputs into one ArchitectureReview:\n"
        "- Deduplicate overlapping findings.\n"
        "- Rank by severity and impact.\n"
        "- If specialists disagree, either resolve via evidence or preserve uncertainty.\n"
        "- Keep the summary consistent with the structured findings."
    ),
)


@synthesizer.instructions
def synthesizer_design_doc(ctx: RunContext[ReviewDeps]) -> str:
    return f"Design doc that was reviewed:\n\n{ctx.deps.design_doc}"

...

Routing Topology: Fan-Out, Sequence, and When to Stop

With roles defined, the next question is routing: in what order do you run them, what state do you pass, and when do you stop? “Agent frameworks” often treat routing as a generic problem (graphs, tool routers, memory stores). Architecture review is narrower. You want a topology that is predictable, auditable, and cheap enough to run often.

There are three common patterns that fit this use case:

Fan-out then synthesize (standard)

Planner → Specialists (parallel) → Synthesizer.

This is usually the sweet spot. The planner scopes and selects lenses, specialists run independently in parallel, and the synthesizer merges. Parallelism gives you speed and reduces the chance that one lens anchors another. The cost is duplication and conflict, which you then handle in synthesis.

Gated sequence (when context is expensive)

Planner → Specialist A → Specialist B → … → Synthesizer.

Sequential routing makes sense when later steps depend on structured state produced earlier (e.g., the planner extracts a component inventory; specialists review component-by-component). The risk is error propagation: a missed component early can cause systematic blind spots.

Two-pass loop (only when you need it)

Planner → Specialists → Synthesizer → (optional) targeted re-asks → Synthesizer.

If you do loops, keep them narrow. The goal isn’t “let the agents think longer”. It’s to repair specific defects: missing evidence, unclear severity, or unresolved contradictions. Targeted re-asks are cheaper and more reliable than open-ended “review again”.

State passing: structured, not conversational

The “chat history soup” failure mode is real: you pass the entire transcript to every agent and hope they find what they need. The result is inconsistent emphasis and increasing token waste. For architecture review, treat state as data:

The raw input (design doc excerpt, constraints, assumptions).
A structured planner output (selected lenses, clarifying questions, scope notes).
A shared contract for specialist findings.
The synthesizer’s merged artifact.

This keeps each step anchored to the same fields, and it makes runs auditable: you can see which agent produced which finding and why.

Stopping conditions

In a lean system, stopping conditions should be boring and strict. Common rules:

Max rounds: no unbounded loops; if you need a third pass, you likely need a better contract or better inputs.
Empty findings: if specialists return no issues, don’t “try harder” unless the planner flagged missing context.
Low-confidence signals: if evidence is missing, prefer explicit questions/unknowns over additional speculative rounds.

Reference topology

The reference topology we’ll use in the end-to-end walkthrough is:

Planner → (Security, Scalability, Operability, Data/Integrity) specialists in parallel → Synthesizer.

If you add anything, add it reluctantly - and only after you can name the specific failure mode it fixes.

I review Python and AI codebases for security gaps, production readiness, and long-term maintainability. If that's something your team needs, let's talk.

Code example: orchestration (planner → specialists → synthesizer)

The same example wires the routing logic directly in Python, passing structured state (planner output, specialist JSON) rather than a growing chat transcript:

...

def run_review(design_doc: str) -> tuple[PlannerOutput, list[SpecialistOutput], ArchitectureReview]:
    deps = ReviewDeps(design_doc=design_doc)

    plan = planner.run_sync(
        "Plan the review lenses and questions for this design doc.",
        deps=deps,
    ).output

    specialist_outputs: list[SpecialistOutput] = []
    for lens in plan.lenses:
        specialist = make_specialist(lens)
        specialist_outputs.append(
            specialist.run_sync(
                (
                    "Review this architecture with your lens. "
                    f"Scope notes: {plan.scope_notes or '(none)'}"
                ),
                deps=deps,
            ).output
        )

    synthesis_input = (
        "Synthesize the final ArchitectureReview from:\n\n"
        f"Planner clarifying questions:\n- "
        + "\n- ".join(plan.clarifying_questions or ["(none)"])
        + "\n\nSpecialist outputs:\n"
        + "\n\n".join(o.model_dump_json(indent=2) for o in specialist_outputs)
    )
    final = synthesizer.run_sync(synthesis_input, deps=deps).output
    return plan, specialist_outputs, final

End-to-End Example: One Review from Design Doc to Structured Output

An architecture review only becomes real when you can run it on a concrete input and get a structured artifact out the other side. This section sketches a single “happy path” run.

Walkthrough inputs

At minimum, you want three kinds of input:

Architecture description: the system diagram in prose - components, dependencies, data flows, and boundaries.
Constraints: what is non-negotiable (compliance, latency targets, cloud restrictions, tenancy model).
Known risks / focus areas: what the team is already worried about (migration, multi-region, PII, cost).

The biggest determinant of review quality is whether these are explicit. If the input does not state assumptions, the reviewer will either guess (bad) or ask a lot of questions (good but slower). Your contract should reward “ask a question” rather than “invent a fact”.

Reference implementation sketch

Conceptually, you define three things:

Schemas (Pydantic models) for the planner output, specialist findings, and the final review artifact.
Agents bound to those schemas (planner agent, specialist agents, synthesizer agent).
Orchestration that routes structured state between them.

Claude is the model behind each agent; PydanticAI is the layer that forces responses to fit the schema and provides retries/repairs when they don’t.

Clone the repository, set ANTHROPIC_API_KEY, and run python run_review.py to reproduce the walkthrough below.

Happy path routing

The run looks like this:

Planner call: read the input, select lenses (e.g., Security, Scalability, Operability, Data/Integrity), and emit clarifying questions if required.
Specialist fan-out: run each lens with the same input plus planner scope. Each specialist emits a list of Finding objects (and questions/unknowns if the contract supports them).
Synthesizer merge: merge the lists into a final ArchitectureReview artifact: dedupe, rank, and normalize severity.

If you log inputs and outputs at each step, this pipeline is easy to debug: you can see whether the planner scoped incorrectly, whether a specialist missed evidence, or whether synthesis merged incorrectly.

Example input and output

The final artifact should be something you can paste into a PR comment and parse as data. A good output has:

A short summary (what’s good, what’s risky).
Ranked findings with clear severities and categories.
Evidence tied to the input (or explicit “inference” flags).
Actionable recommendations (what to change, what to measure, what to decide).
A small set of clarifying questions that genuinely block conclusions.

The companion repository includes a toy design doc at sample_design.md - short on purpose, but enough for specialists to anchor findings to real statements:

# Example design doc (toy)

We are building a multi-tenant SaaS API that ingests events from customer apps.

## Components

- Public REST API behind an API gateway.
- Worker service that processes events asynchronously.
- Postgres for tenant metadata and configuration.
- S3 for raw event payload storage.
- Redis for rate limiting and job deduplication.

## Constraints / assumptions

- Tenants are identified by an API key.
- Peak: 50k events/sec across tenants, spikes up to 5×.
- PII may be present in event payloads.
- 99.9% availability target for ingestion endpoint.

## Known risks / focus

- We previously had incidents from retry storms.
- We need to support data deletion by tenant (GDPR-style).

Running python run_review.py against that file produces out_review.json. Here is an excerpt of the structured review (truncated for readability):

{
  "summary": "This multi-tenant SaaS event-ingestion platform has a **high overall risk profile** driven by four converging concern areas: (1) an undefined API key lifecycle… (2) absent tenant isolation controls… (3) PII in S3 with no encryption-at-rest strategy… (4) a GDPR deletion requirement spanning Postgres, S3, and Redis with no coordination mechanism…",
  "overall_risk": "high",
  "findings": [
    {
      "title": "GDPR Deletion Lacks Cross-Store Atomicity, Completeness Guarantee, and Audit Trail",
      "category": "compliance",
      "severity": "p0",
      "evidence": [
        {
          "kind": "quote",
          "detail": "'We need to support data deletion by tenant (GDPR-style)' is listed as a known risk but no deletion workflow, ordering, or rollback strategy is documented."
        },
        {
          "kind": "observation",
          "detail": "Data is spread across three independent stores — Postgres, S3, and Redis — with no described coordination mechanism."
        }
      ],
      "recommendation": "Implement a saga/orchestration pattern for tenant deletion that tracks per-store deletion state…"
    },
    {
      "title": "API Key Lifecycle Management Undefined — No Rotation, Revocation, or Scoping Controls",
      "category": "authn",
      "severity": "p1",
      "evidence": [
        {
          "kind": "quote",
          "detail": "Tenants are identified by an API key — no mention of key rotation, revocation, expiry, or scoping anywhere in the design."
        }
      ],
      "recommendation": "Implement a full API key lifecycle: scoped creation, server-side HMAC-SHA256 hashing, rotation policies, and immediate revocation propagation via Redis…"
    }
  ],
  "questions": [
    "What queue technology sits between the API gateway and the worker (SQS, Kafka, RabbitMQ, etc.)?",
    "…"
  ]
}

Notice how each finding carries explicit evidence kinds (quote, observation, inference) and a ranked severity - exactly the shape your contracts enforce, and exactly what makes the output routable into a PR comment or issue tracker without a second translation pass.

What did that run cost? On this toy doc, a full pipeline (planner + four specialists + synthesizer, using claude-sonnet-4-6) came to about $0.45. That’s reasonable for an occasional architecture review on a real design doc; it’s expensive if you run it on every small PR. Treat this as one data point - cost scales with document length, lens count, validation retries, and model choice - not a fixed price tag. It’s another reason to keep the topology lean and scope lenses deliberately.

The point isn’t the exact field names; it’s that the artifact can be routed into real workflows without a human reformatting it.

Failure Modes: Hallucinations, Conflicts, and Unknowns

Architecture review is a high-trust activity. When a human reviewer says “this will fail under load”, you can ask why, argue about assumptions, or request a benchmark plan. When a model says it, you get a different problem: the statement is often well-phrased but its epistemic status is unclear. Is it anchored in the input? Is it an inference? Is it a generic warning? If you don’t design for that, you’ll end up with a reviewer that either hallucinates confidently or hedges uselessly.

This section is a set of failure modes worth designing against up front.

Unsupported claims: make evidence a first-class field

The simplest guardrail is structural: require each finding to carry evidence. “Evidence” can be a quote, a reference to a section of the input, or a concrete observation about the described architecture. If the input does not contain enough evidence, the finding should not pretend otherwise - it should downgrade severity or convert into a clarifying question.

This one constraint changes behavior. Models are much less likely to invent specifics when they must attach them to an evidence slot. And when they do invent, it becomes visible: the evidence field will be empty, vague, or obviously unrelated.

Specialist disagreement: preserve dissent when it matters

Parallel specialists will disagree. Sometimes that’s a bug (one misunderstood the architecture). Sometimes it’s the point (tradeoffs are real). Synthesis should not always force consensus. A useful pattern is:

If the disagreement is resolvable by the input, resolve it and cite the evidence.
If the disagreement is resolvable by a missing fact, emit a question and present the conditional conclusions (“If X, then P0; if not, then P2”).
If it’s a genuine tradeoff, preserve dissent explicitly and explain the consequence of each choice.

The goal is not to “sound decisive”. It’s to help engineers decide with clarity about what hinges on what.

Unknowns: treat “needs human” as a valid outcome

Most model failures under architecture review are failures of uncertainty handling. The model would rather guess than admit it doesn’t know. Your contract should give it a safe place to put uncertainty: unknown, assumption, or needs_human fields that are treated as valid outputs, not errors.

This is also where you differentiate between “missing input” and “non-determinism”. Missing input can be fixed by asking a question. Non-determinism might require a benchmark, a threat model, or a human policy decision. Your reviewer should surface that explicitly instead of burying it in hedged prose.

Guardrails without over-engineering

You don’t need a full evaluation harness to be safer than the average demo. A few cheap guardrails go a long way:

Schema constraints: enums for severity/categories; required evidence fields; bounded list sizes.
Rubric checks: simple consistency rules (“P0 findings must include a clear blast radius and an action”).
Spot re-asks: targeted second-pass prompts when specific fields are weak (“rewrite evidence”, “justify severity”, “convert speculative claims into questions”).

The point is to fix predictable defects deterministically, not to create an open-ended “think harder” loop.

What to log (so you can debug)

If you deploy this, you want logs that help you answer: “which step failed, and how”? At minimum:

The input digest (so you can correlate runs without storing sensitive docs verbatim).
Planner output (selected lenses, questions, scoping decisions).
Each specialist’s structured findings (including validation failures/retries).
Synthesizer merge decisions (deduping and any conflict resolution notes).

With that, you can debug multi-agent runs like any other pipeline: identify the step that produced bad data, tighten the contract or prompt for that role, and move on.

Operational Heuristics: Prompt Pack and Debugging

If you treat your architecture reviewer like a one-off prompt, it will behave like one. If you treat it like a component in an engineering system, it becomes maintainable.

Build a “review prompt pack”

Instead of hand-editing prompts in code, keep a small prompt pack with:

The role definitions (planner, each specialist, synthesizer).
Your rubric snippets (what counts as severity P0/P1/P2, what categories you care about).
One or two output examples that demonstrate the contract “done right”.

This does two things. First, it creates a shared artifact for the team - people can review and improve it like any other engineering asset. Second, it makes drift obvious: if the output starts violating the rubric, you can update the pack instead of chasing ad-hoc prompt edits scattered through the code.

Version your contracts like an API

Once downstream systems depend on your schema, it becomes an API. Treat it that way:

Make breaking changes intentionally (renames, enum changes, required fields).
Consider adding a schema_version to the top-level review artifact.
Keep migration logic simple: prefer additive changes early, and prune later once consumers catch up.

Most failures in production-like agent systems aren’t “the model got dumber”. They’re “the contract moved and the assumptions didn’t”.

Debugging checklist: contract vs. reasoning vs. routing

When something goes wrong, you want a fast way to localize the problem:

Contract failure: validation errors, missing fields, wrong enum values. Fix with stricter schemas, clearer instructions, or repair prompts.
Reasoning failure: the model followed the schema but produced low-quality findings. Fix with better rubric, better lens prompts, and better evidence requirements.
Routing failure: the right work didn’t run (wrong lenses selected), or state was passed incorrectly. Fix the planner logic and the state model; don’t patch around it in specialist prompts.

This is why structured state passing matters: you can inspect each stage and see whether the pipeline is broken structurally or semantically.

Keep it lean: don’t add features until you feel pain

It’s tempting to add memory, retrieval (RAG), tool routers, and evaluation harnesses immediately. Most of that is premature for a reviewer that’s still proving it can produce a single reliable structured artifact.

Add only what fixes a named problem:

Add memory when you have multi-step interactions that truly benefit from long-lived context.
Add evaluation when you’re shipping frequent prompt/contract changes and need regression protection.
Add retrieval when your reviewer needs access to external specs, policies, or service inventories that are too large to paste into the input.

Extend one lens at a time

The clean way to extend this system is to add a specialist, not to bloat existing ones. If you want a “Compliance” lens, define:

A new role prompt for compliance.
The same output contract as the other specialists.
A planner rule for when to include that lens.

Because the contract is stable and synthesis already knows how to merge, you get extensibility without rewriting orchestration.

Closing: From Demo to Engineering Workflow

The pattern in this article is deliberately simple: contracts + roles + a boring topology. That combination is what turns “LLM feedback” into something you can actually integrate into engineering work.

Contracts are the differentiator. They force the reviewer to produce findings as data, not prose. Roles keep each agent honest: the planner scopes, specialists apply lenses, and the synthesizer merges into a single artifact. The topology stays lean so you can run it often and debug it when it misbehaves.

The practical question is where to plug this in. Architecture review is not a single event; it happens at different points in a system’s lifecycle. A structured reviewer can support a few common workflows:

Design reviews: run it on a design doc draft to surface missing assumptions and obvious risks before a meeting.
PRs for architectural changes: attach the structured artifact as a PR comment, with a short summary plus ranked findings.
ADRs: use the questions and “needs human judgment” fields to drive what the ADR must explicitly decide.

The point is not to replace humans. It’s to make the review loop tighter and more consistent - and to ensure the output is shaped like something your team can act on.

If you outgrow the lean version, the next steps are straightforward:

Add an evaluation harness with a small set of “golden” design docs and expected findings, so prompt/contract changes don’t regress silently.
Add organization-specific retrieval (policies, SLO templates, service inventories) when you repeatedly see “unknown” due to missing institutional context.
Expand lens coverage one specialist at a time, keeping the contract stable.

The best call to action is also the simplest: ship the smallest reviewer that returns structured findings with evidence. Run it on one real design doc. If the output is useful, you’ll know exactly what to improve next. If it isn’t, don’t add more agents - tighten the contract and the inputs until it becomes reliable.

Try the code: github.com/nunombispo/multi-agent-architecture-reviewer-article - clone, point run_review.py at your own design doc, and iterate on contracts and lenses from there.

Want to sharpen your Pydantic skills? This article leans on schemas as the backbone of agent reliability. For a full treatment of validation, serialization, and real-world Pydantic patterns, check out Practical Pydantic: The Missing Guide to Data Validation in Python on Leanpub.

Follow me on Twitter: https://twitter.com/DevAsService

Follow me on Instagram: https://www.instagram.com/devasservice/

Follow me on TikTok: https://www.tiktok.com/@devasservice

Follow me on YouTube: https://www.youtube.com/@DevAsService

uv for Faster Teams, Fewer Environment Fires

Developer Service — Mon, 01 Jun 2026 06:42:54 +0000

Python dependency management is not a developer problem. It is a team productivity problem that shows up as slow CI, painful on-boarding, and a different tool in every repository.

If you run a Python shop with five to thirty engineers, a measurable slice of your payroll quietly funds work that never ships: waiting on dependency installs, re-explaining local setup to new hires, and arbitrating which package manager each team decided to use this quarter. None of it shows up on a road-map. All of it compounds.

uv is a Python package and project manager that replaces pip, pip-tools, and Poetry with a single, faster workflow.

This article is not a tutorial. It is a business case: what standardizing on uv costs, what it returns, and how to run a one-week pilot without stopping feature work. By the end, you will have a go/no-go framework and a minimal team policy you can paste into your engineering handbook today.

The Hidden Tax

Most of this work is invisible. There is no ticket for "waited on a slow install". There is no incident report for "new hire lost two days to a broken environment". It just disappears into the sprint - and the next sprint, and the one after that.

Those costs are not developer preferences. They are line items that nobody is measuring.

Slow CI: On many teams, the dependency install step is the longest part of a pull-request build. Every push pays that cost again. If installs take 10 minutes and you merge 20 PRs a week, you are buying a part-time engineer whose job is watching progress bars.

Onboarding drag: Picture a new hire on day one. Clone the repo, follow the README, hit a version mismatch, ask in Slack, get three different answers, open a doc that was true six months ago. By Thursday they might have a green test run. You have already spent senior time you could not spend on the road-map.

One of the simplest ways to make on-boarding real is to make the “first green test run” path as simple as a single copy-paste:

git clone git@github.com:your-org/your-repo.git
cd your-repo
uv sync
pytest

Tool sprawl: One service pins with pip-tools. Another uses Poetry because a contractor set it up. A third has a Makefile that wraps both. Your CTO job is not to pick the prettiest CLI; it is to know whether your organization can reproduce a production-like environment on demand. When every repo invents its own workflow, you are paying coordination tax on top of everything else.

None of this requires a dramatic failure. It is the steady leak that makes "we are too busy to change tools" sound reasonable, even while the leak itself burns the capacity you are protecting.

The fix is not a weekend rewrite of every repository. It is a policy decision: one default way to create environments, install dependencies, and lock what CI runs. That is where uv enters the conversation - not as hype about Rust, but as a way to stop re-litigating the same afternoon in every channel.

The next section is about what actually changes when pip is no longer the unnamed default on your team.

What Changes When Pip Isn't the Default

Your engineers will talk about uv in terms of speed. You should hear something else: one reproducible workflow across repositories.

At a high level, uv is a single tool-chain for creating virtual environments, resolving dependencies, and installing locked versions. Faster resolves matter because they shorten CI and local setup. Reproducible environments matter because they reduce "works on my machine" and make audits easier. One tool-chain matters because you stop paying the meeting tax every time a repo picks a different package manager.

That is the executive translation. You do not need to understand Rust or memorize flags to approve the direction. You need to know that the team can answer three questions the same way everywhere: how do we create an environment, how do we install what this commit expects, and how does CI prove we did?

You will still hear enthusiasm about performance. Treat it as supporting evidence, not the decision criterion. A startup CTO standardizes on uv when the organization needs a default, not when a benchmark chart wins an argument.

What stays the same is larger than what changes. Your application code is still Python. Your runtime version policy is still yours. Most CI pipelines keep their shape: checkout, install dependencies, run tests. You are usually swapping the install step and standardizing on a lock-file in git, not re-platforming the product.

What changes is the argument surface. Teams stop debating pip versus Poetry versus pip-tools on every new service. Code review stops treating "how we install" as a local choice. On-boarding docs can say one thing. When production misbehaves, you have fewer variables.

The practical policy looks boring on purpose: uv is the default for new Python work; existing repos adopt it when touched or during a time-boxed pilot; lock-files are required; CI uses a frozen install so what merged is what ran. For commands and platform-specific setup, the official uv documentation is the reference. Your handbook holds the rules.

The next section turns to evidence: which metrics to capture before and after so you can tell whether the pilot earned its half-day of attention.

The Productivity Case

Treat uv like any other productivity change: measure before, run a small pilot, then decide with data. If you cannot point to a number, you will default to opinion. And in tooling debates, opinions are loud.

Start with three metrics that map cleanly to cost:

CI dependency time: how long the install step takes on a cold cache (and how often caches miss). If installs are ten minutes and you run fifty builds a week, you are spending roughly eight engineer-hours a week just waiting for dependencies.
Cold clone-to-test time: on a clean machine, how long it takes to go from git clone to a green test run. This is onboarding time, but it also shows up every time a laptop is replaced, a new service is pulled down, or someone needs to reproduce a bug quickly.
Environment support load: count tickets or Slack threads tagged “env”, “pip”, “poetry”, “dependency”, “won’t install”. You do not need perfect accounting. A rough baseline is enough to see direction.

Now set expectations like a CTO, not a benchmark blog post. In a 5–30 engineer shop, you are usually chasing a combination of:

Faster installs (minutes back per build)
Fewer broken local environments (fewer interruptions)
Less variance between repos (less cognitive load)

Set a go/no-go before you start:

Decision: roll out to new repos if the pilot clears either threshold below
CI threshold: saves ≥ 10 minutes per PR in the dependency step
Time threshold: recovers ≥ 10 engineer-hours/month in avoided environment work

If you get even 1–2 minutes back per build on the dependency step and cut a couple of “why won’t this install” interruptions per week, the ROI can be real. The pilot cost is also real, but bounded: a half-day to wire one repo end-to-end and document one workflow. Do the math in hours, not gut feel.

Be honest about when this will not matter. If your Python footprint is small, your CI time is dominated by integration tests, or your dependency set rarely changes, switching tools will not move the needle. In those cases, keep your current setup and spend the time where it buys you more.

If the numbers suggest this is worth it, the next question is execution: how to adopt without turning it into a migration project.

Adopt Without a Migration Project

The easiest way to fail a tooling change is to turn it into a program. The fastest way to succeed is to treat it like an experiment with a tight box around it.

If your first reaction is “we don’t have bandwidth,” take that seriously. But also notice what it implies: you are already paying the bandwidth tax, just in a less visible form. The goal of a uv rollout is not purity. It is a measurable reduction in time spent on installs, environment drift, and repo-by-repo debate.

Run a one-repo, one-week pilot. Do not start with the cleanest service. Pick the repo that creates the most noise: the CI job everyone waits on, or the codebase that makes onboarding a Slack scavenger hunt. If uv can help there, it will help anywhere.

Keep the steps boring:

Add or standardize pyproject.toml and commit the lock-file you want CI to honor.
Swap the dependency install step in CI to use uv and make it “frozen” so CI fails if the lock-file is stale.
Write a short local workflow in the README: install uv, sync dependencies, run tests. One page. No tribal knowledge.

In GitHub Actions, the “swap the install step” part can be as small as this:

- name: Install dependencies (locked)
  run: uv sync --frozen

Define “done” before you start. Good definitions of done are observable:

CI is green and the dependency step is shorter (or at least more stable) on cold caches.
One teammate who did not touch the pilot can clone the repo and run tests on a clean machine in under X minutes.
You can point to a single default command for “get me to a working environment.”

Also define the rollback in one sentence. This lowers the political risk of trying something new. For example: “If the pilot burns more than half a day of engineer time or destabilizes CI, we revert the CI install step and keep the lockfile as-is.” You are buying information, not committing the company.

If the pilot works, resist the urge to immediately migrate every repo. The value of the pilot is not the migration - it is the evidence and the template. Capture both, then write a short policy that makes the choice automatic for every new repo from here on.

A Minimal uv Team Policy

If you want the benefits of uv without turning it into a prolonged debate, write down a minimal policy and enforce it in CI. The policy is not “everyone must love this tool.” The policy is “we have one default workflow and CI proves it.”

Here is a paste-ready version you can drop into your engineering handbook or an ADR.

Policy (minimal)

Default package workflow: For Python repos, uv is the default tool for creating environments and installing dependencies.
Lock-files are required: Dependency changes must update the lock-file and commit it to git.
CI is authoritative: CI installs dependencies from the lock-file in a frozen mode. If the lock-file is stale, CI fails.
Local workflow matches CI: The same “sync from lock-file” approach is the expected developer path.

If you want a single line to standardize on in CI, use:

uv sync --frozen

(Link to the official uv documentation for the exact installation steps and any platform-specific details. Your handbook should own the rules, not the flags.)

Onboarding checklist (one page)

Install uv
Clone the repo
Sync dependencies from the lock-file
Run tests

That is the whole point: fewer hidden steps and fewer Slack rituals.

Governance (light-touch)

Exceptions: Any repo can opt out temporarily, but the exception needs a named owner and an expiration date. Otherwise “temporary” becomes permanent.
New repos: Provide a template repo (or a copy-paste snippet for CI) that already follows the policy, so teams do not reinvent it.
Decision log: If your team uses ADRs, write a one-page “Why uv?” note that captures the motivation (cycle time, reproducibility) and the rollback trigger.

If you want to make that decision durable, use a tiny template:

## ADR: Standardize Python dependency workflow on uv

- **Status**: Proposed / Accepted / Rejected
- **Why now**: CI minutes + onboarding time + tool sprawl
- **Decision**: uv is the default; lockfiles required; CI uses `uv sync --frozen`
- **Pilot**: repo = `your-repo`; start = 2026-05-27; success metric = “save 10 minutes per PR in CI dependency step”
- **Rollback**: if CI becomes flaky or the pilot consumes > 0.5 day of engineer time, revert the CI install step and stop the rollout

This is what standardization buys you: fewer repo-by-repo decisions, fewer environment surprises, and a workflow you can explain in five minutes to a new hire. uv is not the only way to get there, but right now it is one of the lowest-friction ways for Python teams to make the default path fast and consistent.

Run the one-week pilot. Measure one metric you care about. If the numbers agree, expand. If they do not, stop and move on.

Conclusion

Python tooling debates are a proxy for a deeper question: does your team have a default, or does every service get to decide? uv does not answer that question for you. It lowers the cost of picking an answer.

The pattern is simple. You are already paying for slow installs, broken local environments, and tool sprawl - you are just paying in ways that do not show up on a dashboard. A half-day pilot on one noisy repo will tell you whether the numbers justify expanding. A short policy will tell your team what the default is from here on.

That is all this is: one default, enforced in CI, documented in your handbook. The benefit is not speed for its own sake. It is fewer conversations about tooling and more conversations about the work.

Follow me on Twitter: https://twitter.com/DevAsService

Follow me on Instagram: https://www.instagram.com/devasservice/

Follow me on TikTok: https://www.tiktok.com/@devasservice

Follow me on YouTube: https://www.youtube.com/@DevAsService

DEV Community: Developer Service

I Installed Free-Threaded Python - The GIL Came Back Anyway

Why I cared

Setting up the lab

What I measured

The thing that ate my afternoon

Where I landed

Building an MCP Server for Dokku with FastMCP

What MCP actually is

Setup: skeleton and SSH

Tool #1: list_apps

Expand the tool set (and face the parsing problem)

Guardrails: don’t blindly trust AI with infra

Example: from chat to Dokku

Conclusion

Your Voice Pipeline Is Too Flexible

Staging lied about the mic

Don’t ask one prompt to do two jobs

Name what each stage owns

The contract is the spine

When speech is wrong

Try it

What this isn't

The rule, twice

Your Data Structure Is Too Flexible

Staging lied (inbound API JSON)

The cost of flexible-by-default

The rule: strict at trust boundaries

One refactor

What this isn't

Go deeper

The Watchdog Pattern - From ESP32 to Python

Inside the ESP32 Watchdog

Hope Is Not a Strategy

Build It: ESP32 and Python

When Recovery Becomes the Problem

One Pattern, Two Domains

Your Next Breach Won't Be a Hacker. It'll Be an Agent

The Setup: A Real API, A Sandboxed Blast Radius

Turning the Agent Loose

What It Found

Why This Changes the Economics of Attack

What To Actually Do About It

Bottom Line

Building a Weather Home Screen on a Reflective LCD (MicroPython, ESP32-S3)

Friday night: pixels

Why this board

Building st7305.py

Saturday morning: structure

The home page layout

Saturday afternoon: live data

Networking

Config

WiFi

Weather

Main loop

Supporting modules

Sunday: ship it

What it looks like

What you built and what comes next

Tracking YouTube Video Watch Progress with Django

Architecture

Data model

Template

Views

JavaScript

Conclusion

Should Your App Adopt Passkeys?

Plain Terms: Passkeys, Passwords, and MFA

What Passkeys Fix (and Don't)

UX Reality

Readiness Checklist - Score Your App

Show-Stoppers

Pilot, Present, Decide

One-page template for leadership

When to Say "Not Yet"

PydanticAI vs LangChain - Choosing an Agent Framework for Production, Not Demos

What "Production-Ready" Actually Requires

Side-by-Side: The Same Agent, Two Frameworks

LangChain

Tool #1: `list_apps`

Building `st7305.py`