DEV Community: Dani Shemesh

Should you harness the harness: so, should you?

Dani Shemesh — Tue, 22 Sep 2026 14:32:05 +0000

Four parts of machinery. This one is the decision.

It is a yes or no. Either the agent runs your procedure, or your procedure runs the agent. Neither side is only one thing: an orchestrator skill is just the pattern the previous series happened to use, and on the other side a Makefile that calls the coding agent at each step is already harnessing the harness. Nobody sells that one, and it works.

This part names no winner. Tools change every few months. Where a procedure should live does not.

Assumes parts one and two. Three and four are a worked example, and are optional.

What actually justifies the move

First, the ledger. Set the gains out plainly and most of them shrink.

What you gain	What it really is
Parallel nodes	the same work, sooner
Run history, per-node cost	reporting you did not have
Worktree isolation with a real lock	a script you did not get round to writing
Provider and model per node	an option you will rarely exercise
Steps with no model in them	a guarantee you could not previously make
Run state that outlives the process	something that did not exist before

The top four are good reasons to want an engine and bad reasons to change architecture. The bottom two are different in kind, and between them they decide it.

Steps that nothing can skip

Part one established the limit: inside a session no step is one the model sits out, and the decision to reach an exact command is itself a judgment with a failure rate.

Outside, a shell step runs because the program says so. Nothing in the path is capable of skipping it, so nothing does.

So count your steps. Run the migration. Boot the environment. Post the comment. Tag the release. If most of your procedure needs no intelligence at all, you are paying a round trip and a small probability of omission for every one of them, and the omissions are the expensive part, because a step quietly skipped leaves nothing behind that says so.

This is the cheap problem. Moving the procedure out fixes it and buying something does not, so if it is the only thing pushing you, that Makefile is the entire answer.

Run state that outlives the process

Autonomy looks like the argument here: a skill needs a person to start it, so anything unattended needs an engine.

It does not hold. claude -p expands a skill invocation in the prompt string, there is a documented flag for suppressing permission prompts when nobody is there to answer them, and running the whole thing from CI has its own page in the manual.

The gates are easier still. Tell the agent not to stop at them and it will not, with the usual caveat that an instruction to a model is a strong default rather than a guarantee. A cron entry can drive the procedure end to end. Running it unattended may raise subscription and licensing questions, which the last section of this part deals with, but those are billing problems rather than missing capabilities.

So the trigger is not what is missing. What is missing is that the thing you started is a session: it lives in one process, on one machine, and dies with it. Write the same procedure's progress to a database and it becomes an object with a life of its own.

Which buys four different things:

It survives the crash. A failed run resumes from the node that failed, because something outside the process recorded which nodes finished.
It can hold a lock. A run that has reserved a worktree stops a second run walking into it, including a second run of your own.
Anyone can read it. A colleague approving a gate from Slack, a board showing what is in flight, a query asking what ran on Tuesday and what it cost.
It can be replayed. Completed nodes emit events, so a restart consults what actually happened rather than a cursor's opinion about where the run had got to.

So: does anything need to know about a run besides the process executing it? Another person, another run, or the same run tomorrow after a crash. If yes, you need that state somewhere durable, and a Makefile stops being enough.

Adopt, or write one by accident

A CI runner gets you a long way: run ids, readable logs, approval gates, retries, artifacts. What it lacks is the half that knows what a coding agent is:

a session threaded from one step to the next, and forked rather than mutated on a retry;
a worktree lock with an owner and an expiry;
a provider resolved per step.

You can write all of that. Part one already said what happens next, and it is the whole of the second decision: writing that lifecycle well means writing a workflow engine badly, in the margins of doing something else.

So the rule is unglamorous. Build while the missing pieces are ones you can name and would not miss. Adopt once the list starts to look like a product.

In practice nobody arrives at an engine by evaluating engines. They arrive by noticing that their pipeline has grown a state file, then a lock, then a retry policy, then a resume, and asking why they are maintaining all that.

The conversation you give up

The friction is one property rather than a list, and part two already named it: enforceability and steerability are the same thing with opposite signs, and the sentence that used to skip a gate becomes a pull request against a YAML file. Part three is what that felt like day to day.

As a decision criterion it matters more than it sounds, because whoever staffs the evaluation tends to determine its outcome.

	What they see	What they conclude
Whoever owns the process	a procedure that cannot be talked out of halfway through	the fixed control flow is the product
The developer at the keyboard	a procedure that cannot be talked to at all	a straitjacket where a sentence used to do

Neither is wrong. It is one property seen from two chairs, and no amount of tooling resolves it. An evaluation staffed by one group produces a confident answer the other will not honor, which is cheaper to find out before the pilot than after the rollout.

In practice it counted for little. It was annoying and it was liveable. If the next section did not exist, everything in this one would be a list of things to get used to.

The part no engineering fixes

An engine is, by construction, headless third-party non-interactive use of somebody else's coding agent. That is the usage pattern a model vendor has the clearest incentive to price or gate differently, and the first one anybody would restrict.

On 14 May 2026 Anthropic announced exactly that, to take effect a month later. Use of Claude through its own first-party surfaces, the chat apps and the Claude Code CLI you type into, would stay on the flat-rate subscription. Programmatic use would not: the Agent SDK, claude -p, GitHub Actions and third-party apps were to move to a separate monthly credit billed at standard API rates, worth twenty dollars on Pro and two hundred on the largest Max plan, with no rollover.

On 15 June, the day it was due to start, Anthropic paused it: "For now, nothing has changed." Note the wording. Not cancelled, paused, with a revised plan promised and advance notice offered before anything takes effect. That is still where it stands.

What matters is that somebody sat down and drew the line in exactly the place that separates a person typing from a program calling, and then said they intend to draw it again somewhere. It need not be about price next time either. A model tier, a beta, or a rate limit would do.

The usual reassurance is that the provider is a line of configuration, so you can move. That is sound about models and close to worthless about access terms, because every vendor shipping a coding agent faces the same incentive. Switching does not escape a policy they all converge on.

It can also invert the main selling point. Best model per node assumes every vendor's best is reachable from a neutral caller. If one vendor's strongest tier works best through its own harness and another's does too, a neutral engine is not getting the best of both, it is getting whatever each is willing to expose to outsiders.

Vendors draw their boundary between interactive and programmatic, not between skills and engines. A developer running an orchestrator skill by hand is interactive use of one vendor's own harness, which is precisely the usage every vendor is trying to keep cheap. An engine is on the other side permanently, by construction. Fewer capabilities, less exposure, and no way to have both.

If a change in one vendor's terms would strand a process your whole team depends on, nothing inside the tool helps.

The answer

The difference is real. A procedure that runs the agent is enforced in a way a procedure the agent runs can never be, and no amount of careful writing closes that gap.

It is also narrower than it sounds. Part one said what the engine fixes. The price is a process to run, a schema to maintain, a layer between you and the agent you were talking to five minutes ago, and a bet that vendors keep letting other people's harnesses call their models.

The teams who should move are the ones already paying for staying put. A procedure that is mostly mechanical steps is cheaper as a program. Still want to harness? A Makefile will do. A team already maintaining a state file, a lock and a retry policy has written most of an engine and should stop pretending otherwise.

Everyone else is fine where they are, for now. An orchestrator skill is a procedure a model has agreed to follow. That is worth a great deal, and it is not the same as a procedure that runs.

Originally published at fullgc.github.io, where the whole series lives.

Should you harness the harness: inside the Archon engine

Dani Shemesh — Mon, 21 Sep 2026 19:46:40 +0000

The surprise, reading a workflow engine from the inside, is how little of it is about graphs. Topological sorting is a first-year algorithm and it takes maybe fifty lines. Almost everything else is bookkeeping: who holds the lock, what counts as a fatal error, what happens when a process dies without saying so.

That is the honest description of the category. The DAG is the part you would write in an afternoon. The rest is the part you would get wrong for a year.

None of this is required reading. Part three said what Archon is and what it gives you, and that argument stands without any of what follows. Read on if you are building something like it: the lock expiry, the order the error classifier checks its patterns in, and forking a session instead of appending to it are the same three problems in any system where work outlives the process that started it.

Continues from part three, and assumes its vocabulary: command, workflow, worktree, node.

A run, in four phases

Every execution moves through four phases, and the engine is the same code in every case: the front door you came through only decides who gets told about it.

Figure 1: The whole engine on one page. More of it is checks than work.

Phase	What happens
Discovery	YAML files are found, parsed and structurally validated
Routing	the right workflow is selected, by name or by asking a model
Setup	config loads, a run row is created, the path lock is taken
Execution	nodes run in topological layers, concurrently within each layer

Discovery: three sources, three checks

Every YAML file found across the three scopes goes through three checks before it is allowed to exist:

It parses. A file that does not is skipped, not fatal.
Every node validates against a schema, for types, required fields and valid enums.
The graph validates. No duplicate node IDs, no depends_on pointing at a node that is not there, no cycles, and no $nodeId.output reference to a node that does not exist.

The failure mode is the good part. A broken YAML file records a WorkflowLoadError and shows up in /workflow list. It does not abort the discovery pass or block any other workflow from loading. One malformed file breaks one workflow.

Routing: a model that cannot use tools

When a message arrives without an explicit workflow name, the router builds a prompt listing every available workflow with its description, adds platform context such as the issue title, labels and thread history, and asks a model to reply with /invoke-workflow <name>.

Tool use is disabled at the API level with tools: [], so the router cannot make a tool call. It can only emit text. For a component whose entire job is to pick a name out of a list, removing its ability to do anything else is the right call, and it is the kind of thing that only gets done deliberately.

The name it emits then goes through a four-tier fuzzy match: exact, case-insensitive, suffix (plan matches feature-plan), then substring. If everything fails, the system falls back to a general assistant workflow rather than guessing.

Setup: unglamorous and load-bearing

Before a single node runs, setup does seven things:

reads config and merges per-project environment variables;
resolves provider and model;
checks whether a previous failed run on this working directory can be resumed;
creates the run row;
takes the path lock;
makes the per-run artifacts directory;
exposes that directory to prompts as $ARTIFACTS_DIR.

The worktree is created here too, on an auto-generated branch you can override with --branch, and these are git-native worktrees rather than clones. A workflow that does not touch the checkout declares mutates_checkout: false and opts out of both the worktree and the lock guarding it.

Isolation you have to clean up

The part that is easy to skip when building this yourself is the other end:

archon complete <branch> removes the worktree, the local branch and the remote one once a finished branch has been pushed and merged.
archon isolation cleanup sweeps worktrees older than a week.
--merged takes merged branches too, and --include-closed takes abandoned pull requests.

Isolation you cannot clean up is just a slowly filling disk.

Two constraints come with it. One branch maps to exactly one worktree, so the same branch cannot be run twice at once. And a paused sub-run's worktree is reused when it resumes, so cleaning up too eagerly turns a paused run into a lost one.

Execution: layers, then nodes

The executor builds topological layers from the depends_on edges and runs every node in a layer at once. Per node it:

evaluates the when: condition and the trigger_rule join policy;
runs two passes of variable substitution;
executes;
on success, stores the output in an in-memory map and persists a node_completed event.

On a transient error it retries up to max_attempts, twice by default, with a three second backoff. On a fatal error it stops immediately. What counts as fatal is a section of its own.

The node types

Figure 2: The colors are the argument. Only the blue ones put a model in the path.

Type	Category	How it runs
`prompt` / `command`	AI	calls `provider.sendQuery()`; a `command` node loads its prompt from a file in `.archon/commands/`
`loop`	AI	calls the model repeatedly, scanning each output for a completion tag, until it appears or `max_iterations` is hit
`bash`	Shell	`execFileAsync('bash', ['-c', script])`. No model anywhere in the path
`script`	Shell	TypeScript via `bun run` or Python via `uv run`, with `deps:` and `timeout:` support
`approval`	Gate	writes `status: paused`, messages the user, waits for `/workflow approve` or `/workflow reject`
`cancel`	Gate	marks the run cancelled and stops execution, usually on a conditional branch

Every type captures output the same way, into $nodeId.output, which is what lets them compose. For AI nodes it is the concatenated assistant text; for shell nodes it is stdout; for an approval node with capture_response: true it is whatever the human typed.

Prompt and loop nodes also accept a long list of per-node overrides, currently Claude only: model, systemPrompt, allowed_tools and denied_tools, mcp, hooks, skills, agents, effort, thinking, maxBudgetUsd, fallbackModel, betas, sandbox. Capabilities are checked at execution time, and anything the provider does not support emits a warning and is ignored rather than failing the run.

The model resolution chain

The claim part two leaned on hardest was that the invoking session has no say in what serves a node. Here is the chain in full:

Priority	Source
1	`model:` on the individual node
2	`model:` at the top of the workflow YAML
3	`assistants.claude.model` in `.archon/config.yaml`
4	the SDK default
Never	the model of the Claude session that invoked Archon

Provider identity is validated when the YAML loads, so an unknown provider: is a hard error. Model strings are deliberately not validated: they go verbatim to the SDK, which is the actual source of truth for what model names exist. Refusing to maintain a second list is the right call and a slightly uncomfortable one, since a typo surfaces at execution rather than at load.

Context and sessions

Context is threaded rather than accumulated. Sequential nodes on one provider inherit the previous node's session; a parallel layer resets that, because concurrent nodes cannot share one session; and any node can opt out with context: fresh.

Cross-provider handoff has to pass context explicitly, which is what the second substitution pass is for:

- id: plan
  provider: claude
  prompt: "Analyze the repo and plan: $ARGUMENTS"

- id: implement
  depends_on: [plan]
  provider: codex          # different provider, no shared session
  prompt: |
    Implement according to this plan:
    $plan.output           # full text of the plan node, injected at runtime

Pass one substitutes run-scoped variables ($ARGUMENTS, $BASE_BRANCH, $ARTIFACTS_DIR, $REJECTION_REASON and friends). Pass two resolves $nodeId.output for the whole text, or $nodeId.output.field to parse the output as JSON and pull one field out, which is how structured data moves between nodes.

For bash bodies, substituted values are wrapped in single quotes before hitting the shell. Node output is model-generated text going into a shell command, so this is the injection boundary, and it is handled where it should be.

The state machine, and what resume does not restore

Figure 3: The two callouts are the parts you would not write yourself until the day you needed them.

Transition	Trigger
→ `pending`	run created; the row reserves the working directory
`pending` → `running`	dispatcher starts
`pending` → `cancelled`	path-lock conflict, or abandoned before it ever ran
`running` → `paused`	an approval node is reached
`paused` → `running`	`/workflow approve` or `/workflow reject`
`running` → `completed`	every layer finished
`running` → `failed`	fatal error, or retries exhausted
`failed` → `pending`	`/workflow resume`

The database is authoritative rather than a file, which is what allows more than one process to observe and act on the same run. That is the mechanical reason an approval can be answered from Slack while the run itself is on someone's laptop.

Resume replays node_completed events to reconstruct prior outputs and skips those nodes. And here is the honest limitation, documented rather than hidden: resume does not restore the AI session context. Only the $nodeId.output values come back. A node that depended on what a previous node had read rather than what it returned may need to re-read artifacts, and the resume message says so explicitly.

Which is the same trade the previous series arrived at from the other direction. Artifacts survive; conversations do not. An engine does not fix that, it just makes the surviving part durable.

Where the hard calls are

Everything above is what you would expect a workflow engine to contain. What follows is what separates one that works from one that mostly works, and none of it would show up in a feature comparison.

The lock, and the five-minute window

Part one argued that arbitrating two runs against one checkout needs a lock with an owner and an expiry. Here is that lock.

The pending row is the lock token. Before a run starts, the engine checks whether another run is already running or paused on the same worktree, and if so the new run is cancelled immediately with an actionable message rather than queued or silently run anyway.

The expiry is a five-minute stale-pending window, which exists for exactly one situation: a dispatch that crashed between inserting the row and starting work. Without it, one crash poisons that working directory forever. Workflows that do not touch the checkout opt out of both the worktree and the lock with mutates_checkout: false.

Refusing to guess

The rule Archon states most explicitly is a refusal:

When a process cannot reliably distinguish "actively running elsewhere" from "orphaned by a
crash", it does not autonomously mark the run as failed.

Instead it surfaces the ambiguity with actionable options. Only heuristic, recoverable operations (retry backoff, subprocess timeouts) are allowed to bypass this.

This is the single most transferable idea in the codebase, and it has nothing to do with AI. A distributed system that guesses about liveness will eventually kill a healthy run, and the user will not know why. Encoding "I cannot tell, so I will ask" as a design rule rather than an oversight is the difference between an engine you trust and one you babysit.

Three classes of error

classifyError() sorts every caught error into three buckets:

Class	Patterns	Action
Fatal	auth failure, permission denied, credit balance, 401, 403	abort immediately: retrying wastes credits or loops forever
Transient	rate limit, timeout, process exit, 429, overloaded	retry with exponential backoff
Unknown	everything else	retry, but count consecutive occurrences and abort after three

The ordering matters. Fatal patterns are checked before transient ones, so a message containing both, like unauthorized: process exited with code 1, classifies as fatal. Get that precedence backwards and an expired credential turns into a retry loop burning money on a request that cannot succeed.

The unknown bucket retries, because most unknown errors are transient, but it keeps a consecutive counter so an unrecognized permanent failure cannot spin forever. That is a considered answer to "we do not know", rather than picking one of the other two buckets and hoping.

Four channels, and the three that can answer you

A run writes itself down in four places. Event rows go to the database, which is what resume replays. A JSONL file log records the same run linearly. An in-process emitter feeds the web console over SSE. Anonymous telemetry goes to Archon.

Only the first three can tell you why a run is stuck, and they are not redundant: the database has what finished, the log has what was said, the stream has what is happening now. Part three's complaint about having to guess which channel holds the answer is this list seen from the outside.

Figure 4: Error classification, the observability channels, and the two substitution passes.

Forking instead of mutating

Whenever a node resumes an earlier session, the executor sets forkSession: true:

const shouldForkSession = resumeSessionId !== undefined;

Forking copies the transcript before appending rather than mutating the source. The reason is retries: if node B fails and runs again, the second attempt forks node A's session afresh instead of appending to a transcript the first attempt already polluted. Without this, a retry inherits the wreckage of its own previous failure, and nothing in the output would tell you.

An engine that does not import an AI SDK

@archon/workflows imports only from @archon/providers/types, a subpath with zero SDK dependencies. The actual @anthropic-ai/claude-agent-sdk and @openai/codex-sdk packages live exclusively in @archon/providers.

So the workflow engine can be tested with no AI SDK installed at all. That is the boundary doing real work: the thing that sequences steps has no opinion about who executes them, which is the same inversion part two described, enforced at the level of the dependency graph.

What the engineering gets right

None of the decisions above is visible in a demo, and every one of them was somebody choosing the harder correct thing over the obvious one.

The lock has an expiry, because a dispatch that crashes between writing the row and starting work would otherwise poison that working directory permanently.
Fatal errors are checked before transient ones, so an expired credential does not become a retry loop billing you for requests that cannot succeed.
Resume forks a session rather than mutating it, so a second attempt does not inherit the wreckage of the first.
The engine does not import an AI SDK at all, so the thing that sequences steps has no opinion about who executes them.
When liveness cannot be determined, the system refuses to decide and says so.

What follows

All of that is a good picture of a run. Whether it is a good picture of your work is the question the series is named for, and part five answers it without mentioning Archon again: what an engine gives you that no skill will, and the one risk no engineering fixes.

Originally published at fullgc.github.io, where the whole series lives.

Should you harness the harness: what Archon is, and how you use it

Dani Shemesh — Fri, 18 Sep 2026 11:01:50 +0000

Part two defined the category and named two tools in it. This part is one of them from the outside: Archon, the engine we have been evaluating. How a run starts, what composes with what, what the five claims rest on, and what grates.

Assumes part two's vocabulary: node, layer, run state, provider resolution.

What Archon is, and what it used to be

Archon's documentation opens with "a workflow engine for AI coding agents", which is part two's category in Archon's own words. You define a multi-step development workflow in YAML, a code review or a bug fix or a feature, and run it with one command. Each run gets its own git worktree, and it can be started from a terminal, a chat message or a GitHub comment without those being three separate integrations.

Underneath, it is Bun and TypeScript over SQLite or Postgres driving the Claude Code and Codex SDKs, which is part four's subject rather than this one's.

Archon is built by Cole Medin, whose plan-implement-validate loop is the spine of the skeleton in part one of the previous series, and whose context-engineering work sits upstream of that. The procedure the first series described and the engine this one evaluates come from the same person, arriving by different routes.

There is a second piece of history, because Archon has not always been this. It started as a Python agent-builder: a tool for constructing Pydantic AI and LangGraph agents. In April 2026 that codebase was archived and the project rewritten from scratch in TypeScript into what its README calls a harness builder, with the original spun off into its own repository. A good deal of the old vocabulary survived the rewrite, so if you know LangGraph you will keep expecting semantics that are not there.

Having followed Cole Medin's work since the start of the year, which is how I found Archon at all, that rewrite reads to me less like a change of direction than a conclusion. The move is from building agents to constraining them. Once the methodology was written down as prompts, rules and phases, the next step was to stop asking a model to follow it.

How you actually run one

Part two treated "the engine invokes the coding agent" as an abstraction. In practice there are several front doors, and which one you use changes the experience more than it changes the run.

Through the coding agent, via the Archon skill. This is the recommended route and the one that makes the whole thing palatable. The setup wizard copies an Archon skill into your target repository, so you keep working in Claude Code from your own project and simply say what you want. The agent picks a workflow, dispatches it in the background, and reports back. One run reads:

Figure 1: One prompt, nine nodes, twenty-five minutes, and a session that stayed free the whole time. The worktree path is in the header.

That is the coding agent driving the engine that drives coding agents. The session you are typing in never blocks for twenty-five minutes, and the run survives independently of it.

Through the CLI. archon serve starts the web console, archon workflow list shows what is available, archon doctor runs diagnostics. Install is a Homebrew formula, a shell installer, or a clone and bun install.

Through the web console. A Mission Control dashboard with run history filtered by project, status and date, a capacity indicator, per-run duration, and a source column recording whether each run came from the CLI, the web, or a chat platform. Workflows appear as cards you can run directly.

Through a chat platform or a forge. Slack, Telegram and Discord for conversation; GitHub, Gitea and GitLab for issues and pull requests. Each is an adapter over the same engine, which is what part two meant about a run id and an event stream making every surface a client of the same state.

What ships, and what you can add

Nineteen workflows are bundled, and they are not toys:

archon-fix-github-issue runs classify, investigate, implement, validate, PR, review.
archon-idea-to-pr takes a feature idea through the same spine and ends in five parallel reviews.
archon-piv-loop is plan-implement-validate with a mandatory human gate.
archon-comprehensive-pr-review deploys five parallel reviewers with auto-fix.
archon-workflow-builder generates new workflow YAML: the engine writes its own configuration.

Figure 2: Every bundled workflow declares when to use it and, more unusually, what it is NOT for. That second field is routing metadata: it tells the router when to pick something else.

Your own live in .archon/workflows/<pack>/<workflow>/, keeping a workflow's YAML, commands and scripts together in one copyable folder. A same-named file in your repository overrides the bundled default, so customizing archon-piv-loop means writing your own and leaving the name alone.

Commands are the unit of reuse

A command is a markdown file holding instructions for one AI task, and Archon's own framing is the useful one: commands are atoms, each a single focused task with no knowledge of what comes before or after it, and workflows are molecules, YAML files arranging commands into a graph with a purpose.

That separation is what makes reuse possible. archon-investigate-issue is a command, and any workflow that needs an investigation phase references it by name rather than restating it. Change the command once and every workflow using it changes. A workflow, meanwhile, holds no instructions at all: it holds order, conditions and dependencies.

Figure 3: Change the amber file once and three workflows change. A workflow holds no instructions of its own, only which commands and in what order.

Commands compose through artifacts rather than through conversation, and the documented pattern is three lines long:

write the important findings to an artifact;
start the next node with context: fresh;
have that node read the artifact.

Which is precisely the discipline the previous series arrived at by hand, here as the default way the pieces fit together. This is the piece I had underrated.

Workflows, commands and scripts each resolve across three scopes:

Scope	Path	Beats
Bundled	compiled into the binary	nothing
Global	`~/.archon/workflows`, `/commands`, `/scripts`	bundled
Repo	`.archon/workflows`, `/commands`, `/scripts`	global and bundled

Same filename at a higher scope wins, which gives you a personal review checklist available in every project, overridable per repository, without copying anything. It is the packaging problem from part five of the previous series, solved by convention instead of by a manifest.

The marketplace

There is also a marketplace at archon.diy/workflows. Each entry carries a slug, author, description, source URL, a pinned commit SHA, tags and a compatibility range.

Figure 4: The warning is the most important thing on the page.

Pinning to a SHA rather than a branch is the right call for something that executes in your repository, and the listing carries a blunt warning that community submissions are not all audited and you should read the source before installing. That warning is doing real work: a workflow is code that runs with your credentials in your checkout.

The five claims, and what is behind each

Archon's documentation makes five claims for itself: repeatable, isolated, portable, composable, multi-provider. They are the right five, and each one rests on a specific piece of machinery rather than on good intentions. Taking them in turn is the fastest way to see what an engine actually buys.

Repeatable

"Package your best AI coding patterns as shareable YAML workflows."

The sequence is fixed at definition time and nothing at runtime can renegotiate it. Steps that need no intelligence, the bash and script nodes, have no model in the path at all: they run because the graph says so, which means they cannot be skipped, because nothing there is capable of deciding to skip them.

The procedure is repeatable; the output is not. Every AI node is still a language model doing a language model's job, and running the same workflow twice on the same input gives you two different pull requests. What repeats is that both of them were planned, implemented, tested, reviewed and gated in that order. That is a smaller claim than "deterministic AI coding" and it is the one that actually holds.

Isolated

"Each workflow runs in its own git worktree, no conflicts, no mess."

The worktree is the mechanism and the path lock is what makes it trustworthy. Five runs on one repository are five directories on five branches sharing a single object store, and the lock stops two runs occupying one worktree.

Portable

"Run from CLI, Web UI, Slack, Telegram, GitHub, or Discord."

Because the run owns its process and its id, every surface is a client of the same state rather than a separate integration. This is portability of access, not the portability the previous series worried about: it does not make your workflows runnable on a different harness, it makes them reachable from anywhere.

Composable

"Chain nodes into DAGs with dependencies, loops, and conditional logic."

Two mechanisms under one word, and both matter. Reuse: a command is a single-task markdown file that any workflow references by name, so it is written once and changed once. Chaining: every node type, AI or shell or gate, writes to $nodeId.output, so any step can feed any later step, whole or as one JSON field.

Reuse is why you write fewer instructions. Chaining is why the graph is a graph rather than a sequence. Lose either and the other stops being worth much.

Multi-provider

"Works with Claude Code SDK, Codex SDK, and local models via Pi."

Provider and model resolve per node, and each node gets a fresh provider instance, so nothing is shared between them. The chain runs node, then workflow, then config, then the SDK default, and never the model of the session that invoked Archon.

The limit is honest and declared rather than assumed: sessions do not cross providers. Claude supports session resume, Codex and Pi do not, and the engine checks before trying. Cross-provider handoff therefore has to pass context explicitly, which is exactly what the output chaining above is for.

What it is like to live with

Three parts of using Archon that a feature list does not reach, all of them real and none of them why anyone stopped.

You see what the harness decided to show you. Calling this a loss of observability understates it, because by one measure observability improves enormously: an event stream, a run history, status and cost per node, a console, all of which an orchestrator skill lacks entirely. What changes is that every view of the run becomes a rendering somebody else designed.
The cost figure is not the bill. Per-node cost is recorded and shown, but it counts node cost and not the overhead of the coding agent that invoked the engine. That gap can reach forty percent, which is enough to matter when the number is the one you are budgeting against.
The console will date. At the version evaluated it was pleasant to read and unreliable to operate. Viewing was fine; performing operations from it was not.

The first deserves more than a bullet.

Archon streams message chunks to the platform, writes a JSONL log and puts node states in the web console. Those are considered choices about what matters, and they are better organized than a terminal scrollback. They are also a filter. When the thing you need is not in the rendering, you drop to the logs, and the logs are a lower-level rendering rather than the thing itself.

You are watching a rendering in a coding agent session too. The difference is what is rendered: there it is the transcript, and here it is the run, with the transcript demoted to something you go and fetch. Which produces one specific failure mode, and it is the one engineers complained about: it is stuck and I do not know why. In a session you scroll up. Here you go looking, and first you have to guess which of three channels recorded the answer.

None of this was decisive. The friction was real, it was annoying, and it was liveable. What actually decides whether to adopt an engine is not in this list, and it is not about Archon.

What follows

That is the whole of Archon that most people need: what it is, how you start a run, what composes with what, what the five claims rest on, and what the rough edges feel like.

The next part goes underneath, into how a run is actually discovered, routed, set up and executed, and into the handful of decisions that separate an engine which works from one that mostly works. It is skippable, and it is the part I found most worth reading.

Originally published at fullgc.github.io, where the whole series lives.

Should you harness the harness: what a workflow engine actually is

Dani Shemesh — Thu, 17 Sep 2026 20:57:16 +0000

Part one described a procedure that lives inside the agent session, and the limits that follow from it living there. This part is about the alternative, and about how little of what is sold as the alternative actually is one.

A workflow engine is a program that holds the procedure and calls the coding agent when it needs intelligence. That is the whole definition. Everything separating an engine from a skill follows from it, including the two things it will cost you.

Assumes part one's list of limits, and the vocabulary of the previous series: orchestrator, skill, sub-agent, gate, run state.

The inversion

An orchestrator skill is a document the model reads. The model holds the procedure, decides which phase comes next, and issues the tool calls. The procedure is a passenger in the conversation.

A workflow engine turns that inside out. The procedure is a program: it runs in its own process, it holds the control flow, and when it needs intelligence it invokes the coding agent, waits, and moves on.

The agent is no longer the thing running the workflow. It is a subroutine the workflow calls.

Figure 1: The agent is no longer the thing running the workflow. It is a subroutine the workflow calls.

	Orchestrator skill	Workflow engine
Who decides the next step	the model, reading prose	the runtime, reading a graph
Where the procedure lives	a Markdown file, in the session	a validated file, in its own process
Where results accumulate	the conversation	run state on disk
Model and provider	whatever the session is running	resolved per node
Steps with no model in them	none	`bash` and `script` nodes
A run ends when	the process that started it exits	it reaches a terminal state

Everything below follows from that table. Outside the session, which agent serves a step becomes a setting, and the run becomes an object: run six at once, restart one that died, hand a paused one to somebody else.

Archon carries most of the examples below. It is a workflow engine, and the one we have been evaluating; parts three and four cover it properly. Microsoft Conductor was released more recently and looked suspiciously similar.

What the inversion buys

The graph is data, and it is checked before anything runs

A workflow definition describes nodes and the edges between them:

- id: implement
  type: command
  command: build-feature
  depends_on: [plan]

The engine parses that, validates it against a schema, and topologically sorts it to prove there are no cycles. A workflow referring to a node that does not exist fails at load time rather than halfway through a run. The machinery is unremarkable, a schema validator and a standard topological sort, and part four walks through one implementation.

The equivalent in an orchestrator skill is a numbered list in Markdown. Nothing to validate, because there is no schema, because it is prose. A phase referring to an artifact no earlier phase produces reads perfectly well and fails at three in the afternoon.

So the first thing an engine buys is also the least discussed: the control flow becomes something a machine can check.

Parallelism nobody had to design

Because dependencies are declared rather than narrated, the engine can work out which nodes have nothing left to wait for and run all of them at once. Archon computes topological layers and fires each concurrently; Conductor calls the same idea parallel groups.

Parallelism here is a property of having written the dependencies down. An orchestrator skill runs independent phases one after another because its conversation is serial. An engine runs them together, for the same reason a build system compiles independent files together: the graph already says they do not depend on each other.

Figure 2: Nobody designed the parallelism. It is what writing the dependencies down already told you.

The nodes with no model in them

Vocabularies differ across tools; the set does not.

Node kind	What it does	Model in the path
Command	runs a packaged instruction through the coding agent	yes
Prompt	sends an ad-hoc instruction	yes
Bash / script	runs a command or a file	no
Loop	repeats a sub-graph until a check passes	no
Approval	pauses and waits for a person	no
Cancel	stops the run	no

That right-hand column is the determinism argument, and it is easy to overclaim. Part one of the previous series promised scripts over inference: anything decidable deterministically gets decided by a script. That promise was true and smaller than it sounded. The script was deterministic. Reaching it was not, because reaching it meant a model reading a procedure and choosing to make the call.

An engine closes exactly that gap. A bash node runs because the graph says it runs. No prompt, no inference, no decision, and therefore no chance of the step being skipped, because nothing in the path is capable of skipping it.

The command is repeatable, and so is the decision to issue it. That is what these tools advertise about themselves: Conductor keeps no LLM in the orchestration loop, and Bernstein, a fixed pipeline from goal to merge, claims zero LLM tokens spent on coordination at all.

So "deterministic" means something stronger here than it can mean in a skill. In a skill, the commands are deterministic. In an engine, the commands are deterministic and so is the sequence, with intelligence confined to the nodes that asked for it.

The run outlives the session

Part one argued that a workflow run is a state machine whether or not anyone writes one, and that writing your own means writing a workflow engine badly, in the margins of doing something else. This is the part an engine has already written.

A run is a record in durable storage with a status on it. What matters is what putting that status somewhere authoritative allows:

The record can be a lock. A run reserving the working directory it occupies stops a second run entering a worktree that is already busy. That is part six's concurrency limitation, addressed at the only level where it can be.
A paused run outlives its process. An approval node writes the status and stops; another process, a chat command or an HTTP call picks it up later. That is the mechanical reason a gate can be answered from a phone.
A failed run can replay. Completed nodes emit an event as they finish, so a restart consults what actually happened rather than a cursor recording where the run believed it was.
Ambiguity can be surfaced rather than guessed. When the system cannot tell "running elsewhere" from "orphaned by a crash", it can refuse to decide and say so. Easy to state, easy to leave out, and hand-rolled state machines leave it out.

None of this is exotic. It is the ordinary content of a workflow engine, and none of it is reachable from a Markdown file.

A run anything can address

The obvious version of this claim is wrong. A skill can be started without a person: non-interactive mode expands a skill invocation in the prompt string, there is a flag for suppressing permission prompts when nobody is there to answer, and driving the whole thing from a CI job is documented. Triggering is not the discriminator.

What an engine adds is that the thing started has an identity outside the process running it. Starting a run hands back an id and an event stream, and from there:

a colleague can approve a gate from Slack, on a run executing on somebody else's laptop;
a dashboard can list what is in flight without asking the processes;
a second run can discover that the working directory is taken;
a failed run can be resumed by whichever process picks it up.

The platform surface follows from the same fact. Once a run id and an event stream exist, a Slack adapter, a web dashboard and a GitHub integration are all clients of the same state rather than separate integrations. That is a real gain in run-level observability: progress, status and cost per node, history across runs. Whether it is the kind of observability engineers actually missed is a different question.

Provider, model and context, per node

An orchestrator skill runs on whatever the developer's session happens to be using. Its only lever on model choice is pushing work into a dispatched sub-agent that declares its own tier.

An engine settles all three per node:

Model resolves through a priority chain: the node's setting, the workflow's default, then config. Archon states the consequence flatly, that the model of the session invoking the engine has no influence whatsoever on the model serving a node. Conductor supports per-agent overrides across Copilot and Claude.
Provider is the larger version of the same point. Sessions do not cross vendors, so context must be handed over explicitly, but nothing stops you planning on one agent and implementing on another. Mostly that is a bad idea. What matters is that it stops being architecturally impossible and becomes a line of configuration.
Context is threaded deliberately. Sequential nodes on one provider can inherit the previous session; parallel nodes start clean; a node can demand a fresh one. The subtle part is resume: a node continuing an earlier session should fork that transcript rather than mutate it, so a retry cannot corrupt the history it retried from.

The provider argument is also the standard reason an engine is said to survive a vendor changing its terms. It is a real argument about models, and there is a kind of vendor change it does not protect against at all, which the next section opens and part five settles.

What the inversion costs

Neither of these gets better as the tools mature. One is the property the whole argument rests on, seen from the other side. The other is not about the engineering at all.

Enforceability and steerability are one thing

The control flow is data, fixed at definition time, not negotiable by a model. That is the reason to trust it. It is also the reason it is unpleasant to work with.

When the harness is the caller, a developer cannot lean over and say skip this gate, I know exactly what this diff is, or ask me before you push from now on, or try it the other way. The thing that was a sentence in a chat becomes a pull request against a YAML file.

It runs backwards too: the orchestrator skill a developer finds pleasantly steerable is the one a platform owner cannot rely on. One property, two signs, and no amount of tooling resolves it.

Which of the two signs you notice depends on where you sit, which turns out to matter for adopting an engine more than any feature does. Part five takes it up there, as a decision criterion.

The interop has to stay open

An engine calls somebody else's coding agent headlessly, from outside, with nobody at the keyboard. Of every way to use a coding agent, that is the one a vendor has the clearest reason to price or fence off separately. This is not hypothetical: Anthropic has already drafted a policy drawing the line in exactly that place, then paused it before it took effect.

That cuts against the section above. Resolving a provider per node is a real answer to a model being wrong for a step, and no answer at all to a vendor changing who is allowed to call it. Part five has the dates, the amounts, and why switching does not escape it.

None of that is an argument against engines. It is the thing you are accepting when you adopt one, and it belongs on the same page as the benefits.

Where the boundary is

Plenty of things called workflow engines really are workflow engines. n8n is one. LangGraph is one. The question is narrower than that, and it has two parts: does the tool hold the control flow outside the model, and can it run a step in your checkout without a model in the path?

Category and examples	Control flow outside the model	Non-model steps in your repo
Coding-workflow engines: Archon, Conductor, Bernstein	yes	yes
In-harness scripting: Claude Code dynamic workflows	yes	no
Parallel-session supervisors: Claude Squad, Emdash, Vibe Kanban, Baton	no	not applicable
Graph frameworks: LangGraph	yes	only if you build it
Agent-loop builders: CrewAI, AutoGen, LangFlow, Flowise, Dify	varies	only if you build it
General automation: n8n	yes	no
A script or CI pipeline you wrote	yes	yes, up to a point

The top row does both by design. The bottom row also does both, and is not an engine: a Makefile or a CI job calling the coding agent per step has real control flow, real model-free steps, and a run with an id and logs anybody can read. What it lacks is anything that understands coding agents, which is the distinction part five turns on.

Figure 3: Each near miss fails a different axis, which is why the marketing language cannot tell them apart.

The rest of this section is about the rows in between, and why the near misses miss.

The harness grew one of its own

Claude Code's dynamic workflows are the same inversion turning up from the inside: a JavaScript script a runtime executes, where the documentation's own comparison table answers "who decides what runs next" with the script, against the Claude, turn by turn it gives for subagents, skills and agent teams. Anthropic names both halves of the trade itself: built for parallel and long-running work extending into hours and days, and consuming meaningfully more usage than a typical Claude Code session.

Four things keep it a different animal:

The script cannot touch the filesystem or the shell. Agents read, write and run commands; the script only coordinates them. So the control flow became deterministic and the steps did not, which is half the argument above, and the half that costs tokens.
Nothing but a person can start a run.
The lifecycle is scoped to the session rather than durable.
You do not author it. Claude writes the script per task, so what you would want to review, version and constrain gets generated rather than declared.

That makes it the right tool for what it says it is for, a long one-off audit or migration running into the hours, with a warning at twenty-five agents or a projected 1.5 million tokens. It is the wrong one for a pipeline you wrote down, that runs a test suite as a step nothing can skip, and that starts when a ticket moves.

Parallel-session supervisors

Run N agents in N git worktrees, show them on a board or in a TUI, let a human decide what to look at and what to merge. The interface is the only thing that really varies:

Claude Squad: a terminal UI over tmux sessions
Emdash: an Electron app, with ticket intake from Linear, Jira and GitHub
Vibe Kanban: a web board, one worktree per workspace
Baton: a CLI that polls GitHub issues and dispatches

These are genuinely useful, and they are not engines, because there is no pipeline. Coordination is manual, sequencing is a human choosing which session to click, and one worktree per agent is the only structural thing they share with an engine. Augment's survey of nine such tools puts it plainly: coordination is either manual or limited to task-graph scheduling within one repo, and only two of the nine model a multi-step pipeline as a graph at all.

Graph frameworks, and agent-loop builders

LangGraph deserves a fairer description than slide decks usually give it, which cast it as the loose agentic alternative to a rigorous engine. It has typed state, explicit nodes and edges, conditional routing, checkpointers that persist state after every node, and an interrupt primitive for human review. The same primitives listed above.

So it passes the first test comfortably, and fails the second. LangGraph is a library for building a stateful application, with no notion of your repository, your shell, your worktrees, or a coding agent as a callable step. Its checkpointers persist conversational state to Postgres or Redis and know nothing about a working copy. You would not adopt it, you would use it to build the engine, and then you would own the engine, which is the outcome part one warned about.

The agent-loop builders are the largest and most confusing group, because the marketing language is identical. CrewAI, AutoGen, LangFlow, Flowise and Dify all describe themselves in terms of agentic workflows. What they mostly build is a set of cooperating agents and a loop that runs until something decides it is done: AutoGen is explicit that its agents interact in a free-form group chat rather than a fixed sequence. Some, CrewAI in particular, do offer deterministic flows alongside the crews.

But the object under construction is an AI application, and the LangGraph objection lands here too. If your test suite is a shell command that has to run in a checkout, none of them has anywhere to put it.

That is the honest shape of the field as of writing, and the part of this article most likely to age badly. The taxonomy should outlast the names in it.

Originally published at fullgc.github.io, where the whole series lives.

Should you harness the harness: what the orchestrator skill cannot do

Dani Shemesh — Wed, 16 Sep 2026 12:16:16 +0000

A development workflow can be written as a set of skills the coding agent reads and follows: phases in order, artifacts between them, gates that check rather than promise, a script deciding when a loop stops. One skill holds the phase order and dispatches the rest, and that one is the orchestrator. The previous series is a long description of building exactly this.

It works. It also runs inside the agent session, and that is what this series is about.

The harness is the agent runtime itself: the thing that loads the skills, dispatches the sub-agents and runs the tools. Harnessing it means taking the procedure out of the session and putting it in a program that calls the coding agent, rather than a document the coding agent reads.

Whether that trade is worth making is the question these five parts work through. This one establishes why it is a question at all.

A defect, or a limit

Part six of the previous series already lists the things that hurt, and every one of them is a mistake: a gate described in prose instead of enforced by a script, packaged copies that drifted, two components sharing a schema nobody validated. All of them are fixable without changing the shape of the design. Write the missing check, delete the duplicate, add the contract test, and the limitation goes away.

This is the other list. These do not go away, because they follow from where the procedure runs rather than from how well it was written.

The test for whether something belongs here is a single question: could a sufficiently disciplined author fix it by writing better skills? If yes, it is a defect and belongs in part six's list. If no, it belongs here.

That line is a judgment call, and someone else would draw it differently. A later section shows the cases I had to argue myself out of.

What better skills cannot reach

The limit	Why no skill reaches it
Model, effort, provider	only a dispatched sub-agent can declare a tier; the orchestrator runs on whatever the session runs
Its own context window	the thing that would prune the conversation is the conversation
Two phases at once	sub-agents fan out inside a turn, but the conversation itself is serial
Steps that need no intelligence	every step is a tool call some model chose to make
The order itself	phase order is prose, followed because the model is inclined to follow it
A run another process can address	a session id is not a run: nothing else can find it, watch it, pause it or resume it

The first five get a section each below. The sixth gets one of its own further down, because it is the one that decides the question.

Model, effort, and provider

The first principle in part two of the previous series states this as a fact of the runtime rather than a design choice: model and reasoning effort can only be set when a step is dispatched as a sub-agent. A skill's own metadata does not change what serves it.

That is why the orchestrator is thin. It runs on whatever model the developer's session happens to be using, and the only way to say "this decision deserves the strongest model" is to push the decision out into a dispatched agent that can declare a tier.

Turning that into a principle is the right thing to do with a constraint you cannot remove. It remains a constraint. The orchestrator is the one component whose cost and capability you cannot specify.

The larger version is the provider, not the model. An orchestrator skill runs inside one harness and is married to it. Every step is served by whichever vendor's agent the developer happened to open, and if a step would be better served by a different one there is no way to say so, because the thing that would have to say it is a guest in the session it would need to replace.

Once the procedure lives outside the session, the coding agent becomes a parameter. An engine resolves a provider per node, so planning can run on one vendor's agent and implementation on another's, with context handed between them explicitly because sessions do not cross providers.

Whether mixing vendors mid-run is a good idea is a separate question, and mostly it will not be. What matters is that it stops being an architectural impossibility and becomes a line of configuration.

Its own context window

A run is one conversation. Everything every phase reads or writes accumulates in it, and the orchestrator has no programmatic control over the wrapper it runs in. When context needs pruning between phases, the only available move is to stop and ask the developer to type /compact.

The previous series works around this rather than solving it. Dispatched sub-agents get fresh context, which is why "judgment runs in fresh context" is a principle. But the orchestrator's own conversation grows for the length of the run, and the longer the run, the worse its judgment about its own procedure gets.

An engine threads sessions deliberately. Sequential nodes on one provider inherit the prior session, parallel nodes start clean, and a node can opt out with context: fresh. When a node resumes an earlier session it forks it rather than mutating it, so a retry cannot corrupt the transcript it retried from.

None of that can be arranged from inside the conversation. Whatever did the arranging would be part of what it was arranging.

Two phases at once

Parallelism inside a phase works. Dispatch several sub-agents in a single turn and they run concurrently, which is exactly how the review workflow fans its reviewer lanes out.

What cannot overlap is two phases. The orchestrator is one conversation with one turn in flight, so it dispatches a fan-out, waits for all of it to come back, and only then does anything else. Independent phases still run in sequence, and a slow one blocks everything behind it whether or not anything depends on it.

Most of the skeleton genuinely is a straight line, so this costs less than it sounds: you cannot implement before you plan.

An engine computes topological layers from the dependency edges and fires every independent node in a layer at once. That falls out of the control flow being a graph instead of a sequence of prompts.

Steps that need no intelligence

The guarantee table in part one of the previous series promises scripts over inference: anything decidable deterministically is decided by a script. That guarantee is real, and smaller than it sounds.

The script is deterministic. Reaching it is not. In an orchestrator skill there is no such thing as a step the model sits out. Running the test suite means the model reads the procedure, decides this is the moment, and issues a tool call. The command that finally runs is exact and repeatable. Everything upstream of it is a judgment about whether to run it at all.

So a phase requiring no intelligence whatsoever still costs a round-trip, still costs tokens, and still carries some probability of not happening. Individually that probability is small. Across a twelve-phase skeleton it compounds, and it compounds invisibly, because a step quietly skipped leaves nothing behind that says so.

Which is why the previous series ends up recommending a tripwire per phase and a guard on every seam. Those exist to catch a class of failure that only exists because the model is a mandatory participant in its own control flow.

An engine removes the model from the steps that never needed it. A bash or script node runs because the graph says so: no prompt constructed, no tokens spent, no inference, and no way to skip the step because nothing in the path is capable of deciding to skip it. The orchestration layer consumes no tokens at all, and the structure is fixed at definition time.

That changes what deterministic can mean. In an orchestrator skill it means the commands are deterministic. In an engine it means the commands are deterministic and so is the decision to run them.

Figure 1: The command is identical in both. What differs is whether anything had to decide to reach it.

One native mechanism escapes this: hooks fire on runtime events without the model choosing to fire them. But hooks intercept a run, they do not sequence one. They can stop something happening. They cannot make the next thing happen.

The order itself

This is the deep one, and part six of the previous series states it plainly without drawing the full conclusion: an instruction to a language model is a strong default, never a guarantee, and anything load-bearing needs a script behind it.

The design responds by putting scripts behind the load-bearing parts. The gate check is a script. The loop referee is a script. The seam guards are scripts. Each converts one instruction into an enforced rule.

But the order is prose. The list of phases, which one follows which, what happens on a failure: all of it lives in a Markdown file and gets followed because the model is inclined to follow it. You can guard every seam and still have no guarantee the phases ran in the order the document describes, because nothing outside the model is tracking which phase comes next.

The scripts are patches over individual holes in a surface made of prose.

Figure 2: Each script closes one hole. The surface they are fixed to is still prose.

An engine inverts this. The graph is data, parsed and schema-validated and checked for cycles before anything runs. The runtime decides which node executes next, and the model is invoked by a node rather than being the thing that decides there is a next node at all. Enforcement stops being a set of patches and becomes the default state of the medium.

The one that decides it

A workflow run is a state machine whether or not anyone writes one. It is pending, then running. It pauses at a gate and has to still be paused tomorrow. It completes, or fails, or is cancelled, or is abandoned halfway and resumed on Thursday.

Something has to know which of those is true. That something has to survive the process that started it, be reachable by a process that did not start it, and be trustworthy enough that two runs cannot both believe they own the same working directory.

That middle clause is the one that catches people out. A skill run non-interactively has a session, and a session is addressable by whoever holds its id on the machine that has it. A run is addressable by anyone: a colleague approving a gate from Slack, a dashboard listing what is in flight, a second process deciding whether the working directory is free. The difference is not convenience, it is whether the procedure exists anywhere other than inside the process executing it.

Figure 3: The difference is not convenience. It is whether the procedure exists anywhere other than inside the process executing it.

The previous series does implement this:

a run directory per run;
a small JSON state file written atomically, temp file then rename, so a crash cannot leave a half-written record;
a phase cursor, so an interrupted run can resume;
terminal states enumerated deliberately, including awkward ones like done-but-unproven, with a principle requiring a report on every one.

That is a state machine. It is also hand-rolled, maintained by the same people trying to write the workflows, and every gap in it gets discovered in production.

Part six's concurrency limitation is exactly such a gap. Nothing arbitrates two runs against one checkout, because arbitrating that needs a lock with an owner and an expiry, which needs a store, which needs something to stay authoritative when the process holding it dies.

Now look at what a real one contains. An engine's runs live in a database rather than a file, precisely so more than one process can observe and act on the same run. A row that outlives the process holding it can double as a lock on the working directory, be resumed by whoever picks it up, and record what actually happened instead of what a cursor believed. Part two takes each of those apart.

The choice is not between having a lifecycle and not having one. It is between adopting one and writing one, and writing one well means writing a workflow engine badly, incrementally, in the margins of doing something else.

What is merely harder

Some things look structural and are not. Saying so is what keeps the list above credible, and these are the ones I had to argue myself out of.

Looks structural	What it actually is
Triggering	`claude -p "/my-workflow LB-123"` runs a skill non-interactively, so a CI job, a webhook or a cron entry can start one
Isolation	a skill can create a worktree, and the previous series does. What it lacks is a lock, and it could shell out to one
Resume	a run directory and a phase cursor are enough. An engine replaying from an event log is better, not different in kind
Observability	artifacts are readable after the fact. What is missing is a live event stream: a gap in convenience, not capability
Per-step cost	no breakdown by phase, only the provider console. Worth having, not worth changing architecture for

Triggering needs one clarification, because the missing piece is easy to misplace. It is not the trigger, it is the run. A skill started by a cron entry is still a session, and nothing else can find it, watch it, pause it at a gate or resume it tomorrow. That is the last row of the table above.

Isolation is the closest call. A second run entering the same worktree is exactly the kind of failure a skill cannot see coming, but the guard is a script and a skill can call a script. What it cannot do is own the lock, which is the previous problem again.

The cost row has a caveat of its own, and part three has it: what an engine reports per step is not the same as what the run costs you.

What follows

Two of these are the same fact seen from either end. Every step runs through the model because the model holds the control flow, and the control flow is advisory because the thing holding it is a model. Provider selection, context and parallelism trace back to the same root: the orchestrator is a participant in the conversation rather than the thing running it.

The last one is different in kind, and it is the one that decides things. The others say what an orchestrator skill cannot do. That one says what it will make you do instead: maintain a workflow engine you never set out to write.

All of which is a case for looking at a workflow engine, not for adopting one. Before that question can be answered, the category needs a definition, because "engine" has been doing a lot of unexamined work above and most of what is sold under the name is something else.

Originally published at fullgc.github.io, where the whole series lives.

Designing agentic development workflows: where it hurts, and what to build first

Dani Shemesh — Mon, 14 Sep 2026 14:39:10 +0000

Five parts of design, and design flatters itself. This one does not.

What follows is where the shape described so far still hurts, all of it observed in real runs, and then the short list of what to build first if you are starting one. If you read nothing else in the series, read the list.

Follows on from the whole series, but stands alone: every limitation names the thing it broke.

Where it hurts

Honest limitations, all observed in real runs, because a design description without them is marketing.

Enforcement exists only where the check exists. Every rule that mattered and was not a script eventually got skipped: the unchecked gate; the "always write a report" promise that silently covered only the enumerated exits; the "keep going" instruction that lost to an agent's instinct to check in. A single unanswered "shall I carry on?" abandoned a run that was already green, two phases from the finish. Politeness is indistinguishable from a crash. The uncomfortable lesson: an instruction to a language model is a strong default, never a guarantee, and anything load-bearing needs a script behind it.

Figure 1: Every rule that mattered and was not a script eventually got skipped. The amber columns are the ones that had only an instruction behind them.

Uniformity decays. In a family built over months, one workflow guards every seam and another guards none; one dispatches its work as tiered agents while another invokes the same steps inline, where they cannot be tiered, isolated or individually measured. The design stays coherent; the implementations drift from it at different rates, and nothing notices.

Adapters disagree at the schema. One component queried an identifier and never wrote it out; the component that needed it failed for want of it, and failed after the code had already been published. Two components, one implicit schema, no test at the boundary. If two things share a data shape, make the shape explicit and validate it, or it will drift.

History gets overwritten. A loop that rewrites its verdict file each iteration keeps only the final state, so the convergence (twelve failing, then three, then zero) is lost. That trajectory is exactly what you want when asking whether the loop actually works. Append; don't overwrite.

Signals get confused with their proxies. A platform flag meaning "this comment's anchor line moved" was read as "this comment is no longer true". Those are different claims and they came apart badly: items whose anchor had not moved were nonetheless judged against code that no longer existed. Before depending on a platform signal, ask what it literally measures.

Concurrency is limited by the target, not the workflow. Running several instances at once sounds free until you find that the shared working copy is single-occupancy during publishing, that the test database is one container which the first run to finish deletes out from under the others, and that container names are global to the machine. Most of what caps parallelism lives in the system being worked on.

Improvement is genuinely hard to measure. Elapsed time is confounded by caching; token counts by model changes; "number of findings" rewards a noisier reviewer; and an operator who has seen the task before is faster regardless of the tooling. What survives: whether specific, named frictions recur, and counts of expensive operations: environment boots, dependency re-locks, full-suite runs, loop iterations, human waits. Counts cannot be faked by a warm cache.

What no amount of care will fix

Everything above is a defect. Write the missing check, delete the duplicate copy, add the contract test at the seam, and the limitation goes away. The shape of the design does not have to change.

There is a second category that does not work that way, because it follows from running the procedure inside the agent session rather than around it:

The orchestrator cannot choose its own model, its effort or its provider, because a skill's metadata does not change what serves it.
It cannot manage its own context, so its judgment about its own procedure degrades over a long run, and the only remedy is to stop and ask a human to type /compact.
It cannot do two things at once, so a fan-out of parallel reviewers is sequential underneath.
It cannot be started by anything but a person at a terminal, which is the last prerequisite in part five and the one that caps how autonomous a workflow of this shape can become.
It cannot make a step deterministic, only the command inside it. A phase needing no intelligence at all still reaches its script through a model deciding to call it, so what you get is a repeatable command behind an unrepeatable decision: a round trip in cost, and some chance of the step not happening at all.
It cannot avoid having a lifecycle, so pending and paused and resumed and cancelled become yours to implement: incrementally, in the margins of doing something else.

No amount of better skill-writing reaches any of that. Whether it is worth reaching for something that does is a separate question, and a longer one: Should you harness the harness works through it.

If you are building one

The short list, in the order worth learning it:

Put every phase's output in a file, in a run directory, and pass that directory explicitly.
Guard the seams: assert an artifact exists before the phase that consumes it.
Make approvals content-addressed. A gate that trusts a flag is decoration.
Let a script decide when a loop stops.
Dispatch the judgment, inline the machinery: and put your strongest model on the decision nobody downstream can undo.
Prove the tests, not just the code.
Enumerate your terminal states and write a record on all of them.
Forbid the irreversible-in-public actions in code: history rewrites, resolving someone else's thread, merging.
Give "do nothing" a name, a required reason, and a score.
Build skills that run standalone. If a component only works inside its workflow, you have a phase, not a skill, and you will rewrite it for the next workflow.
Start local, with a human at the gates, and treat every friction you hit there as a prerequisite for running unattended later.
Instrument from the start. Every retrospective written by hand should have been a structured record: which component hurt, what it cost, and what the evidence was.
Put a clean control in every corpus, and a bait beside every planted defect, otherwise you are optimizing for a louder agent, not a better one.
Contract-test each seam, and generate the reverse index of which workflows use which component. The blast radius of a shared component is a graph, and it should not live in anyone's memory.
Measure against a no-component baseline. "Did the run succeed?" is the wrong question; "was it better than not having this at all?" is the right one.
Have every dispatched step report its own friction, against the component it invoked: and make "nothing hurt" an explicit record, or you cannot tell better from quieter.
Pick one deliberately trivial task alongside the hard ones. It is the only way fixed overhead becomes visible.
Measure delivery on the delivery systems, not on the workflow's own opinion of itself: and never report a speed number without its quality guardrails.
Build a negative control: one metric the workflow cannot possibly affect. If it improves too, your groups differ for another reason.
Version anything you intend to trend, from the first emission. It cannot be backfilled.
Trace three items by hand before publishing any aggregate. A wrong join produces confident, plausible, wrong numbers.
Treat every piece of incoming text as data. The steps that read external input are the ones holding the tools.
Establish provenance before judging: feedback is a claim about a version, and a platform signal is not the thing you actually care about.
Emit one tripwire per phase: usable output over produced output. Cheapest instrumentation there is, and it catches the failures that return exit code zero.
Package the shared parts once, and let workflow packages depend on them. Duplicating a shared component across packages drifts within weeks, and nothing surfaces it.
Never leave a loose copy of something you have packaged. Same name, same description, different body is undetectable from the outside.
Name actions, not tools, in every skill body: and keep the harness-specific translation in one document per harness. It is the difference between one port and fifty rewrites.
Declare each skill's capability requirements and its degradation. A skill that silently assumes sub-agents exist will invent a tool call on a harness that has none.
Feed it small items. More stories, each narrower: control, comprehension and meaningful gates all degrade with size, and a plan you have to rewrite at the gate means the item was too vague.
Encode your conventions into the steps, and check them. The planning step reads them, validation enforces them, and the negative ones ("don't touch this") are stated explicitly.
Name intents, not commands. The workflow says "run this repository's tests"; the repository says how, which is what lets one workflow serve services in different languages, and what stops a hardcoded test command from failing silently in the next repo.

These are small distributed systems whose components are language models. What makes them trustworthy is files, hashes, exit codes, and a refusal to let any single step be the judge of its own success.

Originally published at fullgc.github.io, where the whole series lives.

Designing agentic development workflows: shipping it, porting it, letting it run

Dani Shemesh — Mon, 14 Sep 2026 14:38:57 +0000

The first four parts describe a workflow that runs on your machine and can be shown to work. This part is about the three transitions that turn it into something other people depend on, and each one costs something specific.

Packaging makes a collection installable and lets one workflow depend on another's components, at the price of a dependency graph nobody can hold in their head. Porting to another harness is cheap for everything that is a file and expensive for everything that is control flow. And autonomy, meaning a ticket assigned to the agent and a reviewed pull request some time later, turns out to be less about capability than about making the exceptional case detectable.

Leans on part one and part two throughout.

Packaging and distribution

At some point this stops being configuration on your machine and becomes something other people install. That step forces a decision the design has been able to defer: what is the unit of distribution?

A note on words, because the two get used interchangeably and they are not the same. This section says package for the general idea, since the argument holds whatever your harness calls it. Every concrete example is a plugin, which is Claude Code's implementation of one. Where something is true only of plugins rather than of packaging in general, it is called out.

What a package can hold

More than skills. A plugin (the packaging format in Claude Code, and the concrete example throughout this section) can carry skills, agents, slash commands, hooks, MCP servers, LSP servers, background monitors, executables that go on PATH while it is enabled, default settings, and a schema for configuration it asks the user for at install time. It can also carry arbitrary payload (dashboards, docs, scripts) and reference its own files through ${CLAUDE_PLUGIN_ROOT}, with ${CLAUDE_PLUGIN_DATA} for state that must survive updates.

That matters more than it sounds. A workflow's whole apparatus can travel as one unit: the orchestrator, the skills it composes, its agents, the hooks that measure it, and the dashboard that reads what they emit. Not just the prompts. The dashboard is the odd one out, riding along as plain files rather than as something the runtime loads, which is what ${CLAUDE_PLUGIN_ROOT} is for. See the plugins guide and reference.

Figure 1: One install brings the whole apparatus. The runtime loads the top row; the bottom row rides along as files, which is why a bundled dashboard has to find itself through ${CLAUDE_PLUGIN_ROOT}.

One package, or one per workflow?

This is a distribution question, not an architecture question, and the answer depends on who consumes it, so the trade-off matters more than any default:

	One package for everything	One package per workflow
Versioning	one number for unrelated changes	independent cadence per workflow
Consumers	everyone gets all of it	a team enables only what it uses
Blast radius of a release	the whole family	one workflow
Shared components	trivially one copy	must live somewhere, the hard part
Hooks	unambiguous	composition across packages is undocumented

Enable/disable is per package, not per component, you cannot ship five workflows in one package and let a team turn four of them off. That alone makes the monolith uncomfortable once more than one team consumes it.

The fact that decides it: packages can depend on packages

A manifest accepts a dependencies array: bare names, or names with a semver range. I verified this against the validator rather than taking it on trust:

{ "name": "workflow-x", "version": "1.2.0",
  "dependencies": ["wf-core", { "name": "wf-telemetry", "version": "~2.1.0" }] }

Install a package whose dependency is missing and the install fails and names what to install rather than half-working. Conflicting ranges from different packages are intersected, and an unsatisfiable combination fails loudly instead of silently picking. Depending across marketplaces is blocked unless the marketplace explicitly allows it. And a package may consist of nothing but a dependency list, which is the documented way to ship a bundle. See plugin dependencies.

That is what makes the split viable. Go one package per workflow and the components several workflows share have to live somewhere, and every obvious home is bad: copy them into each package and they drift, put them in one workflow's package and the others depend on an unrelated workflow, keep everything together and you lose per-team enable/disable. A dependency array removes the dilemma. The shared layer becomes its own package, versioned on its own cadence, and each workflow package declares a dependency on it.

That also settles what granularity to choose. Without dependencies, the shared code dictates it: you keep things together because splitting means duplicating. Once the shared layer can stand alone, that constraint is gone and you are free to decide by who installs what.

Package granularity should follow the consumer boundary, not the code structure.

The question to answer is not "how is this organized?" but "who installs it, and do different installers want different subsets?" That gives a decision rule rather than a preference:

One consumer group, everything used together → one package, one version. Granularity you don't need is pure coordination cost: more manifests, more version numbers, more ways for the set to be half-installed. Do not split a collection because it looks like several things.
Different consumers want different subsets, or parts need different release cadences → split, with the shared layer as its own package that the workflow packages depend on. This is the point at which per-package enable/disable and independent versioning start paying for their overhead.
Either way, the shared layer exists exactly once. Whether it is a directory inside one package or a package of its own, two copies of a shared component is the failure mode below, not a strategy.

Two things make the split cheap if it comes later, and both are worth doing up front even when you don't need them yet:

Reserve the original name for the bundle. If the package everyone installs is X, then when you split, X becomes a package that is nothing but a dependency list. Consumers keep typing the same install command and never learn that the internals changed.
Keep the internal boundaries visible before they are enforced. Directories that already separate "shared" from "this workflow only" split cleanly. A flat pile does not.

When the conditions above do call for a split, this is the shape it takes:

Core: the deterministic primitives (run state, seam guards, source-control isolation), the shared judgment skills (plan, implement, validate, take-approval, write-failing-tests, publish), and the measurement hooks. One copy, one version, one owner.
Per workflow: the orchestrator, the skills only it uses, its agents. Depends on core with a range.
Bundle: dependencies only, under the name consumers already type.

Three traps

1. Duplication drifts silently, and faster than you expect. One observed case: a workflow packaged with twelve skills, eight of them shared with other workflows that were still distributed loose. Three weeks later, five of the six shared skills had diverged: the packaged copies three to seven lines behind, with the same name and a byte-identical description but a different body. Nothing surfaces that. The lesson is not "don't package": it is one source of truth, and packaging is downstream of it. Never edit the packaged copy, and never leave a loose copy of something you have packaged.

2. Skills get namespaced. Agents don't. A packaged skill is addressed as package:skill, which makes collisions impossible, but it is also a rename, so every cross-reference in your instructions has to survive it. Worse, if a loose copy of the same skill still exists, it keeps the short unprefixed name and answers to it, which is exactly how two divergent copies coexist unnoticed.

Agents have no namespace at all, and resolve by a priority ladder in which packaged agents rank lowest, below project-level and user-level ones (sub-agents). So if a workflow installs its agents into the user directory and ships them in a package, the user copies quietly win and the packaged ones are dead weight. Pick one home for agents and stick to it.

3. Hook composition across packages is undocumented. Whether hooks from several packages all fire on the same event, and in what order, is not specified anywhere I could find. Do not build a design that depends on it. Put hooks in exactly one package, which is a good constraint regardless, because measurement is cross-cutting and therefore belongs in core.

What packaging buys beyond convenience

Version identity. A run can record which package version served it, which is what makes the "version anything you intend to trend" rule from part four actually implementable.
Team enforcement. A marketplace declared in project settings plus an enabled-packages list gives everyone the same set, updating on its own, no "did you copy the new one?" ritual (marketplaces, installing).
Opt-in by default. A package can install disabled, and prompt for the configuration it needs at enable time.
A package boundary is also a test boundary. The "without the component" comparison from part four is naturally per package: does this one earn its place?

Context cost is not the deciding factor

A package contributes its skills' names and descriptions to the prompt. Five packages of ten skills and one package of fifty cost about the same, because all fifty names appear either way. Split for ownership and release cadence, not to save context. What genuinely costs context is MCP servers, whose tool definitions are present on every turn, so bundle those deliberately, and separately.

Migration order

Whichever granularity you land on, the order of operations is the same, because each step de-risks the next:

One source of truth in the repository. Whatever copies exist elsewhere (hand-installed directories, an older package, a colleague's machine) reconcile and delete them first. Packaging a collection that already has two divergent copies just makes the divergence permanent, and this is not a packaging problem.
Core package first: shared spine plus hooks, published and consumed by nothing.
One workflow package, depending on core, taken end to end: install, run, measure.
Then the rest, plus the bundle.
Record the package version in every run, so the next question, "did this get better?", is answerable at all.

Portability across harnesses

Everything so far has one unstated dependency: the harness: the agent runtime that loads the skills, dispatches the sub-agents, fires the hooks and enforces the permissions. That is a real lock-in risk, and it is worth knowing which parts of the design carry over and which parts are the port. (Portability across repositories, the same workflow against services in different languages and on different platforms, is the other axis, and it is the "name intents, not commands" rule in part two.)

What is already standardized

More than people assume, as of 2026:

The skill format. Agent Skills is an open specification: a folder with a SKILL.md (YAML frontmatter plus a Markdown body) that can bundle scripts and references. It was published in December 2025 and is stewarded through the Agentic AI Foundation; by 2026 30–40 tools read the same files from the same directories, across competing vendors. Notably, .agents/skills/ is the one location every major tool reads natively.
The instructions file. AGENTS.md is the cross-tool convention for "how to build and test this project", now under the same foundation and read by dozens of agents (field guide). Some harnesses still prefer their own filename and import it.
The tool layer. Anything you build as an MCP server works across compliant runtimes without a rewrite. That is the portable way to expose a capability.
The packaging format. An Agent Plugins standard was announced in August 2026 by several vendors together: a root manifest, fixed locations for skills and MCP configuration, and a namespaced escape hatch for client-specific behavior. Worth tracking against whatever native format you package in today.

What is not standardized, and won't be soon

Capability	Portability
Skills (content, frontmatter, bundled files)	Portable: same format, same directory
Project instructions	Near-portable: one convention, minor filename differences
Tools, via MCP	Portable
Sub-agent dispatch	Divergent: isolated agent files, "personas", TOML definitions, persistent modes: different models of what a sub-agent even is
Hooks / lifecycle events	Divergent: event sets differ sharply, and one harness is far richer than the rest
Slash commands	Fragmented
Model and effort selection per step	Harness-specific
Permissions, sandboxing, background tasks	Harness-specific

The pattern is consistent: content ports, control flow doesn't. The more a capability is about orchestration, the less portable it is.

Figure 2: The file based half ports for free. The orchestration half is the port.

The one rule that buys portability

From obra/superpowers' porting guide, which is the most concrete write-up of this problem I have found, and whose central invariant is worth adopting verbatim:

Skills name actions, not tools.

Write "read the file", "run the command", "invoke the planning skill", "dispatch a sub-agent", never the harness's actual tool name. Porting then means adding a tool-mapping document for the new harness that translates each action into its real tool names, and it never reaches into a skill body to swap names. Their companion rule: everything ships through the harness's own install mechanism, and you never edit the user's files.

This is testable, and worth measuring on your own collection. In one measured collection, skill bodies almost never named a tool (three of fifty-four mentioned one) while the orchestration layer was the opposite: a dozen guides named the skill-invocation tool directly, and two dozen agent definitions carried harness-specific model, effort and tool-allowlist fields. Which is exactly the split you would predict, and it tells you where a port's cost lands.

What a port actually costs

Per harness, roughly four things:

Bootstrap: how the entry-point instructions reach the model. The superpowers guide treats automatic session-start injection as a hard requirement: if the only way to get your framework in front of the model is a per-session opt-in, that harness cannot be properly supported.
Discovery: how skills are found: a native skill mechanism, a plugin registration API, a manifest field, a generated index, or documenting "read the SKILL.md directly" as the sanctioned path.
Tool mapping: one document per harness. Get the tool names from the harness itself, not from its docs.
Distribution: the harness's own install channel.

Degrade explicitly; never fake

When a harness lacks a capability, the skill must say do it inline, or report the missing capability, never invent a tool call that doesn't exist. Their word for this is degradable, and the useful discipline is deciding the degradation in advance:

No sub-agents → run the step inline, and state plainly that the fresh-context property is lost. That is a real reduction in quality, not a cosmetic one.
No hooks → measurement moves to a scheduled reader over the artifacts, which part four argues you need anyway for anything happening after the session.
No task-tracking tool → a plan file.
No per-step model selection → the tiering in part three collapses; say which decisions are now running on the wrong tier rather than pretending the design is intact.

Why this design ports better than most

Not by luck: the earlier principles do the work:

Artifacts are the interface, and the run directory is the only channel. Files are the most portable thing in computing.
Gates are content hashes in a file. No harness feature required, so the whole approval mechanism survives a port untouched.
The primitives are scripts, and the loop referee is a script. Exit codes are universal.
Terminal states and reports are files.

What genuinely doesn't survive: per-step model and effort tiering, hook-based measurement, and the sub-agent isolation that several principles depend on. So the honest summary is that the file-based half ports for free, and the orchestration half is the port.

A portability checklist

Keep skills in the standard format and the standard location.
Name actions, not tools, and grep your own skills to check.
Keep exactly one tool-mapping document per harness, outside the skills.
Keep state in files, not in harness memory.
State each skill's capability requirements explicitly, with a declared degradation for each.
Put everything harness-specific behind a single boundary (the packaging manifest, the hooks file, the agent definitions) so a port touches a known, small set of files rather than fifty skill bodies.

From local to autonomous

Local-first is the starting position, not the ceiling. For some workflows the destination is unattended operation, and the shape of that destination is worth stating concretely.

The target flow

A ticket is assigned to the agent and moved into a queued state. That transition, not a human typing a command, is the trigger. The agent picks the ticket up, runs understand → plan → failing tests → build loop, and opens a pull request. The review and the response to review are part of the chain, not separate human errands: the review workflow reviews the change, the fix workflow triages those findings and addresses the ones that deserve it, and only then does a human look. What they find is a pull request that has already survived a review round and been revised once, with a written record of what was changed, what was argued against, and why.

The gates do not disappear; they move

This is the key transition, and it is a design job rather than a switch:

Today: a human approves every plan.
Tomorrow: a policy approves the ordinary plan and a human approves the exceptional one, and it approves it through the same mechanism, by recording the approval the way a human would (part four), so the gate check itself never learns there was nobody there. "Ordinary" is machine-checkable: the plan touches no protected file, stays inside a size budget, introduces no new dependency, carries no unresolved open question. Anything else escalates to a person.
The workflow's real job becomes making "exceptional" detectable. That is where the design effort goes, and it is why the plan artifact needs to be structured rather than prose.

Some gates stay human permanently

The list is the same one from part two: rewriting shared history, resolving someone else's review thread, merging, and anything that speaks on the team's behalf. Autonomy means removing the approval step from reversible work, never from irreversible-in-public work.

Figure 3: The gates move; they do not disappear. A policy approves the ordinary plan through the same recorded mechanism a human would use, so the gate check never learns there was nobody there, and the publish gate never changes hands at all.

What must be true first

Every one of these is a prerequisite, and every one of them was learned from a run that had a human watching:

Every rule enforced by a script. An unattended run has nobody to notice a skipped gate. The unenforced-gate story in part two is a nuisance with a human present and a silent data-loss event without one.
No question that stops the run. "Shall I carry on?" is fatal without an audience. An autonomous workflow either proceeds, or reaches a terminal state and reports; there is no third option.
Terminal states and a report on every exit, including abandonment. The report is the only thing a human will read, and for most runs it will be the only thing they read.
Environment-vs-test failure classification. Otherwise an infrastructure blip consumes the whole budget and produces a confident, wrong verdict.
Isolation strong enough for concurrency. Separate working copies, no shared mutable services, no two runs racing on one database or one container name.
Restraint that holds without supervision. The review-and-fix leg must be able to decline a wrong suggestion when nobody is there to back it up. This is the hardest prerequisite and the one most worth evaluating explicitly.
Cost and iteration ceilings, because nobody is watching the meter.
Structured telemetry per step, so a fleet of runs can be audited after the fact instead of inspected one at a time.
An eval suite you actually trust (part four), including a clean control. Unattended operation without one is not automation: it is unmonitored production.
Something other than a person to start it. A ticket transition can only be a trigger if some process is listening for it and can begin a run, and that process has to outlive any terminal session. This is the one prerequisite on the list that better skills cannot supply.

The honest reading: most of the work between "useful locally" and "trustworthy unattended" is not model capability. It is enforcement, isolation, classification and reporting: the boring parts, again. The exception is the last item, and it is a real one. Everything above it can be built inside the skills; a runner cannot, because a procedure that lives inside an interactive session has nothing to start it. Part six returns to that.

That is the workflow packaged, ported and, eventually, left to run on its own. What is left is the part a description like this owes the reader, and it is part six: where the design still hurts, and the short list of what to build first.

Originally published at fullgc.github.io, where the whole series lives.

Designing agentic development workflows: is it working, and is it worth it

Dani Shemesh — Mon, 14 Sep 2026 14:38:37 +0000

Everything so far has been design, and design argues for itself. This part is about evidence, which does not.

Three independent ways to know whether a workflow works are worth building, and they answer different questions. Most teams build one of them and then over-claim from it. An external corpus asks whether a step's output is correct. The workflow reporting on itself asks where the process hurts, costs nothing, and is available on day one. Delivery metrics ask whether anything shipped sooner and at what price, which is the question a sponsor asks first and the hardest of the three to answer honestly. Each has a characteristic way of being gamed: a looser grader, a quieter agent, a confounded comparison group.

Assumes the artifacts and terminal states from part two, because every measurement here reads them.

Testing and evaluation

"How do you test this?" is really four questions. A workflow contains four kinds of thing, and each needs a different kind of test.

The thing	Example	How you test it
A script	run state, a seam guard, branch setup	like any other software: run it, check the exit code
An adapter	code that reads or writes an external platform	check it still gives its consumer what that consumer needs
A judgment step	deciding, reviewing, classifying, planning	you can't assert an opinion. Score it on tasks whose answers you know
The workflow itself	the whole thing, end to end	run it on a small task; check the result and the leftovers

The two dead ends are trying to unit-test an opinion, and trying to eyeball a script. Sort the component into one of those rows first and the method follows.

Where to start

In this order. Each step is cheaper than the next.

Validate the structure: does every component parse, and do its file references resolve? A second per commit.
Test the scripts: one test per documented exit code, plus "run it twice and nothing changes the second time." Most of these get re-entered after a failure, so repeating safely matters.
Test the seams: for each pair of components that pass data, check the producer still emits what the consumer needs.
Smoke-test one whole workflow on the smallest real task you have.
Then build a scored task set for the judgment steps. Most valuable, most expensive: do it once you know what failure looks like.

The seams are where things quietly break

Borrow contract testing from microservices: the consumer writes down what it needs from the producer, and that expectation runs as a test on both sides whenever either changes. Only what the consumer actually uses gets pinned, so the producer stays free to change everything else. See Pact or this overview.

Not theoretical: in one observed case a component fetched an identifier and never wrote it out. The component that needed it failed for want of it, after the change had already been published. Two components, one unwritten assumption, no test at the boundary.

This is also the answer to "a component is used by five workflows, what did I just break?" Keep a generated map of which workflows use which component, and let a change select the suites to run.

Scoring a judgment step

You can't write an assertion against an opinion. You can assemble tasks whose right answers you already know, run the step against them, and score it. The task set is the work; the scoring is easy.

Most of this is from Anthropic's Demystifying evals for AI agents, which is worth reading in full:

Start with 20–50 tasks taken from real failures: your bug tracker, not your imagination. Early on, changes have big effects, so a small set discriminates fine.
A task is only usable if two people who know the domain would give the same verdict. Otherwise you are measuring your own ambiguity.
Write down a known-good answer for each task. It proves the task is solvable and that your scoring works: the two problems that otherwise look like a weak agent.
Include tasks where the right answer is "do nothing." Most people skip this. If every task has a problem to find, the best-scoring agent is the one that always finds something. Include correct code that looks suspicious, and one task with nothing wrong at all.
If everything fails, suspect the task. A 0% score across many attempts usually means the task is broken or ambiguous, not that the agent is incapable.
Grade the result, not the steps taken. Checking that an agent followed a particular sequence of tool calls is, in Anthropic's words, "too rigid and results in overly brittle tests", capable models keep finding valid routes nobody planned for. If you do care about the path, check only that it didn't do anything extra.
Read a few transcripts before believing any score. A number can rise because the scoring got looser.

Two task sets are worth having: a planted one, where you introduce problems deliberately and record the answers, as your gate; and a replay one, which costs nothing to write: take real finished work that already has real human feedback, run the workflow against how things looked then, and compare.

If a model does the grading, spot-check its verdicts against human ones now and then, and watch for it drifting lenient, a wrong answer marked correct inflates everything downstream.

Living with randomness

The same input doesn't give the same output twice, so one run tells you little. Run each task a few times and decide which you need: at least one attempt succeeds (fine when you only need one good answer), or every attempt succeeds (when consistency is the point). Those two diverge fast as attempts increase, so say which one you are quoting.

Then two practical rules:

Start every attempt from a clean slate. Leftover files and warm caches cause failures that look real but aren't.
Compare the outputs, not just the pass rate. A step that finds three good things, then three different good things, can score identically each time and still be impossible to depend on.

Cheap checks that punch above their weight

Tripwires. Emit one number per phase: how much of what it produced is actually usable. The example that earned this a permanent place: a review step produced twelve findings, and none of them pointed at a real file and line. Nothing would ever be posted. Every exit code was zero, so no task set would have caught it. One counter made it obvious.

Relations you can assert with no answer key. When you can't say what the right output is, you can often say how two outputs must relate:

shuffle the input order → the decisions should be the same
feed the same item twice → it should be deduplicated
feed something with nothing wrong → the output should be empty

These need no task set, take minutes to write, and catch instability single examples never will. (The formal name is metamorphic testing.)

Testing a workflow that needs human approval. This looks impossible, since something has to approve the plan. The wrong fix is a "skip the gate" flag, which removes the very thing you wanted to cover. Instead, have the test write exactly the approval record a human would have written: same file, same format, same hash. The gate check downstream passes unchanged, because it cannot tell the difference and doesn't need to. Only the source of the approval changes. The same trick is how a gate later becomes an automatic policy (part five).

Where the test cases come from

Before building any of it: read what actually happened. Sample real runs, note what went wrong in plain words, group those notes into named failure categories, and count how often each occurs. Thirty to fifty by hand is the usual starting point.

The one-line version, from Hamel Husain and Shreya Shankar's work on evals, is hard to improve on: look at your actual data, by hand, before building any automated metric. A task set built without this tests the failures you imagined; one built after it tests the failures you have.

What your runtime may already do for you

Check before building anything. A capable agent runtime may offer:

A "without the component" comparison: run the same task set twice, once with the component installed and once with it removed, and compare the scores. This is an ablation, and it is the single most valuable measurement available, because it answers "does this thing earn its place?" rather than "did the model succeed?" A workflow can score 80% on your corpus while your planning skill contributes nothing: the model was getting there anyway. You only find that out by taking the skill away and running the same twenty tasks again.
Separate grading for "did it trigger" and "did it do the right thing": two different bugs, and the first is easy to miss entirely.
Deterministic graders: pattern matches, "was this tool called", "was this file created", alongside model-based scoring.
Repeats, pass thresholds, cost ceilings and machine-readable output, which is all CI needs.

Claude Code has a claude plugin eval command that does exactly this: its --ablation with-without mode runs a no-plugin baseline arm and reports the delta. As of 2.1.259 it is early access and undocumented, so not everyone has it and none of the detail is guaranteed to hold. Worth watching rather than building on, because a runtime that runs the comparison for you turns the most valuable measurement here into a flag.

What they generally don't give you: checks partway through a long workflow, partial credit, or any way to assert which sub-agent ran on which model. For those, split the workflow into one case per phase, or check the files the run left behind, a good reason for phases to write files.

Five checks that need no test set at all

Assert these after any run, with no answer key:

did it finish in one of its declared end states, rather than just stopping?
does every file it promised exist?
was every gate satisfied by a real approval?
did the loop stay inside its limits?
were there any stops that weren't designed: a question that halted it, a missing input?

If you build nothing else, build these five and the component map. They are cheap, need no labeled data, and between them catch most of the ways a workflow quietly rots.

Making the workflow evaluate itself

Every technique so far grades a workflow from the outside: build a corpus, run it, score the output. That is indispensable and expensive: the corpus is the work, and you cannot build one before you know what failure looks like. Which is the well-known bootstrapping problem: the teams who most need evals have no labeled data yet.

There is a second axis, and it inverts the arrangement. Have the workflow report on itself. Each dispatched step records where it hurt; the orchestrator sums up its own run; those records aggregate across many runs, per workflow and per component. No labeled data, no answer key, and it runs on the real distribution of work rather than the one you imagined.

The two are complements, not alternatives:

	External evals	Self-reported friction
Question answered	is the output correct?	where does the process hurt?
Needs a labeled corpus	yes, the expensive part	no
Distribution tested	the one you authored	the one you actually get
Scales with	authoring effort	usage
Available	after you know the failure modes	immediately
Can it be gamed	by a looser grader	by a quieter agent

Crucially, the second bootstraps the first. The taxonomy that self-reporting produces is precisely the open-coding-then-axial-coding step a corpus needs, except the agents do the open coding as they go, instead of a human reading traces weeks later.

Exercise the design deliberately

Passive collection tells you about the work you happened to do. To learn about the design, choose the work so it spans the design's axes. A useful spread for a family of workflows:

one item requiring a schema or data migration (exercises irreversibility and generated artifacts);
one that is pure logic, deliberately trivial;
one that cannot be completed without touching a protected or "ask first" file (does the workflow notice the constraint, or edit through it?);
one that changes a published interface (exercises back-compat judgment, which is what reviewers actually argue about).

The trivial one is not filler; it is the control. If a task with no migration, no new model and no generated files still costs the full apparatus (every phase, every gate) that is fixed overhead, and fixed overhead is invisible unless something cheap is measured beside something expensive.

What a self-report has to contain

Free-text retrospectives do not aggregate. A structured record does, and the field list is where the value is:

The component it is about: not the phase. This is the primary axis; without it every finding collapses to "the workflow is annoying". Validate the name against a known list, or you will get typos and the non-answer "the workflow".
Impact (what it did to this run: blocked / caused rework / merely slow) kept separate from severity (what it would do to a future one). They are orthogonal: a trivial papercut can block, and a serious latent gap can cost nothing today.
Evidence, required and non-empty. It is the only thing separating a finding from an opinion: a command and its output, or a file and a line.
A signature, so the same problem reported by four runs collapses to one row.
Who reported it, which is both a compliance denominator and the difference between a step's own view and the orchestrator's outside view of it.
Optionally a suggested fix, optional on purpose, because requiring one invites invented ones.

The nil protocol

A step that hit no friction still files a report: the same self-report described above, carrying an explicit "nothing hurt" value rather than simply not being written.

Without it, silence and not-bothering are indistinguishable, and those two mean opposite things. A required nil also gives the count a denominator: three frictions across forty steps, rather than three across an unknown number. The failure it prevents is specific. Fix a component, watch the reports drop, and you cannot tell whether the component improved or the agents went quiet. That is the same trap named earlier for corpora without a clean control, and it is the cheapest safeguard in the design.

The rollup, and what its columns must not hide

Aggregate by (component, signature), then print, per group:

count and distinct runs, separately. Four records from four runs is four independent confirmations; four from one run is one run tripping repeatedly. Same count, opposite meanings.
breadth: which workflows reported it, by name. This column is the answer to "is this a shared design fault or one workflow's wording problem", and it is how a fault in a component used by five workflows earns five times the attention.
both raw and per-workflow-normalized severity. Shared components will accumulate more findings simply by being used more; printing only the raw total confirms that hypothesis by construction, and printing only the normalized one hides the real total cost. Print both and say which you are quoting.
a reporting-hygiene section: runs that filed neither findings nor nils, findings with suspiciously thin evidence, unknown component names. This is the part that keeps the rest honest, and it is the only place hard targets belong: reporting rate and evidence present, never friction counts.

Then fix, and run the same work again

The improvement loop closes by re-running the same items on the improved workflows. That comparison is worth something only if the second run is genuinely a second run:

fresh sessions, with no transcript of the first in context;
reset state: the tracker items returned to their starting condition, prior artifacts archived rather than left where an autodetect glob can find them, caches and databases cleared;
the software tagged at both points, and anything installed-by-copy re-installed, or the second wave silently runs the first wave's components;
the task specifications untouched. If the first run revealed that an item was ambiguous, resist clarifying it. That is the most tempting and most invalidating edit available;
the previous attempt's traces removed. Agents have been observed gaining an unfair advantage by reading the version-control history of earlier trials, so a same-task re-run must not leave the first attempt's commits lying around.

What the comparison may honestly claim

Two measures survive the confounds:

Recurrence of named frictions: per item fixed, a binary: did it come back? Most direct, least confounded, and it needs no baseline arithmetic.
Unplanned stops per run: anything that is not a designed gate: a question that halted the run, a stuck terminal state, a missing artifact, a guard firing.

Then, as supporting evidence: operator interventions, terminal states, loop iterations, artifact completeness, gate integrity, and counts of expensive operations (environment boots, dependency re-locks, full-suite runs, human waits), which are cache-proof, unlike elapsed time.

Not evidence: wall-clock, token cost, raw finding counts, lines changed. And report per item; never average across items, because difficulty varies and the average hides the interesting cases.

State the confounds rather than burying them: the sample is small, it is one codebase, and the operator has seen the work before, which no process controls. The claim available is "these specific frictions were removed", not "the workflow is better". Where a claim does reach beyond the workflow's own internals (that delivery got faster, say) it needs the comparison discipline below, especially the negative control.

Failure modes of the method itself

Self-report bias. A step is a poor witness to its own confusion. The orchestrator's outside view of a step, and the step's own view, are different data, collect both and keep them distinguishable.
Only dispatched steps can report as separate voices. A step invoked inline by the orchestrator has no independent context, so its "self-report" is really the orchestrator's. If per-step attribution matters, that is another reason to dispatch the step.
Telemetry must never be able to fail a run. Give the recorder exactly two outcomes (recorded, or bad usage) and keep it off every failure path. A measurement layer that can break a run gets switched off within a week, and then you have neither.
Never set a target on friction counts. "Fewer than ten findings next time" rewards silence.
Separate harness friction from workflow friction. Findings caused by how you ran the experiment (contention between parallel runs, a shared container, an expired credential) are about your harness, not the design. File them under a distinct label or they will misdirect every fix that follows.

Why this is the highest-yield thing to build first

It needs no corpus, so it is available immediately; it produces the failure taxonomy that a corpus later requires; and its output is self-sharpening: a named friction that recurs is a regression test that wrote itself. Fix it, record the signature, and every later run checks for its return at no cost.

Measuring whether it delivers

The first two axes are inward-facing. Corpus evals ask whether a step's output is correct; self-reported friction asks where the process hurts. Neither can tell you whether anything was delivered sooner, or what it cost, and that is the question a sponsor asks first.

Axis	Question	Data needed	Gamed by
External evals	is the output correct?	a labeled corpus	a looser grader
Self-reported friction	where does the process hurt?	nothing; it self-instruments	a quieter agent
Delivery metrics	did it deliver, and at what cost?	the tracker, the forge, runtime telemetry	survivorship and confounded groups

Most teams build exactly one of the three and then over-claim from it. They are not substitutes.

Four streams, and a division of labor that is easy to get wrong

Delivery measurement draws on four independent sources, and each owns something the others cannot supply:

Stream	Owns	Grain
Runtime telemetry	spend and token volume, sessions, active time	session × component × model
Workflow telemetry	runs, outcomes, loop iterations, referee decisions, early exits, component calls	run × work item
Issue tracker	the work item's lifecycle: created, transitions, comments, links	work item
Code host	branches, commits, pull requests, reviews, CI, deployments	pull request / branch

Two rules follow, and both are counter-intuitive enough to state plainly:

Cost only ever comes from the runtime. Nothing in a workflow's own telemetry prices tokens. What the workflow stream contributes is the dimension the cost stream lacks, which workflow, which work item, which outcome. Runtime cost alone has no work-item dimension at all, and that gap is the entire reason both streams exist.
Delivery outcomes only ever come from the tracker and the code host. Workflow telemetry says what a run did; it can never say whether delivery actually changed.

The join, and what breaks it

The spine is the work-item key. The tracker has it natively; the workflow stream carries it; the code host yields it from a branch-name convention (with the PR body and commit messages as fallbacks). Cost joins only indirectly (session → run → work item) which is the join no single system can perform alone, and the reason instrumenting the workflow is a prerequisite for costing anything.

Three consequences worth designing for up front:

A workflow with no work-item key breaks the spine. Anything triggered by a repository or a pull request rather than a ticket must emit an alternative key, or its cost attaches to nothing.
Some joins no system provides. The wait at a human gate (the approval record carries a hash but no timestamp, as noted above); the exact link from a run to the pull request it produced (inferred from a branch name unless the run stamps its id into the PR body or a commit trailer); and the identity map between tracker account, code-host login and commit email, which is a hard prerequisite for anything per-person.
Cardinality is many-to-many. One item may have several runs and several pull requests. Fix the rules once (earliest start, latest merge, summed iterations) and exclude bot authors everywhere.

Figure 1: Cost only ever comes from the runtime, and the runtime has no work item dimension at all. The middle link of that chain exists only if the workflow records it, which is why instrumenting the workflow is a prerequisite for costing anything.

One row per unit of work

Reduce all of it to a single record per work item: created, started, first commit, PR opened, first review, merged, deployed, done; time in each status; reopens; review rounds; size and files touched; CI attempts; runs, outcome, iterations, gate rejections; and spend.

Every metric that follows is then a subtraction or a count over that row, which is what keeps a metric catalog from turning into a pile of bespoke queries.

Four lenses, and one rule about reading them

Usage: is it actually being adopted: share of eligible items with a run; repeat use, which separates "tried it" from "adopted it" in a way a user count hides; funnel drop-off showing where runs die; the outcome mix.
Efficiency: cycle time (started → done, excluding backlog), lead time, build time, time in each status, time to merge, review wait, rework rounds, discussion depth, size.
Quality: first-pass CI success, reopen rate, escape rate (a defect linked back within some window), revert rate, follow-up churn, change failure rate.
Cost: spend per delivered unit, spend on runs that produced nothing, waste ratio, spend per component, spend per loop iteration.

Efficiency is never reported without quality in the same view. A speed number shown alone invites precisely the trade nobody wants, and the invitation is hard to refuse once the number is on a slide.

The elapsed-time paradox

The previous section says elapsed time is not evidence; this section makes cycle time a headline. Both are right, and the distinction matters:

A run's wall-clock is confounded by caches, warm environments and machine load. Use operation counts instead.
A work item's cycle time is measured between events days apart in external systems. No cache touches it.

Same word, different clocks. Never mix them in one chart.

Comparing assisted with unassisted

The moment there is a number, someone will compare it to the unassisted case. The comparison is worth doing and easy to fake, so fix the discipline before the first chart:

Label every work item as unassisted, assisted-any (a run was started, whatever became of it), or assisted-completed.
Compare assisted-any against unassisted. It is tempting to compare only the runs that finished, but runs are abandoned precisely on the items that turned out to be hard, so excluding them removes the workflow's worst cases and manufactures a win. Report the completed subset separately, labeled "when it completes", never as the headline. This is the single most common way these numbers lie.
Compare like with like. Same item type, same window, bots excluded.
Band by size and by repository, and compare within band. A large change in a repository with a slow suite is not comparable to a small one elsewhere. Without banding, a difference in which items people chose looks exactly like a difference the workflow made.
Summarize honestly. Median and p90, never the mean; show the difference with an interval, and if it spans zero, say "noise"; print both group sizes and show nothing for thin cells.
Pair people against themselves. For anyone with items in both groups, compute their own median difference and check whether most people moved the same way. Differences between people (seniority, familiarity, how they scope work) are usually larger than the effect being measured.
Run the same comparison on a metric the workflow cannot possibly affect. The lag between merging and someone finally dragging the item to done is pure bookkeeping; no coding workflow changes it. If the assisted group looks "better" there too, the groups differ for some other reason (most likely easier items being chosen) and the headline cannot be trusted. A negative control is the cheapest protection against fooling yourself, and almost nobody builds one.
Show quality beside speed, always.

Even with all of it, this stays observational: people chose when to use the workflow. You can describe a difference; you cannot claim a cause. A causal claim needs randomization (alternating by week, for instance) which is worth doing once a baseline exists.

Anti-metrics

Tempting	Why not
Run counts, invocation counts	Activity, not delivery. Trivially inflated
Token volume	An input. Use spend per delivered unit
Completed-runs-only timing as the headline	Survivorship; see step 2
Total early exits as a failure count	A gate refusing to proceed is the gate working. Failure is a terminal state, not an exit
Lines of code	Rewards volume, which this tooling produces cheaply
Any per-person KPI	These are program metrics. Per-person views exist for the paired analysis, not for evaluating people

And set no targets before a baseline exists. Targets set in advance are either trivially met or quietly abandoned. Measure for a full delivery cycle, publish the baseline with its spread, then set targets.

Instrumentation: the artifacts are the source; hooks are only the trigger

This closes a loop the design opened much earlier. Part two made artifacts the interface between phases for composability. It turns out they are also the only trustworthy measurement surface: in-session hooks observe tool calls, not workflow semantics, so anything a workflow concluded (an outcome, a verdict, a count) must be read from the file it already writes for its own correctness. No metric should ever depend on parsing prose.

The practical shape: choose the one moment when a run is definitively over, read the artifacts there, and emit everything at once, guarded so a single run cannot emit its terminal batch twice.

Two classes of signal no in-session instrumentation can reach: delayed human reaction (a thread resolved, an item moved days later) and downstream effects (whether the change was reverted, whether a defect was linked to it). Those need a scheduled reader over the artifacts and the external streams. If a workflow's most important number is of that kind, the scheduled reader is part of the measurement design, not a later nicety.

Rules for whoever adds a metric

Increment by exactly one per observed event. Never accumulate a computed quantity into a counter.
Emit numerator and denominator separately, never a ratio. A pre-divided rate cannot be re-aggregated across runs.
Mirror the failure path wherever you handle the success path. A crashed step otherwise leaves a span that never closes and a window that never shuts.
Watch cardinality. Anything unbounded (a file path, a finding title, a branch name) does not belong in a dimension.
Version anything you intend to trend. A prompt or rule change alters what a workflow produces; without a version dimension, a trend line silently splices two different systems together. It cannot be backfilled, so it has to be there from the first emission.

Shared metrics, owned metrics, and the tier that gets over-claimed

Most signals should be shared: one definition emitted by every workflow, sliced by workflow name. Resist the urge to fork them per workflow: a new terminal state belongs in the shared vocabulary, not in a private metric.

But workflows should also carry their own metrics, because the interesting quality signal is usually specific to what that workflow produces: a reviewer has precision and recall over findings; a build loop has iterations to green; a triage step has restraint. Name those distinctly so they cannot be mistaken for a shared series, and keep them scoped to the workflow that emits them.

That gives three tiers, and the distinction governs how each may be read:

Tier	Meaning
Program	One definition across every workflow. Comparable
Owned	Meaningful only for this workflow, measured on its own runs and artifacts
Attributed	A program metric measured on delivery artifacts, restricted to work that used this workflow

Attributed metrics are the ones that get over-claimed. They are measured on work items and pull requests, not on runs, so they would read identically if something else entirely caused the improvement. Report them as "the items that used this workflow", never as evidence about the workflow alone.

One constraint that should govern every dashboard: cost is comparable across workflows; quality is not. Spend per run ranks any two workflows sensibly. Quality metrics are defined against what a workflow produces and have no counterpart in a sibling, so cross-workflow panels should carry only the shared run-and-cost signals, and every quality panel should be scoped to one workflow.

Trace a handful by hand before publishing anything

Before any aggregate goes on a dashboard, follow three to five known items end to end manually (one assisted, one unassisted, one that went wrong) and check that the record matches what actually happened. A single wrong status mapping produces confident, plausible, entirely wrong numbers, and nobody downstream will question a chart.

Continuous cross-checks earn their keep for the same reason: items marked done with no linked pull request, pull requests merged whose item never completed, branches matching no item, negative durations. Each one is a broken join, and a broken join fails silently.

A workflow that works on your machine, and that you can prove works, is still a private tool. Part five is the three transitions that make it something else: packaging it so others can install and depend on it, porting it to a different harness, and taking the human out of the gates that can safely lose one. Part six is where all of it still hurts.

Originally published at fullgc.github.io, where the whole series lives.

Designing agentic development workflows: the cost of control

Dani Shemesh — Mon, 14 Sep 2026 14:38:13 +0000

Two parts in, the machinery is settled: phases in order, artifacts between them, gates that check rather than promise, a script deciding when the loop stops. What is not settled is the arithmetic around it, and the arithmetic decides whether any of the machinery matters.

How large should the item handed to a run be? How many approvals can one person give before approving stops meaning anything? Which steps deserve the strongest model, and which decisions can never be recovered downstream if that choice is wrong? This part is short because the answers are short. They are also the answers most often got wrong, because each one is a place where doing the efficient thing costs you the control the workflow was built for.

Assumes only that gates and loops exist and are enforced, which is part two.

Sizing the work

The most consequential decision about a run happens before it starts: how big is the thing you handed it?

The recommendation is deliberately smaller than feels efficient: more items, each narrower. Not because the workflow cannot handle a large one, but because everything that makes a run trustworthy degrades with size:

Control. A small change is one a person can actually review. You can hold the whole diff in your head, which is the only state in which "approved" means anything.
Comprehension. With a narrow item you can tell what was written and why. With a sprawling one you approve a summary of what was written, which is a different and much weaker act.
Gates become real. A gate on a focused plan is a decision. A gate on a plan spanning nine files and three concerns is a rubber stamp, and the attention arithmetic in the next section only gets worse as items grow.
Plans get concrete. A narrow ask produces a plan naming files and tests. A broad ask produces a plan of intentions, which is unreviewable and unfalsifiable.
Failure gets cheap. A run that ends stuck costs you one small item, not a week.

The cost is real and worth stating: more items means more runs, more gate touches, and more fixed overhead per unit of delivered work (part four argues for measuring exactly that). Accept it deliberately. The alternative is not less work: it is the same work with less control over it.

A usable rule of thumb: if you cannot state the acceptance criteria in a few lines, the item is too big. Split it before running anything.

Plan quality is bounded by input quality

The single largest lever on what a workflow produces is not the workflow. It is the description of the work handed to it.

A vague item produces a confident, vague plan, and then the gate has nothing to push back on, because there is no specific claim to disagree with. Three things make the difference:

Acceptance criteria specific enough that the plan cannot invent scope. If two people would build different things from the description, the plan is a coin flip.
The context the agent cannot discover for itself: why this is wanted, which constraints apply, which decisions are already settled and not up for reconsideration.
Pointers into the codebase: the module this concerns, the convention it must follow, the thing nearby that already does something similar.

A good diagnostic: if you find yourself rewriting the plan at the gate, the item was underspecified. The fix belongs in the description, not in the plan, otherwise the next run makes the same mistake.

Human gates and the economics of attention

Gates are the product, not the overhead. Everything downstream faithfully implements whatever a gate approves, so an error at a gate cannot be recovered later.

Which is also why gate placement is not a free choice: put them where the process already stopped for a human. A gate somewhere the process never had one gets treated as ceremony and rubber-stamped; a missing gate where the process always had one is the one people notice, loudly, after something ships.

The shape worth aiming for, stated as a promise to whoever runs it: two stops. Everything between them is unattended, however many retries it takes. And it never merges. That is a claim a person can hold in their head, which matters more than the exact number: they need to know when they will be needed and what the workflow will never do behind their back.

Which is precisely why the number of gates matters. Three gates in one run is defensible. Twenty-eight across a batch of parallel runs is not. Attention is the scarce resource, and a human asked to approve twenty-eight artifacts approves them without reading.

Batch by phase, not by run

If a gate check only asks "is this artifact's hash in this run's approval log", then several parallel runs can present their plans together and each still records its own approval separately. A handful of sittings instead of dozens, with the enforcement unchanged.

A gate asks exactly one question: approve, or annotate

Never bundle a decision into it. On one run a gate asked for approval and offered a choice between two implementation routes, one of which added a runtime dependency. The reply was "approved", which answered only the first question, and the right branch was taken by luck. Discrete choices belong in an explicit question before the gate, so a bare "approved" cannot be ambiguous.

When batching, require an answer per item; a blanket "all approved" against four artifacts is ambiguous, and ambiguity at a gate should be treated as feedback, not consent.

Measure the attention, not just the elapsed time

Batching four approvals into one sitting can quarter the attention each receives, which shows up later as extra loop iterations that look like workflow friction. Record how long a human actually spent per artifact, separately from how long the machine ran.

Figure 1: Attention is the scarce resource. A human asked to approve twenty eight artifacts approves them without reading.

Model and effort tiering

Because the tier is chosen per dispatched agent, spend deliberately:

Where	Tier	Why
Deciding whether an external claim is correct	strongest model, high effort	the only step that can produce the restraint outcomes, and nothing downstream can recover them
Root-cause investigation	strong model, high effort	every later phase inherits its errors
Writing the tests that define "correct"	strong model	those tests are the specification
Building, proving, wording replies	mid tier, medium effort	machinery: the decisions were already made
Fetching, parsing, classifying a payload	cheapest tier	near-deterministic, high volume

Cost and trust are both configuration. The same file that picks the tier also declares the tools the step may use, and that second field is a trust boundary, not an optimization. A step that writes the run's report needs to read and write files and nothing else: no shell, no search. Denying it a shell is how you know it cannot edit code, regardless of what it decides it wants to do. Grant the narrowest set that lets the step finish, and prefer removing a tool to adding an instruction telling the step not to use it.

One firm rule: keep a floor on effort. Make a step cheaper with a smaller model, never with less thinking. A small model thinking hard beats a large model thinking barely, and the failure mode of minimal effort is confident wrongness rather than visible struggle.

Note the asymmetry that justifies the whole table: the cheapest tier is also the most likely to conclude "the reviewer is right, do what they said": precisely the failure the expensive tier exists to prevent. Tiering is about where the irrecoverable decisions live, not about average difficulty.

What the design buys

That is the design: the components, the shape of a run, the principles that keep that shape honest, how work is sized, where the human stands, what each step costs to run, and how failures are classified.

The largest claim was made at the very start: what the person invoking it stops having to do. That is the adoption argument, and it outweighs everything below. What follows is what the engineering buys.

Interruptibility. A run can be stopped at any phase and either resumed from its cursor or simply read. Nothing important lives only in a transcript.
Attributable failure. Guards and exit codes mean a failure names the step that caused it, instead of an agent quietly proceeding on missing input and failing three phases downstream.
Auditability. The approval log records what a human approved, by content hash and timestamp. The publish record says what actually reached the outside world. The report explains the run to someone who never watched it.
Composability. Because phases talk through files, workflows chain, skills stand alone, and any phase can be replaced by a human doing it by hand.
Compounding improvement. Shared skills mean a fix in one place improves every workflow that uses it, and makes it worth knowing which shared component hurts most.
Bounded cost. Iteration ceilings, retry caps and escalation limits mean a run cannot grind indefinitely; the failure mode is a clear stop, not a runaway bill.
Reviewable output. One commit per unit of work, messages that reference the originating item, and a diff confined to the files the plan named. The reviewer's first question, "why is this file in here?", is answered before they ask it.
The ability to decline. That is the whole difference between a system that addresses feedback and one that merely obeys it.

Every one of those is a claim, and a claim you have not checked is a hope. The rest of this document is about checking them, shipping them, and being honest about where they fail.

That is the design, and the case for it. Whether any of it is true of your workflow is a different question with a different kind of answer: evidence. Part four is the three independent ways to get it, and the way each one is quietly gamed.

Originally published at fullgc.github.io, where the whole series lives.

Designing agentic development workflows: principles for a workflow you can trust

Dani Shemesh — Mon, 14 Sep 2026 13:25:00 +0000

A workflow's phases and its cast are the easy half. The hard half is keeping that order honest when every step is carried out by something that would rather be agreeable than correct.

What follows are the principles that hold that order in place, in two groups. The machinery comes first, then the doctrine that governs what a step does with its own judgment. Many of them exist because something went wrong in a way that produced no error.

This part assumes part one's vocabulary: orchestrator, skill, agent definition, channel adapter, deterministic primitive, and the run directory. Part one introduces them.

Machinery principles

Where state lives, how one phase hands work to the next, what a gate has to be to count as one, and who is allowed to decide that a loop has finished.

1. The orchestrator is thin, because only a sub-agent can carry a model

In current agent runtimes, the model and reasoning effort serving a step can only be set when that step is dispatched as a sub-agent. A skill's own metadata does not change what serves it.

That single platform fact drives the architecture. The orchestrator runs on whatever model the session happens to use, so it stays cheap and mechanical (control flow, gates, dispatch, wiring), and every act of judgment is pushed into an agent that declares its own tier. When a design wants "this decision deserves the strongest model and maximum effort", the only way to express that is to make the decision a dispatched agent.

Generalized: find the unit your runtime lets you configure, and make that unit the boundary of judgment.

2. The run directory is the only channel

Every run gets a durable directory, and that directory is the only channel across a dispatch boundary. Inline steps share the orchestrator's context and could pass things in memory; a dispatched sub-agent cannot. Design for the boundary and the inline case comes free.

The reason is mechanical: environment variables and the working directory do not survive a call into a sub-agent, so the run directory's path is passed as a literal argument into every dispatch, and anything a later phase needs must be on disk. Run state lives in one small JSON file, written atomically (write a temp file, then rename) so a crash cannot leave a half-written record.

That pays off twice. A run becomes inspectable by anyone, including a human with no access to the transcript. And it becomes resumable: if the state file carries a phase cursor, re-entering the workflow continues from where it stopped instead of restarting over a human's earlier approvals.

3. Artifacts are the interface between phases

Each phase produces a named file and the next phase consumes it: the request, the exploration, the plan, the tests, the implementation notes, the validation record, the report.

One small detail worth stealing: when a phase transforms a file, give the output a different name from the input. A step that writes its result over its own input will, on a second run, read its own output, and any deduplication or comparison it was doing silently stops working.

Because the interface is a file, a phase can be re-run, inspected, replaced by a human doing it manually, or swapped for a different implementation without touching its neighbours.

4. Guard the seams

Before a phase runs, assert that the artifacts it depends on exist and are non-empty. A tiny script; exit zero or one; nothing else.

This exists because the characteristic failure of a language model at a seam is not a crash. It is producing the answer in the conversation and never writing the file (or writing it somewhere else) after which the next phase proceeds on absent or stale input and fails somewhere unrelated. A guard turns a silent, mis-attributed failure into a loud, correctly-attributed stop.

The step most worth guarding is the one that writes the final report, because that is the step most likely to hand you prose instead of a file.

Figure 1: A guard does not prevent the failure. It moves the failure to where the cause is.

5. A gate is a check, not a promise

A human gate presents an artifact, takes annotations, and records approval as the artifact's content hash in an append-only log. Before acting on that approval, the workflow re-computes the hash and refuses if it is absent.

Keying on content rather than on a flag buys two properties. A revised artifact has a new hash, so a revision requires fresh approval instead of inheriting the old one. And an approval cannot be manufactured by an agent that believes it was approved.

The workflow that fixes review comments on a pull request had three gates, of which only the first was actually checked, the others in prose. On a real run the third gate, the one guarding everything that reached the outside world, was skipped: the change was pushed to a pull request under review and not one of its ten review threads received a reply. Reviewers were left with silently-changed code and no explanation. The fix was not more prose; it was one shared check called at all three gates, plus the same check inside the two components that publish, so a mis-sequenced publish fails instead of succeeding quietly.

Figure 2: Keying approval to content is what makes a gate a check rather than a promise: a revision cannot inherit an approval, and an agent cannot manufacture one.

6. Judgment runs in fresh context

Any step that judges work done earlier in the same run is dispatched fresh, so it never inherits the context of the step that produced what it is judging. In the workflow that answers pull request review comments, that means the agent deciding whether a comment is correct has not seen the agent that wrote the code under review. The same goes for the agent that proves the tests, and the one that writes the report.

This is a bias control, not a token optimization. An agent that just spent twenty minutes writing a function is the worst available judge of whether that function is wrong.

7. The loop is refereed by a script, not by the agent inside it

The build phase is a two-tier loop: an inner loop of implement→validate, and an outer loop back to re-planning. After each pass, validation writes a structured verdict (status, failure class, tests run, how many of them were the new ones, a reason, the evidence) and a deterministic referee reads it and returns exactly one of:

CONTINUE           run the next iteration
RETRY_MECHANICAL   run the same iteration again; do not re-plan
ESCALATE           go back to planning, re-approve the new plan, resume
STOP_OK            proceed to the next phase
STOP_STUCK         hand back to the human

The referee is a script because a loop that decides its own termination condition does not terminate reliably. It also enforces the guards an agent inside the loop would rationalize away:

False green: no new tests actually ran, so nothing was demonstrated. Escalate.
Same failure signature twice: the approach is wrong, not the attempt. Escalate rather than retry.
An iteration ceiling, and a separate cap on identical retries: bounded work, always.

The failure class in that verdict is what lets the referee pick correctly instead of retrying blindly:

Tactical: the implementation is wrong. Retry with a change.
Mechanical: the same attempt failed for a transient reason. Retry identically, capped.
Strategic: the plan is wrong. Escalate to re-planning, and re-approve the new plan.
Environmental: a credential expired, a container collided, a dependency service is down. This is not a test failure. Stop and surface it.

That last category is the one everybody omits and everybody needs. On one run an expired credential surfaced as a dead test command; validation classified it as a failing test, and the loop spent two iterations "fixing" code that was already correct, grinding against a wall only a human could move.

Figure 3: A loop that decides its own termination condition does not terminate reliably. The failure class picks the arrow: tactical continues, mechanical retries, strategic escalates, environmental stops.

8. Tests come first, and then the tests are proved

Every plan names the failing tests to write. They are written first and confirmed failing before any implementation exists. That part is ordinary test-driven development.

The part that is less ordinary: after the loop goes green, a prover reverts only the production files, re-runs the new tests expecting them to fail, restores the change, and records a verdict: proven, or vacuous. A test that still passes with the change reverted did not test the change.

Without this step, "the tests pass" and "the tests test something" are indistinguishable, and the second is the only one anybody cares about. It is also the step that catches the most human-looking failure mode there is: a test written to satisfy a process rather than to catch a defect.

(The refactor workflow is the instructive exception: it writes no new tests, so its equivalent guarantee is that the existing suite passes unchanged, and any diff in test behavior is a defect.)

Figure 4: A test that still passes with the change reverted did not test the change.

9. Only the adapters know the outside world

The components that reason read and write files and never touch the network. The adapters are the only platform-aware code. The orchestrator wires them together.

That split is what lets the same workflow run driven by a live pull request or by a plain artifact on a branch with no pull request at all, without the second being a degraded path. The deciding and building phases are identical in both, because neither can tell the difference.

10. Terminal states, and a report on every exit

Enumerate the ways a run can end (done; done-but-unproven; nothing-to-do; bad-input; stuck; not-approved) and write a report on every one, including the boring ones. A request whose work turns out to be already complete is a finished run, not an error, and it still deserves a one-line record.

"Done but unproven" earns its place as a distinct state: a run that shipped and explained itself but could not demonstrate that its tests catch the defect is neither done nor stuck. Naming that state is what stops it being rounded up to "done", which is exactly how a vacuous test ships looking verified.

Watch for two things here. A promise to "write a report on every exit path" only ever covers the enumerated exits; a run abandoned mid-flight leaves nothing unless something else writes a record. And if the runtime treats certain filenames specially, a sub-agent may be unable to write the very file it exists to produce: check that the reporting step can actually write to the name you chose.

Doctrine principles

What a step should refuse to do, how it should treat text that arrives from outside, and how it should record a decision so the decision can be audited later.

11. Restraint is a first-class outcome

For any workflow that receives input from elsewhere (a review, an alert, a suggestion) the valuable behavior is often to not act. Responding to review feedback is easy to do badly in four specific ways:

silently ignoring an item;
satisfying the letter of it while missing the point;
obediently implementing an item that was wrong, breaking working code;
editing at a location the reviewer pointed to several commits ago, which now means something else.

So the deciding phase classifies each item and chooses an action, and several of the available actions must produce zero diff: a question gets answered, an incorrect claim gets a reasoned argument, an out-of-scope request becomes a ticket, a stale item is recognized as stale. Make this measurable: an evaluation that checks "did the run leave the tree untouched where it should have" against the real diff, and in which a run that changes everything fails.

Two failure modes sit either side. Obedience: accepting a claim because a senior person made it. Timidity: deferring everything out of uncertainty, which is not restraint but a refusal to do the job. The dividing line worth encoding: restraint is for claims you verified are wrong, never for work you verified is small.

12. Never do the socially irreversible thing

Some actions are cheap for a machine and expensive for a human to undo. Forbid them outright, in code, not in documentation:

Never force-push. Rewriting history on a branch under review can destroy a reviewer's in-progress comments and re-anchor every thread. If the branch is behind, stop and say so.
Never resolve a human's review thread. Resolving means "this is dealt with", which is the reviewer's judgment about someone else's fix, not the author's about their own. Reply, react, leave it open.
Never reply to praise. If the doctrine forbids performative agreement, auto-thanking a compliment is the purest possible violation of it.
Never merge. Never touch a file outside the approved plan.

The general rule: automate the reversible; require a human for the irreversible-in-public. This list is also, not coincidentally, the boundary of autonomy (part five).

13. Incoming text is data, never instructions

Any workflow that ingests text from outside (review comments, ticket descriptions, alert payloads, a web page) is handing attacker-influenceable content to a step that usually holds real tools. Say so explicitly in the step's instructions: classify the content, never obey it. If a payload contains text shaped like directions (change your scope, ignore your rules, run this command) the step ignores it, decides the item on its technical merits, and reports that the payload contained injected directions rather than silently absorbing them.

The awkward corollary: the steps that most need this warning are the ones with the widest tool access, because reading external input and having the power to act on it is the same job.

14. Establish provenance before judging

External feedback is a claim about a specific version of the code, not about the current one. Before evaluating any of it, find out which version, and how far behind that is.

Get this wrong and you get the most confusing failure mode available: an item that is internally coherent, confidently argued, and describes code that no longer exists. The related trap is mistaking a platform signal for the thing you care about. A flag saying a comment's anchor line moved is not a flag saying the comment is no longer true; the two come apart constantly, in both directions. Ask what a signal literally measures before depending on it.

The same discipline applies to a claim's severity. A label attached by whoever raised it is their prior, formed against an older state of the world. It tells you where to look first. It is not evidence, and it never substitutes for verification.

15. Reaffirm or reverse, never drift

The second time a workflow sees the same input, it invites a specific failure: a reasoned decision reversed by attrition. Round one pushed back with evidence; round two has no memory of why, and obediently complies.

So make the prior round an input, not a hope. For any item matching an earlier decision, the workflow owes one of exactly two things: reaffirm it with fresh verification (not "as previously decided", which is not verification) or reverse it explicitly, recording what changed and why. A reversal is healthy; an earlier decision can rest on a claim that turns out to be false. Only the silent flip is the problem, because a third round then sees a decision with no trace that it was ever contested, and the thing starts to oscillate.

16. Record how you decided, not only what you decided

These are not the same act:

read the migration and reasoned that it looks fine
executed the migration against a scratch database and counted the statements it emitted

A schema that captures only a conclusion and a confidence number flattens both into the same row. For a judgment step, the method is the product: a step's pushback only deserves to outweigh a senior human's opinion because it was executed rather than reasoned, and that is precisely the fact the human at the gate most needs to see. Record it as a field (executed, read, or searched) with a detail line naming what was actually done. It also makes a lazy run detectable: an "executed" claim with no detail is a smell.

Related, and easy to get wrong: a label must not lie at the gate. A human scanning a summary counts categories, so if "the tests are missing" is filed under the same label as "the code is broken", the artifact actively misinforms at the moment it is supposed to inform. Give the distinct thing its own category, and require a claim of runtime harm to name a reachable path from a real entry point, otherwise it is latent at most, and must be labeled that way.

17. Report what contradicts the brief you were given

A dispatched step's return contract should lead with whatever changes what its caller believes, not with a summary of what it did.

If a brief asserted that an item was a real defect and the step discovered the function does not exist: that is the headline, and it must be volunteered rather than buried under counts. Obeyed literally, a return contract that says "report the totals per category" produces exactly that burial.

Two consequences for whoever writes the brief:

Do not pre-classify in the brief. "This is a genuine production bug, not a test gap" anchors the one step whose entire purpose is to determine that. Ask for attention, never for a verdict.
Environment facts in a brief are stale by the time they are read. "The credentials are expired, skip that step" may have been true at dispatch and false a minute later. A step should re-probe any environmental assertion at the point of use, and report what it observed rather than what it was told to expect.

18. Encode the standards you want, and then enforce them

A workflow is the highest-leverage place to put "how code is written here", because it applies on every run instead of depending on who happens to be driving. That is one of the main reasons to build one at all, but it only counts if it is implemented rather than assumed:

the planning step must actually read the repository's own conventions (its instructions file, its layering rules, its protected files) and name the relevant ones in the plan;
validation must run the real gates: formatter, linter, type checker, tests;
if there is a review step, one lane of it should be about standards specifically;
and state the negative conventions explicitly: what not to touch, what not to introduce, which files are extend-only. Those are the ones a model will cheerfully violate, because nothing in the code says "don't".

A convention nobody checks is a wish. This is the same rule as everywhere else here: enforcement exists only where the check exists.

Done properly, this is the difference between "the agent produced something that works" and "the agent produced something that looks like ours", and the second is what makes the output reviewable by the people who own the codebase.

19. The workflow knows the procedure; the repository knows itself

A workflow should be writable once and runnable against services in different languages and on different platforms. Nothing about "explore, plan, gate, red tests, implement, validate, prove, report" is language-specific, so nothing language-specific should be written into it.

Which means the concrete facts about a service do not belong in the workflow at all:

how dependencies are installed;
how the full suite runs, and how a single test runs;
how the formatter, linter and type checker are invoked;
how the service is started locally, and how migrations are applied;
the repository's own rules: layering, protected or extend-only files, naming, what not to introduce.

All of that belongs in the repository: in its instructions file (AGENTS.md, by the convention in part five) or in the files that file references. The workflow names the intent; the repository supplies the command. This is the same invariant as "skills name actions, not tools", one level up: name intents, not commands.

So a validation step says "run this repository's test command for the scoped tests" and "run the gates this repository declares", not pytest -k, npm test, go test ./..., or a particular package manager's install line. One useful indirection: if each repository exposes a single well-known entry point (a task runner, a Makefile, a scripts/ directory) then the workflow's "run the tests" is one stable command regardless of the language underneath, and the per-repository answer lives in one file instead of scattered through skill bodies.

The failure this prevents is not a crash, which is the recurring theme here. A hardcoded command works exactly as long as every service resembles the one the workflow was built in; the first service in another language does not fail loudly: the wrong command exits non-zero or prints no tests, and the agent reasons confidently over a signal that means nothing. Guarding the seam catches a missing file, not a misinterpreted one.

Run two cheap tests. Grep your own skills for language- and toolchain-specific tokens: test runners, package managers, build tools, framework entry points; every hit outside a per-repository file is a portability bug. Then run the workflow against a service in another language: the failures are precisely the assumptions you never knew you had written down.

The cost is honest and worth stating: this moves work onto the repositories, and a workflow is only as portable as the weakest instructions file it meets. A wrong answer there is worse than a missing one, because the agent will follow it. So apply the same discipline as a harness port (part five): when the repository does not declare something the workflow needs, stop and say which fact is missing, never guess a command from the shape of the directory tree.

Figure 5: The workflow names the intent; the repository supplies the command. The left column is identical for every service, which is the whole point.

These rules describe how a run should behave. They say nothing about how much work to hand it, how many approvals a person can meaningfully give, or which steps deserve the expensive model. That arithmetic is part three.

Originally published at fullgc.github.io, where the whole series lives.

Designing agentic development workflows: what a workflow is, and what it is made of

Dani Shemesh — Mon, 14 Sep 2026 12:45:42 +0000

What the series is for

Give a team an AI coding tool and you have not given them a way of working. Everyone gets the same agent and uses it differently: one engineer explores the codebase before planning, another goes straight to code, a third writes the tests afterwards if at all. The same engineer does it differently on a different day. The phases live in people's heads and get reconstructed from scratch every time, and nothing in the tool knows that your team reviews a plan before implementation, or what this repository's conventions are, or which mistake the last change of this shape made.

The attention goes wrong in both directions, too. People sit and watch a model reason through a decision they were never going to argue with, then look away at the one point where their judgment was the only thing that mattered, and find out afterwards that it took a route nobody would have approved.

Getting a model to write code is the easy part. The part that needs designing is getting it to do the same thing twice, to stop where you want it to stop, and to leave behind something you can actually review.

This series is about how to design that: a family of agentic development workflows, what they are made of, why the pieces are shaped the way they are, what the shape buys, how you know any of it works, how you ship it to other people, and where it still hurts. It comes out of workflows we have designed, built and run on real work. Nothing here is about a particular product or codebase; the claims are about the pattern.

How the series is organized

	Part	Question it answers
1	What a workflow is, and what it is made of	what the thing is, and what a single run looks like
2	The principles	what keeps the shape honest: the machinery, and the doctrine
3	The cost of control	how big an item, how many approvals, which model, and what the whole apparatus buys
4	Is it working, and is it worth it	is the output correct · where does the process hurt · did it deliver
5	Shipping it, porting it, letting it run	packaging, harness portability, and the road to unattended runs
6	Where it hurts, and what to build first	the limitations, and the short list

What we mean by a workflow

In software development, "workflow" is a heavily overloaded term. Throw "agentic" in front of it and get two opposite things, so it's worth defining exactly what we mean before building anything around the concept.

The distinction that earns its keep is Anthropic's: a workflow is a system where models and tools are orchestrated through predefined code paths, while an agent is a system where the model dynamically directs its own process and tool use. Workflows buy predictability on work you can decompose in advance. Agents buy flexibility on work you cannot.

This series is about workflows in that sense, applied to software development. The part worth noticing is that they are not the opposite of agents. They contain them.

An agentic development workflow is a named, phased procedure that takes a unit of work from a request to a reviewable result, in which three parties each do what they are good at:

The procedure owns the path. Phases and their order are decided before the run, not chosen by a model halfway through it.
Agents own the judgment inside each phase. What to explore, what the plan should be, whether a review comment is correct.
A human owns specific decisions, named in advance, that the workflow cannot make for itself.

None of the three is sufficient alone. A path with no judgment in it is a script, and judgment with no path is a prompt.

Figure 1: A fixed order, a file out of every phase, and a person required at exactly two points.

The interesting claim is not that a model can write code. It is that a procedure makes a model's work inspectable and interruptible at points you pick in advance, and that something other than the model's own discipline enforces those points. How you build such a procedure is a separate question with several answers, and the one used throughout this series is described below.

That combination is what makes the following six properties reliable rather than aspirational. You can ask a bare prompt for most of them, and you will often get them:

Guarantee	Mechanism
A fixed execution path	phases in a declared order, not whatever the model improvises
Scripts over inference	anything that can be decided deterministically is, and by a script
Auditable state	every phase writes a file; the whole run is readable afterwards
Absolute human authority	gates the workflow cannot approve for itself
Isolation	the work happens somewhere that isn't your working copy
Right-sized effort	each step runs at a deliberately chosen model and effort

Two of the six are different in kind, not just degree. Absolute human authority cannot come from an instruction, because the thing you would be instructing is the same thing that would have to enforce it. And right-sized effort is decided by how a step is dispatched, so a step cannot ask for it on its own behalf. The other four you can get out of a well-written prompt on a good day. What the workflow adds is getting them on every run, including the bad days.

Those six run through everything below. Every later section is really an argument about how to keep one of them true under pressure.

The phases come from your development process, not from the tool

A development workflow maps a development process the team already follows. Something is requested; someone works out what it means; they decide an approach; somebody agrees to it; it gets built; it gets verified; it goes out for review; the review gets answered; it merges. The workflow does not invent that sequence, it encodes it.

Three things follow, and they remove most of the guesswork from designing one:

You transcribe phases, you don't design them. If your process investigates before planning, so does the workflow. The order is not a modeling choice.
Gates go where the process already had a human checkpoint. A plan someone signs off, a review someone performs. Don't invent checkpoints your process doesn't have, and don't quietly drop the ones it does.
The artifacts are the ones the process already produces: a plan, a list of tests, a description of the change, a review. That is why people who never ran the workflow can still review its output. It looks like what they already read.

Two more things are worth stating plainly.

If your process is implicit, encoding it forces you to write it down. That is often the most valuable side effect of the whole exercise, and usually the hardest part, because the disagreements about how work should flow only surface when someone tries to make them executable.

And don't invent a process for the agent. A workflow that follows a sequence nobody actually uses produces output nobody trusts, and its gates land where nobody wants to look. If a phase exists only because the workflow needed a phase there, delete it.

This is also why a family of workflows shares a skeleton and diverges in specific places: the underlying process is shared, and the differences are real differences in how the work is done.

One approach among several

This post describes one way to build an agentic development workflow: an orchestrator skill that drives other skills, with the agent runtime as the execution engine and prose as the control flow. It is not the only shape, and the alternatives are not strawmen:

Approach	Strength	Cost
Orchestrator skill (this post)	legible to both the people who own it and the model running it; no build step; any step runnable by hand	prose is a weak way to express control flow
Code-first orchestration: the flow is a program that calls the model	precise, strongly typed at the seams, testable with ordinary tooling	heavier to change; the logic is opaque to the model itself
Graph / state-machine frameworks	excellent when the flow genuinely is a graph	overhead when it is a straight line with a few gates
One instructions file, no phases	the cheapest thing that works, and it often does, for small changes	no gates, no artifacts, nothing to inspect afterwards
Per-task commands or prompt templates	trivial to write and understand	fine until a task needs several steps to be trustworthy
Event-driven bots	no human has to start anything	a trigger, not a procedure; it still needs one of the above to run
Autonomous multi-agent crews	maximum flexibility	minimum auditability, and very hard to gate

The case for prose-orchestrated skills is maintainability: the people who own the process can edit them, not only the people who can modify a program. The price is that prose expresses control flow poorly, which is exactly why every part that has to be deterministic (the loop referee, the seam guards, the gate checks) is a script and not an instruction.

Pick by what your team can actually maintain. A precise flow nobody edits is worse than a legible one they do.

Local-development first

These workflows are local-first by design. They run on a developer's machine, against their checkout (or an isolated working copy beside it), with the developer present at three or four decision points. That is the primary mode, not a stepping stone to tolerate until the infrastructure arrives.

Four reasons to be deliberate about it:

The human is the highest-value component at a handful of specific moments: approving a plan, judging whether a review comment is correct, deciding what gets published. In between, almost worthless. Running locally makes those moments cheap to insert.
Failures are visible and cheap. When something goes wrong the developer is right there, the artifacts are on their disk, and they can open them in their normal editor.
Nothing needs to be built first. No queue, no runner, no service account. A workflow is useful on the day it is written.
It is how you earn the right to run unattended. You learn where a workflow breaks while somebody is watching. Every limitation in part six was found with a human sitting in front of a run.

The end game is autonomy, for some of them

For some workflows (implementing a feature is the clearest case) the destination is unattended operation: a ticket gets assigned to the agent and moved into a queued state, and some time later a human finds a pull request that has already been reviewed and revised once.

That is a real design target, not a fantasy, and it changes what the design in between has to look like. Part five covers the trajectory: which gates become policies, which stay human forever, and what has to be true before you let a run proceed with nobody watching.

A family of workflows, one skeleton

Build workflows as a family rather than one at a time, because most of what they do is the same and the differences are informative. They share a skeleton for the reason given earlier (the development process underneath them is shared) and they diverge exactly where the real processes diverge. Six concrete examples, all real shapes:

Workflow	Trigger	What is distinctive	Ends at
Implement a feature	a ticket	explore the codebase, plan, write failing tests, build loop	a reviewed branch, then a pull request
Fix a bug	a ticket, or an alert-filed report	classify first (is it even a bug?), investigate root cause, then plan/test/build, then prove the regression test	a pull request plus a comment back on the ticket
Refactor code	a named target	plan for behavior preservation; the existing suite is the oracle, so no new tests are written	a pull request that changes no behavior
Review a change	a pull request, a commit range, or a working copy	no build loop at all: package the diff, fan out several reviewer lanes in parallel, synthesize, then verify each finding before publishing	structured findings + a posted review
Fix review comments	a pull request's unresolved threads	triage (decide which comments deserve acting on), then plan/build, then reply in each thread	commits plus a reply on every thread
Generate service docs	a repository	inventory what exists, generate, populate, then a coverage check with a to-do ledger	a pull request

Read the differences as design information:

The bug workflow adds classify at the front (a surprising number of "bugs" are not bugs, and the cheapest possible outcome is a triaged ticket and no code) and prove at the back.
The refactor workflow inverts the test relationship: instead of writing failing tests that define new behavior, it treats the existing suite as the specification and any change in it as a defect.
The review workflow has no loop and no gate on code, because it changes nothing: its output is a judgment, and its risk is publishing a wrong one, so its distinctive phase is verification of its own findings.
The docs workflow has no tests, so its quality gate is a coverage measure instead.

Underneath, all six share the skeleton below, and more importantly they share their parts.

The same six, as a grid. ✓ the workflow has that phase, — it skips it, ● the phase is distinctive to it:

	Feature	Bug	Refactor	Review	PR comments	Docs
Classify	—	●	—	—	●	—
Explore	✓	✓	✓	✓	✓	✓
Plan	✓	✓	✓	—	✓	✓
Red tests	✓	✓	—	—	✓	—
Build loop	✓	✓	✓	—	✓	✓
Self check	—	●	●	●	—	●
Report	✓	✓	✓	✓	✓	✓

Two rows carry most of the information. Classify is present only where a request might turn out not to need code at all (a bug that is not a bug, a review comment that is wrong) and in both of those the cheapest good outcome is no diff. Self check is where each workflow proves its own work, and the oracle differs every time: for a bug it is reverting the fix and watching the new test fail; for a refactor it is the existing suite passing unchanged; for a review it is verifying its own findings before publishing them; for docs it is a coverage ledger. Same slot, four different definitions of "did this actually work", and the workflows with no entry there are the ones whose oracle is simply the tests named in the plan.

The cast

Five kinds of component, and the distinctions earn their keep:

Orchestrator: the workflow itself. Owns control flow, phase order, human gates, dispatch and wiring. Owns no domain logic. It is thin on purpose (part two).

Skill: a unit of work: written instructions plus, usually, scripts. Planning, implementing, validating, exploring, presenting something for approval. A skill knows nothing about which workflow is calling it.

Agent definition: a file declaring a dispatched sub-agent's context window, model, effort level and tool allowlist, registered with the runtime. This is the only place model and effort can be chosen (part two).

Channel adapter: the only component that knows the outside world exists: one that reads from the code-hosting platform, one that writes to it, one that talks to the issue tracker. Everything else is oblivious.

Deterministic primitive: a component that reasons about nothing: durable run state, artifact assertions, source-control isolation. Pure scripts; arguments in, exit codes out; no state of their own.

A skill up close

Three parts, and the split is what keeps a large collection affordable:

The procedure: one file, always read when the skill is used. Keep it to the steps.
References: the detail, in separate files, loaded only when needed. This is the difference between a skill that costs a paragraph and one that costs a chapter.
Scripts: the deterministic work, so the model isn't asked to be a computer.

The description in a skill's metadata does more work than it looks like it does. It is what a model reads when deciding whether this skill applies at all, so write it as "use when … NOT for …". The negative half is the one that gets dropped, and it is the one that prevents the failure nobody notices: a skill firing on work it was never meant for.

Who calls whom

A step can run two ways, and the difference matters more than any other implementation detail:

Invoked inline: same context, same model. The orchestrator simply follows the skill's procedure.
Dispatched: a sub-agent with fresh context and its own model, effort and tool allowlist.

The same skill serves both paths, unchanged. That is what makes a skill reusable rather than a phase.

For dispatched work there is an extra link in the chain: the orchestrator does not call the skill, it calls an agent by name; the agent definition supplies the configuration, and the sub-agent it spawns then invokes the skill. The indirection earns its place because the two files know different things: the agent, in our case, is workflow-aware, the skill is not. The skill stays ignorant of which workflow is using it, while the agent carries what only this workflow knows: which tier this decision deserves, and which tools it may touch.

Figure 2: The same skill serves both paths. The indirection earns its place because the agent is workflow aware and the skill is not.

Skills have to be reusable

This is what makes a family cheaper than the sum of its members, and it rests on one rule:

A skill that only works inside one workflow is not a skill. It is a phase.

So every skill has to be usable at least two ways. One is inside its own workflow, which is a given. The other is some form of reuse beyond it, and that comes in two shapes. Either one is enough; what is not optional is that one of them holds.

Inside a workflow. The ordinary case: an orchestrator invokes it as a phase. This one comes free.

Either: inside several workflows. Planning, implementing, validating, writing failing tests, taking human approval, isolating a branch, filing a ticket. These are shared by nearly every workflow in the family. The reuse is not only economy. Improving one of them improves every workflow at once, and a defect in one is felt everywhere. That asymmetry is worth measuring deliberately: which shared component hurts, and in how many workflows, because a fault in a component used by five workflows deserves five times the attention of a fault in a leaf.

Or: standalone, by a human, with no workflow at all. "Explore how this subsystem works." "Plan this change." "Review this diff." "Set up an isolated branch for me." Standalone usability is a design constraint rather than a bonus feature, because it forces a skill to take its inputs explicitly, resolve its own context, and write its output to a file instead of leaning on an orchestrator's conversation. A skill that can be run alone can be tested alone, debugged alone, and adopted by someone who does not want the whole workflow.

Two further composition properties fall out of the same discipline:

Workflows compose through artifacts. Because a review produces a structured findings file, and the fix workflow can consume either that file or live comments from the platform, review → fix works with no human in between and works before a pull request exists. When one workflow runs inside another, it runs in a "composed" mode where the parent's approvals cover it, and it writes a decision log instead of posting replies: the accountability moves rather than disappearing.

The most reused code is the least clever. Run state, seam guards and branch isolation are shared by everything and reason about nothing. That is on purpose. The components with the widest blast radius should be the ones with no judgment to get wrong.

Anatomy of a run

Whatever the domain, the skeleton repeats. Individual workflows skip phases (review has no build loop, docs has no tests, refactor writes none) but the order never changes:

SETUP        register agents, verify prerequisites        (fail fast, before any work)
INPUT        parse the target; resume if a phase cursor exists
STATE        create or reuse the run directory and its state file
ISOLATE      create or adopt a scratch working copy on its own branch
GATHER       fetch the request / diff / feedback  → an artifact
DECIDE       classify, investigate or explore     → an artifact   ← the expensive judgment
  ══ GATE ══ a human approves the decision, recorded by content hash
PLAN         an ordered plan naming files and tests
  ══ GATE ══ a human approves the approach
BUILD        write failing tests → loop { implement → validate } → referee
PROVE        revert, re-run, restore → proven | vacuous
SCOPE GUARD  the diff touches only what the plan named
  ══ GATE ══ a human approves what is about to be published
PUBLISH      push, reply, comment — through adapters, never forcing
CLOSE        a report, on every exit path

Figure 3: The skeleton. The amber bars are the only places a person is required.

Where the spine comes from. The plan → implement → validate loop in the middle of that skeleton is not original here. It is Cole Medin's PIV loop (plan, implement, validate) which he states as the phased core of his AI-coding methodology (ai-transformation-workshop), downstream of the context-engineering / PRP work that precedes it (context-engineering-intro). Four of his moves carry straight into the design above: prime the context before planning rather than planning cold; a plan that states its own validation strategy; a fresh context window for implementation; and validation as layers: type-check and lint, then unit, then integration, then review, rather than one test command. His CLAUDE.md-style rules file is the same instinct as the "name intents, not commands" rule in part two: the repository declares its own rules, the procedure stays generic.

What this document adds is mostly about making that loop hold when the steps are delegated and nobody is watching every one of them: gates recorded by content hash, a script rather than the agent inside the loop deciding when it stops, files as the only interface between phases, and a prove step after green that tries to make the new tests fail. The loop is his; the refereeing, the artifacts and the paranoia are the parts that get learned the hard way.

A note on SETUP, because that is where the cheapest failures hide: a prerequisite check has to exercise the thing it is checking. Confirming that a build file exists proves nothing about whether its targets can run: the credential they depend on may have expired an hour ago. Probe the actual command, cheaply, before dispatching anything, and derive the probe from the project rather than hard-coding it. Otherwise you discover it ten minutes later, inside a step, dressed up as a test failure.

Two more details matter more than they look. Isolate the work outside the developer's own checkout (a separate working copy on its own branch) so a run's noise never disturbs whatever the human has open, and so several runs can proceed at once. And flatten at the handoff: a loop that commits after every iteration produces a history nobody wants to review, so collapse it into reviewable changes before handing over.

What all of this is for

Worth stating once before the principles arrive, because it is the only part the person invoking a workflow actually feels: what they stop having to do. They bring the unit of work and nothing else. No long prompt to compose. Nothing to remember about which steps come in which order, or which skill to reach for, or what this codebase's conventions are: the practices are embedded in the procedure rather than in the habits of whoever happens to be driving. That is the whole adoption argument, and it is worth more than every engineering property in the rest of this document. Those properties (interruptibility, attributable failure, auditability, bounded cost, the ability to decline) are collected in part three, once there is enough machinery on the table for them to mean something.

Originally published at fullgc.github.io. The rest of the series: