Originally published at echonerve.com
If you've wired up more than one LLM agent into a pipeline, you've probably had a version of this bug: every component reports success, and the system is still wrong. This is a writeup of one, with the fix.
The setup
Three Claude agents, each with a narrow role:
- Research Collector — topic or URL list in, structured digest out
- Draft Writer — digest in, full article out, in a fixed template
- Repurposer — approved article in, four platform-specific posts out
Rough condensed version of the Research Collector's system prompt, for a sense of what "specific" means in practice:
You are a research analyst for EchoNerve.
Given a topic or a list of URLs, produce one structured digest:
- 5-8 key insights, each specific and quotable
- data points with named sources
- gaps the sources do not cover
- 2-3 possible article angles
Extract signal. Ignore noise. Flag it if two sources contradict
each other. Do not summarize — identify what matters.
Getting to that version took six rewrites. V1 was just a role — output read like a form letter. V2 added a required output format, which fixed structure but not judgment (the agent still buried the interesting finding in bullet six). V3–V5 tried telling the agent to "be more interesting," which doesn't work as an instruction because it names a quality, not a behavior. V6 held because it named rules instead of a wish.
This tracks with what Andrej Karpathy has written about default agent failure modes — silent wrong assumptions, over-elaborate solutions, edits outside the requested scope. His guidelines on this collected 220k+ combined GitHub stars, which says something about how common these failure patterns are.
The pipeline as a loop, not a chain
Each agent plugs into what I've been calling the Cowork Loop: Human → Agent → Review → Memory → Automation → Human. I pick the topic and constraints. The two Agent stages run (Research Collector, then Draft Writer). Review is me, reading the draft against what I meant to say — the one stage that isn't automatable, because it's checking output against intent, not output against a style guide. Memory is the context file each agent loads before every run. Automation is the Repurposer, which only fires after I approve the draft.
Skipping Review is the actual failure mode here, not agent quality. An unreviewed draft doesn't produce one bad output — the Repurposer multiplies it across four platforms in one run.
The bug: four systems, four answers to "is this done"
My 12th article went live on WordPress on August 19. At the same time:
- Local source file:
status: draft - Dashboard: 11/12 cornerstones published
- Weekly ledger: 12 scheduled engagement slots silently dropped
Every agent, checked in isolation, did its job correctly. Research was collected correctly. The draft was written correctly. The repurposing ran correctly once the article existed. The bug wasn't in any single component — it was that "published" meant something different to WordPress, to the local file, and to the dashboard, and nothing in the system required those three to agree before anything downstream trusted them.
This is worth generalizing: in any multi-agent pipeline, individually-correct components can still produce a system-level lie, if there's no shared, enforced definition of a state transition (here, "published"). Unit-correctness at the component level doesn't imply integration-correctness at the pipeline level — which, if you've done any distributed systems work, isn't a new lesson, just a new place to apply it.
The fix
A release isn't "done" until five things agree: the live URL, the local source file, the pipeline tracking row, the distribution assets, and the metrics ledger. One record, checked in one place, before anything counts as published. ~5 minutes per publish. Would have caught the August 17 gap same-day instead of four days later.
Coincidentally (or not), OpenAI disclosed the same week that they spend roughly 20% of monitored inference compute on verification for their most capable models, with training auto-pausing if a flag goes unresolved for 30 minutes. Different scale, same principle: verification isn't a paragraph you add after the pipeline works, it's a stage the pipeline needs.
Numbers, six months in
- Content time: 8–10 hrs/week → 2–3 hrs/week (all review/decision work now)
- Output: 1 long-form article + 4 social posts + 2 email briefs, weekly, from one research pass
- First-pass draft quality: 70–80% usable
- Edit pass per piece: 20–30 min
If you're starting from zero
Don't build three agents at once. Build order that actually worked for me:
- Repurposer first (~2 hrs) — take one existing article, require 4 fixed-format platform posts back. Teaches you to write a prompt that produces something publishable before adding complexity.
- Draft Writer second (~1 weekend) — feed it a research note, edit the output. Your job shifts from writing to editing.
- Research Collector last (~half a day) — completes the loop, topic in, published content out, review in the middle where it belongs.
And decide on day one what "done" means across every system that tracks your work. I built that rule after the fact. It would have cost nothing to build it first.
Full article, with the complete agent prompts and the Cowork Loop / Knowledge Flywheel framework details: https://echonerve.com/content-pipeline-automation-3-claude-agents-system/
Top comments (0)