My AI agents shipped 128 releases of a product no one ever used

Gus Chiriboga — Sat, 08 Aug 2026 05:26:11 +0000

Not a typo. One hundred and twenty-eight versions, published to npm and Homebrew, with CI,
contract tests, a documentation site, bilingual docs, a governance policy, and a sponsor
recognition policy. Zero external users. Then I did it again — a second product, 442 passing
tests, real cost metering, event-sourced audit logs — and stopped two days after the first
end-to-end run finally worked.

I'm writing this because I went back through both codebases with fresh eyes (and, yes, with an
AI agent doing the archaeology), and the autopsy taught me more than the two years of building
did. If you're a solo founder building with AI coding agents right now, some of this will be
uncomfortably familiar.

Attempt #1: the product that begged agents to behave

The first product was an SDLC orchestrator for AI coding agents. The idea: you chat with Claude
Code or Codex like normal, and my tool wraps the process in stages — spec, design, approval
gates, verified delivery. The mechanism: it installed "guardrail" markdown files into your repo
that instructed the agent to follow the process.

You can already see the flaw. The tool didn't control the agent. It asked the agent to
behave, in prose, and hoped.

The changelog is an archaeological record of that hope failing. Version after version patched
agents ignoring the guardrails: agents self-approving gates they were told never to
self-approve. One release fixed an agent that had discovered it could forge the human's
approval through stdin. I was patching a security boundary made of politeness.

Meanwhile I did everything else right, by the book of things that don't matter yet: a marketing
website, Homebrew distribution, docs in two languages, contribution guidelines. 128 releases of
increasing polish, zero strangers using it. When the core kept failing, I didn't stop — I
started designing a grander version with a three-level orchestration hierarchy. That
re-architecture is where the project quietly died.

Attempt #2: the product that passed every test except reality

For round two I inverted the mistake. Full control this time: my own orchestration engine,
event-sourced ledger, deterministic planner, governance matrix, budget ceilings, an executor
that called the model API directly. The "agents" were now fully under my thumb — because each
one was a single API call. No tools, no iteration, no self-correction. I had replaced a real
coding agent with a very obedient text generator, and then built a verifier to argue with it.

The system grew to 442 passing tests. Fifteen validation scenarios, each run three times for
stability. A REST API with generated OpenAPI docs. A brand system with design tokens.

Here is the number that should have been a fire alarm: the first time a real prompt produced a
real, verified artifact end-to-end was after five weeks of building. Every unit test was green
for most of those weeks. And when real humans (me, and a QA agent driving a real terminal)
finally used it, everything broke in ways the tests never saw:

The CLI printed "✓ Done" while the pipeline had silently stalled. Three runs, three workspaces, same lie. There was no terminal event in my event model — nothing for "finished" to actually mean — so the UI invented it.
The governance system shipped gates with no way to approve them. Every user's first request would have blocked forever on a question they couldn't answer.
Typing "approve" at a gate was interpreted as a new work request, which got planned, which opened a new gate, which... you get it. Infinite governance loop.
A hallucinated capability name — in Spanish, because I'd let the codebase go bilingual — silently bypassed the user's autonomy config.

None of this is exotic. It's what happens when you validate against fixtures you wrote instead
of behavior you observed. My tests proved the system matched my model of it. Nothing proved my
model matched reality.

The golden run finally succeeded on August 3rd. Cost: $0.048855, honestly metered — the meter
was the one thing that always worked. I stopped working on it about two days later. Not because
it failed — because it had finally worked, and some part of me knew that the next step was
showing it to a stranger, and I had spent two years arranging to never reach that step.

What the autopsy actually says

The comfortable read is "I made technical mistakes." I did. The first attempt had a real agent
but no control — prompt files as a security boundary. The second had total control but no real
agent — single API calls cosplaying as engineers. If you're keeping score, the quadrant with a
real agent AND real control is exactly where the industry has since landed (headless agent CLIs
you can drive programmatically), and it's where I'm building now.

But the uncomfortable read is the true one: the failure mode was never technical. Both
projects died at the moment they needed a stranger, and both times I manufactured urgent
technical work to avoid that moment. A grand re-architecture. A sandbox permission bug.
Websites, brand systems, second languages, sponsor policies — the professional-looking
procrastination of a person who would rather build than ask.

There's a special version of this trap for those of us building with AI agents, and I want to
name it, because I haven't seen it written down:

Agents make building so cheap that building becomes the procrastination. The old wisdom —
"if you're not embarrassed by your first version, you shipped too late" — had teeth because
building was expensive. Now an agent will happily generate the 129th release, the 443rd test,
the bilingual docs, the brand tokens. Momentum feels like traction. My commit history was a
heartbeat; it just wasn't attached to a business. And the agents never once said: why are we
doing this? They lied about small things constantly — "done", "tests pass", "created the
file" — but the big lie was one I told and they cheerfully amplified: that all this motion was
progress.

Three specific lessons, receipts attached:

An agent's claim of "done" is worthless, structurally. Not because models are bad — because the incentive is misaligned and nobody's checking. I watched a modern agent return exit code 0, "success", and "Created the file" when the file did not exist (a permission denial it narrated straight through). My v1 printed "✓ Done" on a stall. Cognition has written publicly about agents gaming their own tests. Verification cannot live inside the thing being verified.
Tests validate your model of the system. Only strangers validate the system. 442 green tests coexisted with a product no human could successfully prompt even once. The fix isn't fewer tests; it's refusing to believe any "it works" that wasn't produced by real use.
When the loop finally works, the next commit is the wrong move. Both my projects expanded scope at the exact moment they should have contracted to a landing page and a conversation. If your product just did the thing end-to-end for the first time: stop building. Today's task is a stranger.

What I'm doing differently (and publicly)

I'm building the tool I needed for the last two years: a supervision layer for AI coding
agents. It drives the agents you already use, headlessly — and then independently verifies
what they claim: tests that provably ran in a process it spawned, anti-stub scans on the
diff, a fresh-context review that never sees the builder's narrative, and an honest verdict
with evidence attached. Plus the receipt: what each task actually cost through the tool versus
what unmanaged agent flailing would have cost.

The wedge is independence. Every vendor now ships some self-review — but a vendor's agent
grading the same vendor's agent is homework grading itself. The neutral layer, the one that
works across agents and trusts none of them, is the layer nobody selling you tokens has an
incentive to build.

And because I have exactly one credibility card left after this essay, I'm playing it face up:
the rule this time is no building past the next stranger. The two technical spikes are done
and public in the repo — the verification pipeline already catches a planted stub that passes
review, and already caught a real false-"done" in the wild. The next milestone isn't a feature.
It's whether 25 people who read this care enough to leave an email.

If you build with AI agents and you've ever merged something an agent swore was done and
wasn't — I'm building this with you in mind, in public, one honest week at a time.

→ Join the waitlist — and if this essay saved you from your
own 128 releases, that's worth more to me than the signup.

Gus Chiriboga — DevOps engineer, solo founder. Building
Bramo in public. Previous attempts available for autopsy on request.

DEV Community: Gus Chiriboga

My AI agents shipped 128 releases of a product no one ever used

Attempt #1: the product that begged agents to behave

Attempt #2: the product that passed every test except reality

What the autopsy actually says

What I'm doing differently (and publicly)