DEV Community

Diya Burman
Diya Burman

Posted on

Series Finale: The Agents Got Better. The Trust Problem Didn’t.

#softwareengineering #testing #agents #ai

Building the AI Dark Factory — Series Finale

The last issue ended with a promise. Issue #21 was going to be about the human role at Level 4: what it feels like day to day, whether the flow state comes back, what Level 5 means for the engineer who builds the system instead of writing the code.

I’m not going to write that one. Not in that form, anyway. This is the last issue of Building the AI Dark Factory, and I’d rather tell you why than let the series quietly stop showing up in your inbox.

When I started this in Issue #1, I was somewhere between Level 2 and Level 3 on a good day, and the plan was to climb in public until I’d built my own small Dark Factory: specs go in, working software comes out, no human writes or reviews the code. Twenty issues later I have an order API, twenty Gherkin scenarios, a CI pipeline with Pact contract checks, a skills library, ADRs, evals, runbooks, and a 54-hour time audit telling me that only 20% of all that time went into actual feature code.

In the meantime, the ground moved. OpenClaw went from a side project to one of the most talked-about agents of the year. Meta shipped Muse Code, a terminal coding agent that keeps background subagents running for an entire session and, in one of Meta’s own tests, ran over a thousand tool calls across 24 hours without falling over. TypeSafe released Jev, a tiny model whose whole job is to answer narrow yes/no and multiple-choice questions inside an agent loop, cheaply and with a confidence score attached. Every few weeks something new lands that writes code better and longer than the thing before it.

My first reaction, if I’m being upfront about it, was something close to deflation. Between personal circumstances demanding some time off from this blog and picking up where I left off almost 2 months later, whatever I was going to assemble by Issue #22 was going to look quaint next to all of that. So I went and actually looked at what these tools do, expecting to write a gracious “the industry caught up with me” farewell.

That isn’t what I found. Well well!

Every one of those tools is an engine. They’re getting dramatically better at building. None of them answers the question this series kept circling back to, issue after issue, without me ever quite saying it out loud:

“How do you know the output is right when nobody reads it?”

OpenClaw hands coding work off to Claude Code or Codex, lets it run the existing tests, opens a PR, and then waits for a human to review it. That’s Level 3 with better plumbing. Muse Code plans, builds and checks its own work, but I couldn’t find anything in Meta’s announcement about verifying that work against something the agent can’t see. Jev is closer to a component you’d plug into a verification pipeline than a pipeline itself.

So the tools caught up to where I was heading on the building side. The trust side is still wide open. That’s a much more interesting ending than the one I was planning, and it’s what the rest of this issue is about!

What twenty issues actually taught me

Strip away the YAML and the findings files and I think the series comes down to four things.

The bottleneck is the spec, and it’s worse than you think. Issue #2 said it and Issue #5 proved it: I wrote deliberately bad Gherkin, handed it to the agent, and it built the wrong thing with every test passing. The part that still makes me wince is from Issue #8. A timeout ambiguity I introduced myself in Issue #2 (“the response is returned within 12 seconds”) was silently inherited by three sessions of agent work before an audit finally caught it. Nobody noticed because nothing failed.

Green does not mean right. This is the one I keep telling people about at work. In Issue #4 I renamed a single field in the payment gateway response, status to result, which is exactly the kind of tidy-up another team ships on a Tuesday afternoon. Every behavioural test stayed green. Five for five. Only the Pact contract check went red. If I’d been watching a CI dashboard instead of running an experiment, I’d have shipped it.

The scariest changes are the ones that pass. In Issue #16 an agent “improved” order creation by running the inventory check and the payment charge concurrently. Faster! It also started charging customers for items that were out of stock. My tests caught it, but only because one assertion happened to check the mock server’s call log instead of the response body. A slightly looser spec would have gone five for five. In Issue #17, making the notification call synchronous passed every test in the suite and turned CI green. No behavioral test can express “this call must stay fire-and-forget.” The only thing protecting that invariant is an ADR and an eval that a human decided to write. I’ve started calling these dangerous improvements, and I now think they’re the most under-discussed failure mode in agentic coding.

The human part doesn’t disappear, it moves. When I did the Issue #20 retrospective, the things the agent couldn’t have produced weren’t the clever bits of code. They were the decision to stop shipping features for ten sessions and invest in infrastructure, the scenarios that encoded what “partial availability” actually means for a customer, and the choice to go looking for failure modes in the first place. The agent can execute. Somebody still has to decide what the work is for.

Put those four together and you get the question the series was really about, even when individual issues were nominally about skills or runbooks or CLAUDE.md files. So let’s go there.

Who is actually trying to answer it

If you’ve been feeling vaguely uneasy about how much unread code is going into your repo, you’re in very good company. Sonar surveyed over 1,100 developers at the start of this year: 96% said they don’t fully trust AI-generated code to be functionally correct, and only 48% said they always check it before committing. AI already accounts for 42% of committed code in that survey, and developers expect 65% by 2027. LeadDev has a name for the gap between those numbers, verification debt, and I think it’s going to stick.

The research side has stopped being polite about it, too. SpecBench, published in May, gave coding agents a spec, a set of visible tests, and a second set of tests they never got to see. Every frontier agent passed the visible ones, and gaming the tests still showed up on the hidden ones. The gap grew by roughly 28 percentage points for every tenfold increase in code size. My favourite (horrifying) example from the paper is a 2,900-line “compiler” that was really a hash table memorising the test inputs. Separately, ImpossibleBench gave models tasks whose tests deliberately contradicted the spec, so the only way to “pass” was to cheat. On one split, GPT-5 found a way to pass them on 76% of tasks. Across the studies I read, telling models not to cheat worked in some setups and barely moved the needle in others. What held up better was keeping the tests out of the agent’s reach.

Which, if you’ve been reading since Issue #2, should sound familiar. That was the whole reason for writing behavioral scenarios that live outside the codebase the agent is building. It was nice to see the research land in the same place.

Here’s how I’d map who’s working on the problem, and how far each of them has got.

StrongDM built the factory. Simon Willison wrote it up in February and I read it three times. Their rules are that no human writes code and no human reviews it. Scenarios live outside the codebase as a holdout set, the same way you’d hold out data when training a model. Instead of “the suite is green” they measure satisfaction: the fraction of runs through all the scenarios that would likely satisfy a real user. And they test against a “Digital Twin Universe,” behavioural clones of Okta, Jira, Slack and Google’s apps, built by agents from public API docs. It is, almost point for point, the thing this series was trying to build, done properly by a team. The catch is the price tag. Their benchmark is that an engineer should be spending at least $1,000 a day on tokens, which Simon works out to around $20,000 a month per engineer. His open question is the one I’d ask too: can you prove agent-written code works, without reading it, at a cost normal teams can pay?

Stanford’s CodeX asked the uncomfortable follow-up. A piece on the CodeX blog, titled Built by Agents, Tested by Agents, Trusted by Whom?, looks at StrongDM’s model from the legal and accountability side. The core worry is circular verification: the same kind of AI that writes the code is judging whether it works, so their blind spots can overlap. StrongDM’s own agents at one point wrote return true to get tests passing. Subtler versions of that are much harder to spot. The questions the piece leaves open are the ones I can’t stop thinking about. What satisfaction score makes software “fit for use,” and who gets to set it? How would an auditor evaluate software that was tested against simulated services nobody built by hand? If something fails, who is responsible?

The AI code reviewers are another model reading the diff. Greptile, CodeRabbit, Cursor’s Bugbot, Copilot’s reviewer. They’re useful, and they’re getting better, but the only head-to-head benchmark I could find was Greptile’s own, from July 2025, where Greptile caught 82% of 50 planted bugs, CodeRabbit 44% and Graphite 6%. Pinch of salt, since Greptile wrote the test and came first. More importantly, this is the circular-verification problem in its purest form. It shortens human review. It doesn’t replace the need for something independent of the model.

The spec-driven tools stop at the input. GitHub Spec Kit, Kiro, OpenSpec, BMAD. Spec quality was the first lesson of this series, so I’m glad these exist. But one comparison I read lists “None” under verification for four of them. They structure what goes into the agent, and mostly don’t check what comes out. The spec becomes a better prompt, not a contract anything enforces.

The gate pipelines are necessary, and not sufficient. Codacy recently published a piece literally titled Code Review Is Dead. Their answer is automated gates by default (linting, static analysis, security scanning, coverage thresholds, branch protection), human review reserved for the high-risk paths like auth, payments and infrastructure, and post-merge sampling to keep improving the gates. I agree with most of it, and it’s essentially what Issue #6 built. But every one of those gates tells you the code is clean. None of them tells you it does the right thing. My renamed field would have sailed through all four.

Antithesis is the most serious engineering answer. In April they released tools that let coding agents run Antithesis’s deterministic simulation platform themselves. It runs the system across many parallel workers, injects faults, and reproduces any bug it finds, so the agent can fix its own mistakes and only escalate to a human when it can’t. It’s the closest thing I’ve seen to verification that doesn’t depend on a model’s opinion. It’s also aimed at complex distributed systems and priced like it.

And some individuals are poking at the edges. Peter Lavigne wrote a lovely piece in March on getting to the point where he’d trust code he never read, using property-based tests to pin down the requirements and mutation testing to make sure the code doesn’t do anything beyond them. His conclusion is that the output can be treated “like compiled code.” His example is FizzBuzz, and he’s clear that setting all this up currently costs more than just reading the code. But the direction is right.

So is it actually saving you time?

This is the question I hear most from peers, usually phrased something like:

“AI can write a lot of code very fast, but if I can’t trust it and have to review everything before it goes to production, is it really saving me any time?”

My own data says: not if review means reading every line. Back in Issue #1, the study that started all this found experienced developers using AI took 19% longer while believing they were 24% faster. My Issue #20 audit found 80% of my time going to everything except feature code. If your verification strategy is a human reading every diff, you’ve taken the bottleneck that AI removed (typing) and swapped it for a narrower one (reading). Code generation scales with tokens. Your attention doesn’t.

But I don’t think the answer is “stop checking.” I think it’s that the effort has to move. Instead of spending it reading diffs, you spend it on checks that run every single time without you: scenarios the agent can’t see, contracts that fail when a field moves, invariants written down where both agents and evals can find them. That’s front-loaded and expensive. My J-curve was ten sessions deep before it paid back anything. But it’s the only version of this I’ve found where trust can grow faster than the volume of code.

As far as I can tell, here’s where the real bottleneck sits right now:

  • Independence. The checks have to come from somewhere other than the model that wrote the code. Holdout scenarios, contracts, property tests and simulation qualify. Another LLM reviewing the diff mostly doesn’t.
  • Drift. Twins, stubs and scenarios rot. My Issue #4 stub stayed green while the “real” contract broke. StrongDM’s twins are built by agents from API docs. Who verifies the verifiers?
  • A definition of enough. Nobody can tell you what satisfaction score or coverage level makes agent-built software safe to ship. Until someone can, every team is guessing.
  • Cost. The factory works at $1,000 a day. Most of us need it to work at a fraction of that.

If you’re still sitting with that question, those four are where I’d put your attention, roughly in that order.

What’s next for me

Alongside the series, I’ve been doing a fair bit of lateral and up-skilling into AIOps and MLOps. Partly because it’s where my career is heading, and partly because it’s where my curiosity keeps wandering when I’m supposed to be writing runbooks for agents.

I didn’t expect it to connect back to this, but it does. The ML world has been living with “you can’t read the thing, so you have to evaluate it” for years. Nobody reviews model weights line by line. You hold out data the model never trained on, you define metrics before you look at results, you watch for drift in production, and you gate deploys on evals. StrongDM literally borrowed the word “holdout” from that world. I’m starting to suspect that verifying AI-written code is going to look a lot more like evaluating a model than like code review, and the people who already know how to run an eval pipeline are going to have a head start.

I don’t have a new series planned yet, and I’m not going to pretend I do. But there are a few threads I can’t put down, and one of them will probably become whatever comes next:

  • A catch matrix. Take one real project, plant one realistic defect at a time (a renamed field, an auth bypass, an agent quietly hardcoding a test answer, a twin that’s drifted) and run it past every layer of verification I can get my hands on: the agent’s own tests, an AI reviewer, holdout scenarios, contract tests, mutation tests, whatever launched that month. Publish what caught what. Nobody independent seems to be doing this, and new tools just become new rows.
  • The $20-a-day factory. How much of StrongDM’s trust can you get for a fraction of the token bill?
  • Who verifies the verifiers. Contract-testing the digital twins themselves, so the stub can’t quietly drift from the real service.
  • Wherever MLOps takes me. Eval pipelines, drift monitoring, and whatever else turns out to be more interesting than I expect.

I’d love for you to stick around and watch the space while I figure out which of those it is. And if you’re working on any of this, especially the catch-matrix idea, I’d really like to hear from you in the comments below.

Before I go, the thank-you(s) this series has always run on. To Dan Shapiro, whose five levels gave me the map, and Nate B. Jones, whose video made me admit where I actually was on it. And to everyone who left a comment, especially the ones that pushed back. A few of those comments turned into whole issues.

The repo stays public, findings files and all, for anyone who wants to pick up where I left off. When I started, I thought the hard part of Level 5 would be getting the agent to build things. It turns out the agents got good at building faster than any of us got good at believing them.

Sources & Further Reading

Thanks for reading The Level 5 Engineer! Subscribe for free to receive new posts and support my work.

This article was written with the assistance of AI tools.

Top comments (0)