DEV Community

Will Macpherson
Will Macpherson

Posted on Originally published at getantidote.com

The Autonomous Software Factory is Already Here (But You're Assembling It Wrong)

The factory already exists

AI coding models didn't just "improve" over the last two years; they blew past anything most teams were prepared for. When OpenAI introduced SWE-bench Verified in August 2024, GPT-4o paired with the Agentless scaffold resolved 33.2% of real GitHub issues in that human-validated subset.

By April 2026, Anthropic reported that its restricted Claude Mythos Preview resolved 93.9% of SWE-bench Verified tasks, against 80.8% for Opus 4.6.

At the same time, AI assistants quietly became co-authors of modern codebases. As early as 2023, GitHub reported that Copilot was behind an average of 46% of its users' code, and 61% for Java developers. By January 2026, developers in Sonar's survey said AI accounted for 42% of the code they commit, and expected 65% by 2027.

Engineering leaders look at those numbers and see "velocity." Models resolve most benchmark tickets, AI assistants ghost-write a large share of the code, and in one controlled experiment developers with Copilot finished a bounded task 55.8% faster. On paper, the autonomous software factory has arrived. But in production, something else happened.

Why velocity data misleads

Google's DevOps Research and Assessment (DORA) program tells a different story about what AI did to delivery. In the 2024 report, every 25% increase in AI adoption was associated with an estimated 1.5% decrease in delivery throughput and a 7.2% decrease in delivery stability. In the 2025 report, throughput turned positive, but AI adoption was still associated with lower stability, for a second consecutive year.

Pull-request data shows where the work piles up. Across 8.1 million PRs, LinearB's 2026 benchmarks found that AI-generated pull requests wait 4.6 times longer before review and are accepted at just 32.7%, versus 84.4% for manual PRs. In Harness's survey of 900 engineers and leaders, 72% of organizations reported at least one production incident caused by AI-generated code, and respondents said 45% of deployments involving AI-generated code lead to problems. In other words: we really are writing code faster, but shipping has become more fragile.

243% more incidents per merged PR at teams with high AI adoption, while task throughput per developer rose only 34%. 441% rise in median time spent in code review at the same teams, while PRs merged with no review rose 31%. (Faros AI, AI Engineering Report 2026)

If AI agents can blast out hundreds of lines of syntactically perfect code and open a pull request in seconds, the bottleneck doesn't disappear; it moves downstream.

The queue shifted from "typing" to "trust."

Review is one kind of check

One response to the trust deficit has been to bring AI into review itself. CodeRabbit and Qodo are examples of AI code review tools that analyze pull requests, diffs and repository history for bugs, logic gaps, missing tests and security issues. Cursor and other AI IDEs run inline reviews and suggest patches in the editor.

These tools help teams keep up with the flood of AI-generated changes. Review, whether human or AI, answers a particular question: is this change correct and well made? A unit test can confirm that a calculateDiscount() function returns the right number. Whether a real customer can redeem that discount after an SSO login, a Stripe webhook, an email send and a browser-side state update across three microfrontends is a different question, about the running system, and it needs a different kind of check.

The trust deficit of the AI era

As AI-authored code floods repos, teams are forced into a harsh trade-off in staging. You either stall the pipeline and pour human hours into manual regression testing, or you chase "velocity," skip staging, and let your paying customers discover what breaks.

DORA's stability finding suggests many teams are paying for it later. This creates the ultimate paradox of the AI era: on paper, the pipeline looks fully modern, but in practice, the human is violently pulled back into the loop right before the software faces another human.

96% of developers don't fully trust AI code to be correct, yet only 48% always check it before committing. (Sonar, January 2026)

34% of those who had an incident couldn't tell within 24 hours whether AI code caused it, though 87% of respondents were confident they could. (GitLab and Harris Poll, June 2026)

Electric Twin saw exactly this while building its own software factory: a security scan ran green for a week while real vulnerabilities sat in the code.

"No check caught the problem; an engineer's judgement did." — Jordan Smith, Software Engineer, Electric Twin

Observed quality vs. assumed quality

Teams get trapped because their tools only measure assumed quality ("The tests passed; ship it"). To close the loop, teams must transition to observed quality ("We watched the actual experience under realistic conditions, and it behaves as intended").

This requires checking the running product as well as the code. Infrastructure and configuration are a common place for failures to hide:

10 of 16 major cloud outages in 2025 were caused by configuration or deployment changes, across AWS, Azure, Google Cloud and Cloudflare. (Control Plane, Cloud outage report 2026)

A verification stack that only looks at application code will tell you the application is perfectly healthy while the underlying permissions, network policies, and state mismatches quietly rot. You cannot catch multi-system failures by asking "Does this method behave?"

You have to ask, "Can the user still sign in and pay?"

Where the bottleneck goes next

The teams that will dominate the next decade of software are not waiting for a mythical model that writes bug-free code. They let AI generate code at breakneck speed, let agents review it for syntax, and use autonomous verification to operate the product in staging exactly as a user would.

48% of developers chose "I trust it when I can easily verify the output", the most common answer. Only 6.6% trust AI with important decisions. (Stack Overflow Developer Survey 2026)

Closing that loop is not the end of the story; it's simply the ticket to the next game. When development and QA both drop toward near-zero time, the bottleneck shifts entirely upstream. The future constraint on your business will not be how fast your engineers can type, but how fast you can plan, architect, and encode product strategy as machine-readable intent.

But you don't get to worry about out-planning your competitors tomorrow until you stop drowning in your own PR reviews today. The autonomous software factory is already here, and the models are maxed out. The only question left is whether you will assemble the missing verification layer before your instability tax eats the rest of your velocity.


Originally published on the Antidote blog. I work at Antidote, where we build autonomous verification for web apps.

Top comments (0)