DEV Community

Justin McCarthy
Justin McCarthy

Posted on

AI Coding Agents Need to Keep Their Promises

At AIE CODE SUMMIT 2026, I introduced a notion of brownfield factories. I encourage you to watch the full keynote, but the talk form is intentionally a motivational speech without much technical detail. This article provides some technical direction on how to mine your existing codebase and your existing sources of context to loop faster, cheaper, and safer in your existing code estate.

Although it's attractive to wish we could also search for better, the definition of better is a moving target, especially in the agentic moment where customer expectations are rapidly shifting thanks to Muse, Grokbot, and other forms of consumer AI adoption. Better is in the eye of the beholder. But cheaper isn't.

Cheaper and faster are objective. Cheaper and faster, while not making other trade-offs, are universally good. Safer, as we've all realized from the October 2026 vulnpocalypse, is also an objective good that is rarely a topic of debate. All of us could go on vacation, and if our respective products were faster, cheaper, and safer when we returned, we'd all be pretty thankful.

Achieving these objectively good goals is increasingly within model capabilities, and increasingly these can be pursued autonomously. We would typically use the language “hill climbing” to describe this process. However, when hill climbing on operation cost, it would be easy to unintentionally incur a performance penalty.

At Diffusion, we refer to these tensions as Promise Loops, where some dimension that we value is sustained by one or many loops. Promise Loops are run continuously on a product or code estate, and these loops give us increasing confidence that the system will not autonomously trade performance for security or functionality for cost.

A fancy term for this is a Pareto equilibrium. In a Pareto equilibrium, we have no further moves to make that can benefit one of our priorities without detracting from another.

In some hopeful version of agentic coding, our system should stabilize to a Pareto equilibrium the moment we stop talking. The equilibrium could be perturbed by our future changes or a change in the world. For example, an increase or decrease in customer behavior. Ideally, the loops are autonomously stabilizing the system.

Turning from the theoretical to the practical, what are some concrete techniques which will lead to increasingly autonomous stabilization of our promises?

It should be more than obvious at this point, but for the avoidance of doubt: all of your telemetry systems and all of your error log systems are essential signals to stabilize both correctness and your other Promise Loops in your production systems. It may mean that you need to clone, replicate, or re-index these to have an agentic read profile. I would encourage you to do so. It may entail re-negotiation of some prior commitments around how logs are stored. I would encourage you to do so. If you have to copy these signals into columnar databases that you control and Parquet files in your object buckets, I would encourage you to do so. There’s no excuse for withholding high-resolution facts from your agents.

Another underexploited source of information about the use of your system is code coverage traces. Code coverage has historically been used extensively in test automation.
It does give a bounding box around what customer behavior exercises in the system, and it’s extremely low cost and high fidelity to capture.

Calling on a current trend in the zeitgeist: techniques for formalization. Formalization is a great technique to use for some subproblems, but it’s very easy to be nerd sniped into imagining that we can apply it to every problem. With that disclaimer: go for it!

The history of software testing gives us a useful grab bag of techniques: differential random testing gives the existing system and its replacement the same generated inputs and compares their outputs. Coverage-guided fuzzing uses coverage feedback to find inputs that exercise more of the system. Shadow execution compares the replacement against the existing system on real traffic, with writes isolated. Mutation testing deliberately introduces defects to check whether the tests catch them. Together, these techniques give our promise loops more ways to detect when an improvement has broken behavior we meant to preserve.

Earlier, I imagined a set of Promise Loops improving our system while we were on vacation. But now’s not a time for vacation! Now is a time for ambition. We should be spending our creativity finding new customers, finding new products, and resetting expectations for each other and our firms at a much higher threshold. Of course, ambition also requires that our pager apps not surprise us at 3AM. In other words, we need our existing promises to stay met. The great news is Astra, Fable, Carbon, and friends are ready and willing Brownfield Factory allies.

Top comments (0)