AI Writes Code Faster. Why aren't we delivering?

My linkedIn: https://www.linkedin.com/in/kristiyan-stoyanov-395310184/

If you prefer video format:

AI is generating more and more of our code. But if an engineer can produce the implementation much faster and the feature still takes just as long to reach a customer, what have we actually improved? That is the million-dollar question for me: where is the productivity gain going?

The code might be finished while an engineer is still checking it manually. Then the merge request waits for review. After merge, there is more validation, more waiting, and eventually a release. We have accelerated one part of the software development lifecycle, but a change still has to get through the rest of it.

My argument is that faster coding becomes faster delivery when we redesign the work around it. We need to look at the checks we run, the decisions we ask reviewers to make, the confidence engineers have in those checks, and what we measure as an organization. This is where I think lead engineers and engineering leaders can make a substantial difference.

A manifesto for software delivery

I've put that argument into six commitments. They are the principles I would use when deciding how to change an engineering organization's practices:

• Delivery is the outcome. Useful changes need to reach people. Generating more code means little if we cannot get it into production and keep it working.
• Confidence needs evidence. Our checks need to connect to the behavior we intended and the real systems the change will interact with.
• Automation must change the work. Agree which effort the automation replaces. If everyone still repeats the same manual steps, we have more work to do.
• Trust is earned together. Engineers need dependable feedback and an agreement that they can rely on it. That takes a conversation with the team as well as better tooling.
• Scrutiny follows consequences. Choose controls according to what this particular change could cause, how quickly we would notice, and whether we could reverse its effects.
• Learning outlives the change. Carry discoveries into the next piece of work: better fixtures, better context, better checks, and better decisions about the process itself.

The useful discussion is what these commitments change in an actual team's day. So let's start where the generated code leaves us: the agent says it is done. What would make us confident enough to send that change for review?
The manifesto: https://krisitown.github.io/agentic-software-delivery-manifesto/

Story Time

I had an agent implementing a change in an application that orchestrates calls to several downstream services. I wrote the specification, had the agent build the local integration tests, and reviewed the result. The tests passed.

There was a problem: the agent had made an assumption about a downstream API contract. Think of a field appearing one level deeper in a JSON response than expected. A believable difference. Believable enough that my review missed it too.

The implementation and the test fixtures agreed with each other. Unfortunately, agreement was possible because they shared the same wrong assumption.

That is the kind of experience I keep coming back to when thinking about AI and productivity. Generating the code had become easier. Establishing that we could deliver it still required work, and some of that work was happening too late.

I think there is a substantial opportunity here. But getting it requires changes to how we develop, review, validate, and release software. Buying everyone an agent and keeping the rest of the process intact leaves a lot of that opportunity sitting in queues.

What does a passing test actually tell us?

In my own practice, I have found spec-driven development useful. I work with an agent to turn a business requirement into a specification, then spend time reviewing that specification. This is one of the places where I want to be quite deliberate. If we misunderstand the behavior we need, faster implementation will help us build the misunderstanding faster.

From the specification, the agent writes tests that exercise behavior at a higher level than individual methods. I tend to call these integration tests: they run locally, with Testcontainers for dependencies such as databases and WireMock for controlled responses from downstream services. The agent can run them repeatedly while implementing the change.

This is my short feedback loop. It gives the agent something concrete to work against and gives me evidence I can inspect before sending the change for review.

The local loop connects intended behavior, implementation, and repeatable checks.

The human work includes reviewing the important scenarios and their expected results. I also want to establish that the tests fail for the missing behavior before implementation. Otherwise, we may have written a test that passes without requiring the feature at all. If the agent changes an expectation during implementation, that deserves attention too. There may be a good reason, but weakening the expectation just to get green is not progress.

Even when we do all of this, green has a boundary. It tells us that the implementation satisfies the expectations encoded in those tests. The expectations can still be incomplete or wrong, as they were in my API example.

There is nothing especially mysterious about this failure. An engineer integrating with a service would normally inspect its documentation, send requests through Postman or Insomnia, and look at the responses. If we ask an agent to implement the same integration without that context, we have omitted part of the work we would have done ourselves.

One practical thing I like to provide is working curl requests against the service in an integrated environment. The agent can inspect the actual response and use it to build realistic fixtures. Where appropriate, read-only access to a development database helps for similar reasons: it can see the schema and the shape of the data instead of guessing.

Real examples improve the starting point. They still need to be challenged by running the application with real dependencies.

The longer loop should improve the shorter one

After the change is merged and deployed to an integration or development environment, I want another feedback loop: end-to-end checks that exercise important behavior with the real applications involved.

This is where the discrepancy in an assumed API contract can become visible. WireMock returns what we told it to return. The downstream service returns what it actually returns. Those are different sources of evidence, and the difference matters.

A convincing mock can agree with the implementation and still disagree with the real API.

When this loop finds a problem, fixing the implementation is only part of the job. Correct the local fixtures as well. Carry the discovery back into the tests and context the next engineer or agent will use. Otherwise, we pay for the same lesson again.

I would include relevant performance and other nonfunctional checks here too. The appropriate suite depends on the application, but its output needs to tell us enough to act: what failed, under which conditions, and what we observed. That information can go back into the next change rather than leaving someone to reconstruct the failure manually.

The longer loop costs more time. That is precisely why I want it to improve the shorter one. Over time, more of what we learn about the real system should become available earlier in development. I would also be cautious about removing a useful check simply because it has been quiet for a while. First understand what failure it is protecting against.

Why is the engineer still checking everything manually?

Suppose you have these tests, yet engineers still run through the feature themselves before opening a merge request. It is tempting to call this a trust problem and ask people to change their habits.

I would first ask what they are trying to find out.

Perhaps the tests cover the happy path and miss the scenario they are worried about. Perhaps the fixtures have been wrong before. Perhaps the useful end-to-end test runs after merge, while the engineer needs to make a decision before merge. A result that arrives later cannot settle the question they have now.

Manual checking can be a perfectly rational response to missing or late evidence. Understanding that gives us a concrete improvement to make: improve the test, move feedback earlier, or agree which problems can safely be discovered later.

There is a psychological side as well. You may feel less comfortable with code you did not write. You may have learned that a defect escaping your change will become a personal embarrassment, even if you followed the team's process. Under those conditions, repeating the checks is understandable.

As a lead or manager, you cannot resolve that simply by announcing that the automation is trustworthy. The team needs dependable feedback and a shared agreement about when relying on it is acceptable. When something escapes that agreed process, the response needs to include improving the process. Otherwise, people will quietly keep their personal safety routines.

This is why I think automation must come with a conversation about the work it changes. If we add automated checks and retain every manual step indefinitely, we may improve coverage, but we should not assume we have reduced delivery effort.

The 70-file merge request is still waiting

Another pattern I have seen is the giant AI-generated merge request: 70-plus files, ready for a human to review. If implementation has accelerated and mandatory review still depends on the same small group of people, there is an obvious place for the queue to grow.

Faster implementation can leave more work waiting for the same reviewers.

Smaller changes help when they make the purpose and consequences easier to understand. But splitting a large diff into several arbitrary pieces does not automatically reduce the total thinking required. We need to look at what reviewers are being asked to do.

I want mechanical rules in deterministic checks. Formatting, linting, and rules we can express reliably should not repeatedly consume a person's attention in review. Automated reviewers can also investigate concerns and draw attention to suspicious behavior. Humans miss things; agents miss things. I would judge a review tool by the useful failures it catches and the decisions it supports.

Human review should concentrate on the purpose, structure, evidence, and consequences of the change. Does this solve the intended problem? Does it fit the system? Are the important scenarios represented in the tests? What assumptions remain? What happens if they are wrong?

Those questions can absolutely require reading code closely. They also give that reading a purpose beyond checking every line for personal style preferences.

We need to be explicit about what approval means and who owns it. Adding an automated reviewer without changing anyone's responsibilities can just create another collection of comments for a human to process. Agree which work it takes over, which judgments remain, and who makes the decision to proceed.

How much scrutiny does this change need?

I am uncomfortable with applying the same amount of friction everywhere. I am equally uncomfortable with classifying an entire application as low risk and treating every change accordingly.

A gift-card shop can contain a minor copy edit and a change to how balances are calculated. The application is the same. The consequences of getting those changes wrong are very different.

Choose scrutiny according to what a particular change could cause and whether its effects can be reversed.

For the copy edit, a quick review and a straightforward recovery path may be enough. A balance change deserves much closer attention to expected behavior, failure cases, and what happens to customers if the implementation is wrong.

The questions I would ask are fairly practical: What can fail? How far can the effect spread? How quickly will we notice? Can we reverse the effects?

Canary releases, monitoring, and rollback can help us move confidently, but rolling back code does not undo every customer impact. A wrong balance or an external action may need repair after the deployment itself has been reverted. Before release, someone needs to know how the rollout will be observed and who will respond.

For consequential changes, additional scrutiny is useful work. For less consequential changes, we should be willing to remove steps whose cost no longer buys us meaningful confidence. Both decisions require understanding the change and the business's appetite for risk.

Find where the time actually goes

To know whether any of this helps, I would look at throughput and time to production alongside failures and rework.

Merge-request throughput can point us toward a review bottleneck. It is also sensitive to how we split changes, so a rising count alone is a poor reason to celebrate. Elapsed time to production helps expose a different problem: perhaps changes merge quickly, then wait for a release window, an environment, or another approval.

Look at the waiting between steps as well as the work within them. Faster implementation has limited impact if the change then sits untouched. Equally, faster release is not much of a gain if we spend the following week repairing avoidable failures. Incidents, corrective work, and the effort of recovery belong in the same conversation.

I would start with one recent change and follow its actual journey. Pick one repeated delay you find, then run a bounded experiment around it. Give the improvement an owner who will maintain it and handle failures. Agree beforehand what success would look like.

For example, a team might identify a manual integration check that people repeat because the local fixture is unreliable. Improve that fixture using real responses, add the missing scenario, and compare the waiting time, repeated effort, and failures before and after.

At the end, make a decision together about the manual work you can now reduce. Keep what still provides useful evidence and take what you learned into the next improvement.

That is where I think engineering leadership has a real opportunity. We can help people produce more code, but we also have to make it possible for them to rely on good evidence, spend their attention where it matters, and get useful changes into production.

I would be interested in the specific delay your team is seeing. Once an agent finishes a change, where does it wait longest—and what would you need to know before you were comfortable removing that wait?