Stick to Your Guns: TDD When AI Writes the Code

olsemeno — Sun, 25 Oct 2026 11:00:22 +0000

You probably remember how hard it was to explain to the business why we needed to set aside a substantial percentage of time for writing tests, and why we had to build tools for technical quality control even before we had a product. It would seem that this is already in the past. But is it?

Is there still a place for TDD? And is there still a place for a human among AI agents?

And if so, how do we use this advantage wisely?

A Bit of Research

The first claim I start from: tests before code noticeably improve generation [1].

When tests were added to the prompt, GPT-5-nano's share of unsolved hard tasks dropped from ~76% to ~51%. For all models, pass@1 grew roughly twofold [2].

Second: if tests are written after the code, they turn out to be tied to the current behavior [3]. That is, tests written from the code rather than from the spec systematically tend to lock in the current behavior, bugs included. Besides that, a model tends to ignore a test that contradicts "popular practice" [4].

An important point: when the model writes the tests, the tests and the code "come to an agreement" with each other and miss the requirements together [5].

Tests in the specification, reviewed and locked in by the developer before the code is written, are still an important component and give a tangible gain in both productivity and correctness of the code relative to the documentation.

How to Write Tests Without Sacrificing Productivity?

Writing scenarios by hand would probably be an unaffordable luxury. The pace of service development, as I feel it, is pushing my brain to its limits, and systematizing and constraining the flow of text generated by the model is becoming vitally necessary.

The specification doesn't need to cover all possible scenarios. This is the place where machine and human still work together, and the text still has to be human-readable and straightforward.

According to the TENET study [6], 3–5 tests covering different usage scenarios give a better result than the full test suite: an excess of tests overloads the model and reduces accuracy.

But TENET used smaller tasks. In my experience, breaking the work into subtasks and covering each with 5–10 main scenarios is an excellent choice. This way it's still easy to hold the context, and it's more than enough to steer code generation in the right direction.

There is another interesting option — TiCoder [7]. The model clarifies your intent through tests before producing code. This works great on small but critical and high-load parts of the code. But on large standard tasks there is a noticeable loss of time.

Organizing Tests

For describing test scenarios I set aside the bottom part of the spec. I often go back to it when I need to refresh my knowledge of some functionality: the PRD doesn't give the depth of elaboration that has already been done, and the spec is overloaded with technical information.

I describe cases in one or two sentences. It's important to include the expected result — this is the main value of a test when developing with AI. I avoid mocks at all costs and work out scenarios with external teams as much as possible, to reproduce every case. I ask for webhooks on the external API side to activate the needed case. This way the agent gets the maximum opportunity to catch a potential problem.

With this approach, any mistake by one of the teams is caught as fast as possible by the tests of the other teams. Work goes according to pre-agreed APIs and specs.

In the spec it looks quite short and readable:

Draft can be saved with zero audiences; publish and re-activate return 400 until at least one is attached. Once attached, publish succeeds; wiping attachments then activate → 400 again

In the code, each test already needs to be expanded to simplify review. I leave the description of each test in this format:

/// TEST-ID: GA-U4
/// REQUIREMENT: §5.2.1 — an unknown order is a typed refusal, not a transport error
/// LEVEL: e2e (live partner)
/// MOCKS: none
/// REAL:  Platinum Staging over HTTPS
/// GIVEN: an order id no one has ever created
/// WHEN:  GET /orders asks for it
/// THEN:  the client surfaces ORDER_NOT_FOUND as a stable code
///
/// The poller leans on this: an order Platinum has never heard of must be
/// distinguishable from an edge that is down, because one is a young order
/// whose creation is still in flight and the other is an outage. Both arrive
/// as an Err, and only the code tells them apart.

Such comments are convenient to review together with the QA team. Eagle-eyed QA engineers will quickly fix the missed spots, and at the same time understand the developer's way of thinking and where bugs are possible.

Mocks and Unit Tests

In my practice, unit tests and tests with mocks give a small gain in code accuracy. Very often the model locks in wrong behavior with unit tests and then builds on it.

IMHO, if an integration test is impossible, the right scenario is a detailed review of the mock and the input/output. This is where models have the biggest room for mistakes.

If I can cover all the functionality described in the PRD with e2e tests, I do without unit tests.

Without real requests to external APIs, models almost always make mistakes. This needs to be taken into account, and the environment should be built with the possibility of such communication already at the design stage.

Case: A Race Condition Between Services

A good example. I joined an existing project that uses a third-party service: it can run simple scripts, store a user profile, and has a decent UI and API. One of my first tasks was to deal with a race condition between the services. This is a rather complex engineering task, and for a new person on the project — a real challenge. There were no integration tests on the project, so every attempt to reproduce the bug manually or with mocks failed, and the bug kept coming back to me, causing irritation first for me and then for QA (understandably).

In such a situation the model is no helper at all: analyzing one system out of two is not enough.

The solution was to agree with the team on spinning up a test environment. I used dev containers and a new test environment in the external service with limited keys. As soon as this tool appeared, fixing the hard task became a matter of a few hours. And the work on the infrastructure for e2e tests itself took no more than a couple of days — on a big service of a huge fintech project. Those two days paid off on the very first hard task.

In my experience, e2e tests are also the best entry point for a new employee: having on hand many tests related specifically to their feature, they get into the context faster. This is a great addition to reading the code.

Endgame

So is there still a place for TDD? The more I work with AI, the more I'm convinced that TDD opens up in a new way in the current conditions. And there is still a place for a human too, it has just shifted: the model writes the code, and the human decides what counts as correct behavior and locks it in tests before the first line of code.

What I take into every project:

test scenarios in the spec, written and locked in before the code;
5–10 key scenarios per subtask, not full coverage;
e2e with real partner APIs instead of mocks wherever possible;
a description of each test: requirement, level, what's real, what's mocked, and why the test matters;
tests from the spec are untouchable, the model's tests live and change together with the code.

The next step I see is QA and DEV working together on the technical spec. IMHO the model should get the maximum of input data at the moment the code is written, while in the classic approach the input data is prepared by the developer, and documents from QA come into work only after the code is written. The earlier QA scenarios appear in the spec, the fewer bugs will make it to them.

After that, we can set the model loose. And all that's left for us is to roll up our sleeves and, with pleasure, turn the result into something elegant and, most importantly, understandable :)

References

Mathews, N. S., Nagappan, M. Test-Driven Development for Code Generation. ASE 2024.
Ben Sghaier, O., Delcourt, K., Sahraoui, H. Artificial or Just Artful? Do LLMs Bend the Rules in Programming? 2025 (BigCodeBench-Hard).
Mathews, N. S., Nagappan, M. Design choices made by LLM-based test generators prevent them from finding bugs. 2024.
Cui, Y. Tests as Prompt: A Test-Driven-Development Benchmark for LLM Code Generation (WebApp1K). 2025.
Yu, H., Li, K., et al. TDD-Agent: Test-Driven Reasoning for Code Generation. 2026.
Hu, Y., Jiang, N., et al. TENET: Leveraging Tests Beyond Validation for Code Generation. 2025.
Fakhoury, S., Naik, A., Sakkas, G., Chakraborty, S., Lahiri, S. K. LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluation. IEEE TSE, 2024.

DEV Community: olsemeno