At the end of the previous article, our project hit a dead end. The neural network had quietly plugged in the abstract syntax tree parser tree-sitter, turning the codebase into an absolutely unreadable "black box" for me.
We tried to force it to roll back the changes, but bugs started crawling through the generated code. Dependencies broke, and the script stopped running. Assessing the scale of the disaster, I came to a simple but harsh conclusion: it is much easier to write everything from scratch than to try to fix the AI's architectural glitches, especially given my complete lack of Python knowledge.
So our second prototype went into the trash can. We needed a new approach that wouldn't allow the neural network to act on its own. This tight leash became an uncompromising combination: a detailed formulation of requirements and total automated testing.
The New Contract with AI
The problem with previous iterations was that we set tasks like "parse this file and make it pretty". This gave the AI too much freedom.
To bring the architecture under control, we completely changed our workflow. Before writing a single line of code, we now established a strict "contract" with the neural network. Here is how it looked: first, we fixed strict requirements detailing permitted and forbidden libraries. Then, the AI generated a step-by-step plan for my approval. And most importantly, not a single function was written without a pre-created automated test.
How to Write Tests If You Can't Code?
A logical question arises here: how can a system architect who has never written a single line of Python in his life implement Test-Driven Development (TDD)? The answer is simple — the AI writes the tests too.
I didn't need to invent complex verification logic. I would take the original XML generated by Doxygen and find the required fragment there (for example, the description of a single class). Then I would take the beautiful, formatted HTML code that was supposed to be the final output (the exact one developers were used to in the old system).
I would feed both pieces into the chat and set the task: "Here is the input XML. Here is the final HTML that should be output. Write a test that takes this XML, runs it through our parser, and verifies that the output strings match this HTML character by character".
The neural network obediently wrote the test using pytest. Naturally, upon the first run, it failed with an error (since the parser itself didn't exist yet).
Red. Green. Refactor
We took the classic TDD cycle and adapted it slightly for working with AI. When the test failed (Red), the neural network caught the error in the console itself and immediately started debugging. Then came the Green stage: the AI silently fixed the code, re-ran the check, and the test passed. We got a working piece of code doing exactly what was requested. Finally, if the neural network suddenly decided that the algorithm wasn't elegant enough and tried to rewrite it using regular expressions (Refactor), I no longer panicked. We simply ran the tests again. If they were green — let it optimize as much as it wants, but at the slightest hint of red, we immediately rolled back the changes.
The Magic of Isolation
This approach worked flawlessly. The AI found itself strictly limited by our requirements and tests. It could no longer hallucinate non-existent parameters or quietly change the architecture, because any misstep caused the tests comparing the output HTML with the reference to fail.
Development proceeded incredibly fast. We covered all scenarios with tests (classes, methods, enumerations, parameters). The AI wrote the code, ran the checks itself, and we steadily moved on. In the end, we had a perfectly working engine in our hands that took bulky XML from Doxygen and flawlessly turned it into stunningly beautiful HTML pages. The project could be safely deployed to production, if not for one overlooked detail.
The next day, the Infrastructure Lead came to us and was genuinely surprised: "Why are you still fussing around with HTML when there are SSGs, from the beautiful Docusaurus to the super-powerful Hugo?".
I will talk about how this single sentence forced us to rebuild half the project all over again in the next part.
Originally published on our blog: https://blog.flude.guide/blog/tdd-as-a-leash
Also read us:
Top comments (0)