These days, it appears that anyone can write code. You’ve probably seen plenty of videos online of non-developers using AI to generate working scripts in no time. Code that actually seems to work! It’s like magic.
And it is kind of like magic. These next-generation models like ChatGPT emerged onto the scene in 2023 and blew almost everyone’s minds; even non-technical people were amazed by all the things AI could do, coding being just one of them.
And not only that, it is getting better and smarter by the month. The pace of improvement is absolutely astounding. It seems everything is about to change.
It's a Br-AI-ve New World
But wait… doesn’t that mean developers are about to be replaced by AI?
Surely if grandma can now divide-and-conquer a data structure using an AI-generated merge sort algorithm in O(log n) time without breaking a sweat, how could it not mean exactly that?
Maybe it’d be smart to start looking elsewhere for work. How about gardening? Plumbing? Yoga instructor influencer? Anywhere where we might shield ourselves from this sudden AI disruption.
Well, hold on just a moment. In this new world, code can indeed be generated by AI. Great code, too. But here’s the kicker: how can we gain confidence that this code will actually do what it is supposed to do, in a good way, and in an ever-changing environment? Especially within the context of a software system consisting of other AI-generated code, with which it is going to have to integrate.
How can we as developers harness and benefit from the vast intelligence of these new AI companions, while also ensuring that our customers’ complex software systems remain maintainable, scale well, and function without bugs?
That is the problem that AIDD solves.
In this article and its accompanied video, we'll explore the eight core steps of AI-Driven Development and see how this workflow could have the potential to supercharge your life as a developer. And how it does so not by fighting against—nor surrendering—to the AI, but by joining forces with them.
The Value Of Human Development Skills
But before we do, for the developers out there who might still be concerned about the rise of Artificial Intelligence and how it could affect their job security. What exactly are the reasons why your skills and creativity as a developer will remain crucial to the software construction process?
Here are just five of the many exciting responsibilities you can expect as a future AIDD developer. Some you will be familiar with, some are brand new.
Be the one who has a deep and thorough understanding of the vast space of vague human customer requirements.
Carefully guide the AI through the AIDD process.
-
Create your application’s testing pyramid. From unit to integration to E2E tests. Yes, the AI can and will help you out here, but you are ultimately responsible for the health and maintenance of the test suite.
Repair and modify systems built using AIDD.
Use your skills as a developer to make sure the system fulfils customer requirements while also respecting the many principles of professional software construction:
- Correct (vs buggy, crashing, doing nothing, or not executing)
- Stable (vs brittle, or non-deterministic)
- Readable (vs unreadable, or obfuscating)
- Testable (vs tightly coupled)
- Scalable (vs tightly coupled)
- Extensible (vs tightly coupled)
- Flexible (vs tightly coupled)
- Reusable (vs tightly coupled)
- Cohesive (vs low cohesion)
- Maintainable (vs not DRY, no documentation, dead code, etc)
- Performant (vs slow, and/or expensive)
- Secure (vs insecure)
- Usable (vs frustrating user experience)
- Accessible (vs only for privileged users)
- Portable (vs tightly coupled to specific medium)
- Consistent (vs paradigms or coding styles or formatting mixed together without forethought)
- Minimal (vs over-engineered)
These five competencies—dealing with human customers and their ambiguous requirements, AI guidance, test suite creation/maintenance, repair/modification of AI-created systems, wisdom of high-level software principles—are just some of the reasons why you will still be valuable in the workplace as a developer.
For the foreseeable future, it is reasonable to assume that many of these skills won’t be taken over by AI.
So What Is AI-Driven Development?
AIDD follows the 'red, green, refactor' cycle, just like Test-Driven Development (TDD). It also employs the technique of writing tests before implementation code. In a way, we can say that AIDD is like a futuristic extension of TDD.
However, unlike in traditional TDD where developers are responsible for creating both unit tests and implementation code on their own, AIDD introduces a new approach, emphasizing deep AI teamwork.
Employing the AIDD technique, you are no longer working alone; you have a smart and patient AI ally by your side, ready to come to your aid at almost every stage of development. Instead of you doing the heavy lifting, your AI companion goes to work behind the scenes while you can focus on higher-level development tasks.
The 8 Steps of AIDD
To demo the technique, let’s first go through the steps at a general level. After that, in a video we’ll go concrete with a simple yet practical example, where we construct an isolated function to achieve some goal.
This way, you will start to get a feel for how this process works, and how it can really empower you in delivering value to your customers.
1: Set the goal
Think about the function at an extremely high level; what are you trying to achieve? Consider it only in terms of Input→Output.
☝🏻 Reflect on the function’s API, the ergonomics of actually using the function. Specifically, how many and what kind of arguments should it take? Should they be optional or not? Etc.
2: Formulate the abstract type
Use a strongly-typed language—TypeScript, C#, Haskell—whatever you are using in your project, to manually write the type or interface for the function's input and the output. Strive always for pure functions, unless you have absolutely no choice but to cause side effects.
☝🏻 If you need help in this or any other step, as always—ask the AI.
3: Construct mock functionality
Using the function type you made, give them to the AI and ask it to construct a mock function. This is a function without an implementation; it simply simulates a real function by faking the output.
4: Write tests
Ask the AI to create unit tests for this yet-non-existent-function, as many as you need, based on the function type and a description of what you are trying to achieve.
Carefully review the tests it creates. If necessary, add more manually, until you get the feeling that if these tests pass, you will actually feel confident in the function’s correctness.
Here's another possible way to do step 4: write all the tests yourself, but with only one assertion for each test case. Then, you can ask the AI to generate 5, 10, 20 more similar assertions for each.
With more assertions per unique test case, the probability of the function passing the test by sheer luck decreases.
☝🏻 The unit tests should be lightning fast. This is achieved by not relying on any external state or circumstance, employing techniques like the heavy use of pure functions, monads like Promise or Maybe, dependency injection, mocking, etc.
5: Run tests and expect failure ❌
Run the tests using the mocked function and watch them dramatically fail.
If the majority of the tests are not failing—some probably will pass due to luck—return to step 4 and rewrite the tests to be more comprehensive and all-encompassing.
6: Create concrete implementation
Now finally, it’s time for the magical last step of TDD and AIDD: the refactoring. It is here we actually create and then improve on the function.
Give the AI the type or types you made in step 2. Or give it a simple verbal explanation. Or even all the unit tests you made. And unleash it!
7: Run tests and expect success ✅
Use the function it creates and, hopefully, watch them gloriously pass!
If they don't—which will probably happen quite frequently, at least now in 2023—collaborate with the AI to achieve the goal of the tests passing. You could:
- Ask the AI to try again, supplying any error messages and ideas you might have to help it out.
Or:
- Manually review the function and see if you can fix it, or implement it yourself. Ask the AI for ideas to help you out.
8: Refactor
Once all the tests have passed, it's important to review the function manually and assess whether it aligns with the spirit of professional software construction mentioned earlier. If not, maybe consider refactoring the function.
☝🏻 Again, if you're still not sure why you won't be replaced by AI anytime soon, the step above is one of the main reasons.
If the tests start to fail during the refactoring process, keep collaborating with the AI, following the same approach as in step 7, until all the tests pass again.
(Optional Final Step): Celebrate!
All done! Congratulate yourself and celebrate the fact that you are now one step closer to delivering happiness to your customer! 🎉
☝🏻 In this article we are focusing on unit testing. However, I believe that in general AIDD is applicable to the other testing strategies as well. That said, it should be acknowledged that these other tests typically cover more complex scenarios and therefore require further manual effort and careful thought from the developer.
☝🏻 Although the principles of TDD and AIDD are close to identical, it's not until you start developing that you begin to see the significant differences between the two approaches.
AI-Driven Development in the Wild
Finally, let’s see an actual example of how AIDD can be used in the real world.
Why not attempt to follow along with your own AI companion?
If you did follow along with the video, pause for just a moment and take in the fact you didn’t actually create that function yourself. You didn’t even create all the unit tests yourself!
Instead, you were like a god; guiding your creation with subtle nudges—yet rarely directly intervening.
That, at least to me, is mind-blowing.
Dawid Dahl is a full-stack developer at UMAIN | ARC. In his free time, he enjoys philosophy, analog synthesizers, consciousness, and being with friends and family.
Credit for pyramid and cycle graphics: Jenny Eckerud.
TS/Jest CodeSandbox template used in the video.
Top comments (10)
Interesting read. I think you've covered some pretty important key concepts here. The iterative nature of development with AI and the need for human verification. All of the online discourse I've seen about using AI to generate code uses programming languages which don't produce provably correct code. It takes a human to look at it and say, "This is/isn't what I wanted." What you're talking about seems to take that into account. Testing is going to be extremely important in the future if AI starts to gain wider adoption. As I've used technology like ChatGPT, I've found the best way to get what you want is through iterative prompts. Even then, you may not get exactly what you wanted or you may get something totally incorrect. For example, I've seen problems with ThreeJS code generation from ChatGPT.
Thank you kindly for your comment! And yeah iterative prompts is the way to go. I love how that is possible to do as it actually remembers what happened previously in the conversation, and can build upon it.
I tried many chatbots over the last 10 years and this is the first time when rational iteration-over-time has actually been working. Not perfectly of course, but it’s actually working so well that I’m trusting it more and more these days to give me what I expect.
Really exciting stuff and I can’t wait for it to get even better.
Amazing! I feel like a standart, perhaps this, or something similar could truly enable us to harness the power of the AI, while not polluting the application with badly generated code. As you write also, writing tests and tests is a key concept here (as well :P)!
Thank you so much for reading, Adam! Yeah, I have a lot of hope in this approach. My next article will take it up a notch and show how to work with integration/system testing using AIDD and not just unit testing. Going to be fun! 🥳
So well written and intricate, good read for me as i want to get into testing more!
Thanks so much! Yes testing is super fun.
You know the feedback you get in the frontend when you can visually see something breaking. That gives it all a much more tangible/physical feel right?
Automated testing gives that kind of instant-feedback-quality to backend work as well, making it not so abstract. Which I personally find is great! 🙂
Also AI of course!
So damn good! An actually interesting, well written article amongst the throng of "How to do X in React"
Great article. Interesting read!
Thanks a lot for reading! 🙏🏻