Peng Qian

Posted on Sep 2 Originally published at dataleadsfuture.com

How I Use Aidlc to Improve My Team's AI Code Quality

#programming #ai #productivity #coding

By this August, our team had been writing code with AI for more than half a year. The biggest thing I noticed was that our development speed went up, but code quality stayed pretty rough. Development often took just half a day, but hunting down bugs could take several days.

So recently I spent more than a month restructuring our team's AI coding workflow. I replaced OpenSpec with the AIDLC workflow, using a more detailed software development process and stronger team collaboration. Code quality finally jumped significantly.

In today's article, I want to tell you how we did it.

I put all the source code mentioned in this article at the end. Feel free to grab it and try it out.

This post originally appeared on my personal blog, Data Leads Future, where I keep updating articles and sharing the source code for free.

Introduction

AI coding has been around for more than a year now. People's focus in this field has slowly shifted from which model works better and how much it boosts efficiency, to how we can make sure the code AI generates is actually of good quality.

I recently noticed something. On social media, people talk more and more about questions like "how do I actually make sure my AI coding output has good quality" and "who should be responsible for code AI writes."

If you asked me six months ago, I would have said the best answer was SDD (Spec Driven Development). Based on rule frameworks like SpecKit and OpenSpec, you first talk things through in a question-and-answer style to nail down the requirements and implementation rules. Then you feed the spec files to the LLM as a prompt, and the LLM writes code strictly following those specs. This seemed like the best way to do AI coding.

I even wrote a few articles about tips for using OpenSpec.

It wasn't until I rolled out my method to the whole development team that I realized something. SDD works pretty well for small projects. But once your project gets big, spans several years, and needs collaboration across multiple teams, OpenSpec just doesn't hold up.

What Actually Went Wrong with OpenSpec

When I say it doesn't hold up, let's look at what actually went wrong with SDD frameworks like OpenSpec.

Using it got way too complicated

How many commands from OpenSpec's documentation on GitHub do you still remember? Do you know exactly when to use explore and when to use propose?

When I rolled out OpenSpec to my team, I wrote a whole document explaining when to use each command, and I told everyone to follow the workflow I laid out.

Things still went sideways. Should explore come before propose or after? Why can't I just jump straight to the plan agent? I bet you can't answer that either, right?

Because the commands and their use cases got so complicated, everyone on the team ended up using OpenSpec differently. Some people even used OpenSpec just for show at the start and switched to pure vibe coding afterward. This shows that the extra learning cost didn't really pay off for developers.

It never captured the original intent

Speaking of the difference between the explore command and the plan agent, this brings up another problem. The spec files OpenSpec writes never save the user's original intent or original requirements, and they never keep the conversation the user had with AI during the plan phase.

Because of this, when I reviewed spec files written by another team member, I often felt completely lost. Why was this user story written this way? What problem was this method design actually trying to solve?

Give it enough time, and even the developer who wrote the code might forget why a feature got designed a certain way.

Once you lose the original intent, your project drifts further and further from its original goal with every iteration.

It's hard to get the right level of detail for requirements

OpenSpec's explore mode, or the brainstorm mode in other SDD frameworks, is honestly pretty powerful. This can pull you straight into the trap of wish-based coding without you even noticing. You start hoping that a single sentence is enough, that OpenSpec will chat with you and break your requirements down step by step, then gradually build out a complete product for you.

But AI can't do that yet.

The current generation of LLMs is really good at finding shortcuts. If you describe a vague goal for your project and expect OpenSpec to break it down into fine-grained requirements and iterations, that rarely works. What you get in the end is just a demo with the main features and nothing else. All the detailed implementation is missing, so it's really just a half-finished product that can't go live in a production system.

So experienced developers know that when doing SDD development, the best move is to break the product's features into small pieces and let OpenSpec iterate on them round by round. For example, build the project skeleton and tech stack in the first round, then build the login and authentication system in the second round, and so on. Only this way can a project slowly get polished into a real product.

But it's really hard to control how fine you break these requirements down. Break them too coarse, and your project turns into a demo. Break them too fine and your development speed drops. Sometimes the developer hasn't even thought of some necessary features at the start, so details easily get missed.

Documentation stops matching reality

Next up is the documentation consistency problem. Your team member starts by using OpenSpec to write out the spec docs, then moves on to coding, adjusting the implementation along the way. But because they never explicitly called an OpenSpec command again, none of those later adjustments ever get written back into any document.

There's another situation too. While generating rule artifacts, you notice a problem in tasks.md and fix it, but unless you specifically ask, AI won't go touch design.md on its own.

As your project grows bigger with each iteration, this kind of inconsistency shows up everywhere. Eventually, the spec files completely lose their value as a reference. Then you end up with a pile of messy code.

You never know when to start the process

I once asked a teammate why they still chose vibe coding even though everyone knew SDD could improve code quality.

One reason they gave me was that the commands map to development phases in such a complicated way that they genuinely didn't know when to kick off the process.

Here's an example.

Right after you use apply to finish writing code, you suddenly find a bug in what got generated. Do you ask AI to fix it directly, or do you go run the proposed artifact flow first?

Or say you're in the middle of fixing a bug and suddenly get a great idea to tweak an existing feature, or maybe you just want to delete this chunk of code and rewrite it a different way. Do you run the OpenSpec flow first, or just change it directly?

Team collaboration falls apart

Every open source framework like OpenSpec assumes the same thing. With AI's help, each of us becomes a super developer who can handle everything from start to finish alone.

But everyone on our team has a clearly defined role. Modern business has gotten complicated enough that no single person can handle everything alone anymore.

After you finish writing the requirements analysis doc and user stories, shouldn't you check them with the product team? At the very least, you need to align on how metrics get calculated, right? The same goes for architecture design and coding plans. Shouldn't you run those docs by other developers or an architect? What if the design approach has a mistake somewhere?

But the OpenSpec framework ignores this kind of problem completely. From the moment you start using propose to write spec artifacts until you finish coding, the whole workflow only happens in your own environment. Nobody else looks at the docs you wrote unless you go show them yourself. OpenSpec also makes it really hard to plug in quality gates or review mechanisms for control.

Sure, even the best frameworks have some flaws. But the problems with SDD frameworks like OpenSpec make it much harder for them to succeed in enterprise development teams.

What's My Solution

So how did I solve this whole mess?

At first, I thought like most programmers in the AI era. Build a new wheel myself, and let AI write a brand new workflow under my guidance.

But I quickly realized this wouldn't work. A new development process that hasn't been tested by a lot of development teams just turns into another AI toy. It doesn't have the reliability you need for an enterprise development environment.

So I remembered a piece of news from six months ago about Amazon AWS's AI coding workflow.

Amazon seemed to run into the same problem we did. Their unreliable AI development process caused disasters in production. So they required that any code written by junior or mid-level engineers get approved by another role on the team. Hmm, this matches exactly what we ran into.

I figured that at the time, this response was probably just a temporary fix. Six months had passed, so they must have come up with a more systematic, more engineered solution by now.

After some digging, I found the solution they came up with: aidlc-workflows.

What is AIDLC
AIDLC's dynamic workflow and dynamic team collaboration
How I put AIDLC into practice with my team
My custom tweaks to AIDLC
Best practices for software lifecycle management using AIDLC

Please head over to my original article From OpenSpec to AIDLC: How I Improved My Team's AI Code Quality to read the full content.

DEV Community