The decision to rewrite
For years, rewriting an existing codebase from scratch was considered one of the riskiest decisions an engineering team could make. Is AI changing that calculus, or just making it easier to underestimate the risks?
Both. In the hands of an excellent engineering team, one that knows what they want, knows what "great" looks like, and knows how to build guardrails for AI, the calculus is changing. But a team that's weaker in engineering excellence, or is under incredible pressure from outside to hit unrealistic deadlines, is much more likely to run into complications they didn't anticipate.
For teams that have the right approach, and are measuring the outcomes, it's still a business decision because it's not free: it takes engineering time and focus, and, of course, spend on tokens.
In all cases, though, the "second-system effect" still holds. AI hasn't removed the risks of tending toward more complexity, or giving in to the lure of feature creep.
What are you seeing that makes teams decide to rebuild software rather than incrementally improve what they already have? Is it technical debt, the opportunity to change architectures, the economics of AI-generated code, or something else?
I've talked to a number of leaders whose senior folks have tossed out a "let's rewrite in Rust" type proposal, but I have yet to hear of someone actually doing it with a major production system. From a business perspective, unless you have crippling performance or security problems, this sort of rework can be hard to justify.
The drive to knock down technical debt usually comes from security risks more than anything. That's where we're seeing a lot of pressure to rework legacy systems, as the threat of AI-driven attacks is causing teams to analyze their exposure. For instance: The PHP2Shell Vulnerability Heralds Cybersecurity Hell.
The one category I do hear about is re-architecting to take better advantage of agentic development: changing module boundaries, breaking monoliths into smaller pieces with clear boundaries. This lets your agents work inside the modules separately without conflicts or dependencies.
Can you walk us through a specific rewrite you've seen or been involved in? What was the original system, what did the team hope to accomplish, and how did the actual process compare with their expectations?
I’ll use one of ours as an example. It’s small, but AI was involved at every step. We did a rewrite, but the rewrite missed the problem we wrote it for.
We run several open source static analysis tools on the repos we analyze, and a background task loads the results into our database. That task read each results file whole, so on very large repos it hit memory spikes and timed out. We had workarounds, but we wanted to fix the problem. We had tried to fix this twice before, without success.
Early this year, we tried again. The plan: the scanner writes one record per line, and the reader streams them, falling back to the old format for any job that predated the change. One engineer built it with Claude Code, which co-wrote the plan and every commit of the new reader. Copilot’s code review caught two real bugs before merge, and all the tests passed.
The new format went live, and the same day production showed the task still needed four or five retries on a large repo. Streaming had fixed the memory spikes, but they weren’t what made the task time out. The clue was that each retry made progress, which doesn’t happen when a task is running out of memory. The real cost was one database insert per row. (We shipped batched inserts the next day.)
The validation we’d built to verify the change could not have caught that, because the plan’s large-file check was about memory. The tests checked behavior, but didn’t time the analysis of a large repo end to end.
AI made the build fast and the review thorough. But it wasn’t until we got the code into production that we realized we’d fixed the wrong thing.
What breaks
What's the first thing teams tend to discover they've underestimated once a rewrite is underway? Are there particular problems that consistently show up after the initial excitement of generating a new codebase?
What breaks most are business assumptions. A few weeks back, I was talking with a leader whose company had decided to do a wholesale rebuild of their SaaS product. What surfaced were years of special cases they had built for customers to get deals done or to secure a renewal.
These special cases in the code are often caught during the rewrite process—if you're lucky, they're captured as tests. But often these aren't in the core of the product. They're in integrations and custom adapters built by your professional services team or the customer.
Legacy software often contains years of undocumented decisions, workarounds, and edge cases. What happens to that accumulated knowledge when AI is asked to recreate the system? How do teams identify what they can't afford to lose?
Yeah, that's exactly what I mean! Sometimes they're in the code of the system you're rewriting, and sometimes they sit outside.
I think this is where teams should be spending the most time. In the vein of what I said above, they need to look at the history of product decisions and exceptions. They'd be well-served to spend more time gathering all the special cases their system was built to handle.
There's a flip-side to this question, as well: what can they afford to lose? With the second system effect in mind, teams need to make hard decisions about what the rewrite won't do, to avoid feature bloat and a new but still complicated system.
Where have you seen AI-generated rewrites fail in ways that weren't immediately obvious? I'm particularly interested in cases where the new software appeared to work but problems emerged in production, at scale, or during integration.
I'll point back to the example from above. It's pretty easy to rework a module or service to have the same interface as before, but it's harder to capture all of the ways it's being used.
"Agentic exuberance" is another failure mode we've seen, both in-house and anecdotally in my conversations with other engineering leaders. Even when instructed otherwise, agents tend toward over-building, which can lead to a more complex system than you’d initially planned.
Are there particular categories of software or architectural patterns that are surprisingly good candidates for AI-assisted rewrites, and others where the approach is still a bad idea? What distinguishes them?
Any module or service that has a very clean abstraction is an excellent candidate. And ideally, one where you can keep the existing module or service in place while testing out the new one. This way you can validate the outputs of the new version based on real workloads. (Conversely, any system that only sees very infrequent use is a bad candidate, because you won't get enough new usage to do validation.)
In terms of other bad candidates: the worst are flexible and extensible systems and APIs. Hyrum's Law is real. That is, despite what you've put in the contract of your interface, someone will come to use it in an unexpected manner. Changing that through a rewrite will likely break your system for some number of users.
The engineering reality
How does the engineering team's job change during a wholesale AI-assisted rewrite? Where are developers spending less time, and where are they taking on responsibilities they didn't previously have?
The obvious time-savers are writing the code and writing the tests. The team needs to be spending most of their time on the planning process (making sure the plan of attack lines up with their expectations), on the guardrails (making sure the work doesn't break the assumptions in the middle of development), and on the validating (making sure the final result does what you expect).
If you haven't done this before, or if you're tackling a major rewrite, it's critical to break the work up into pieces and have clear checkpoints where your team is checking the output against the plan and acceptance criteria. These checkpoints are a good moment to change the plan or the guardrails. You don't want to set the machinery in motion, have it work for a few days or weeks, and then start validating the output. You wouldn't one-shot a rebuild of your entire product, so don't treat a rework like that.
How do teams establish confidence that the rewritten system actually does everything the original did? Are traditional tests and code reviews sufficient, or does this require a different approach to validation?
No, tests and code reviews aren't enough. Yes, you need a full suite of unit tests, but for a reworked system, you also need integration testing. Code reviews aren’t going to be enough because so much of the system is changing. As I mentioned before, you want to set up clear checkpoints so your team is validating the output of the rework at those stages before work continues.
I recently heard a talk where the speaker drew an analogy: they said that with agentic software development we’re not heading toward a factory making widgets, but more of a shipyard. There will be plenty of automation, but you’ll still validate each component. If there’s a major problem with a key component, you don’t want to figure that out after the entire ship is assembled (that might be catastrophic). You want to validate each component as it’s produced.
For software, that means reviewing the initial plan and ensuring that you’re validating at the component level, not the PR level.
Does AI meaningfully change the economics of maintaining the resulting software, or does it primarily reduce the cost of creating it? Have you seen teams successfully measure whether the rewrite delivered lasting improvements?
There are reductions in both creation and maintenance costs, but you need to be careful. We are seeing inefficient patterns emerge: extensive rework happening in one agent session, rework of the same code over time by one developer with their agents, and multiple developers/agents reworking each other’s code. This churn is incredibly inefficient and costly over time.
In terms of lasting improvements, we’re too early as an industry to make that sort of claim. Teams are using platforms to measure outcomes as they adopt agentic development, but it’s still early days. We are certainly seeing initial benefits in speed of response and resolution for bugs, but time will tell how lasting these improvements are.
What comes next
Do you think AI-assisted rewrites will become a normal part of software maintenance, rather than the exceptional, high-risk projects they've historically been? What would need to change for that to happen?
I do. As teams lock down their planning and validation processes, they’ll lower the risks and rewrites will be more common. At minimum, rewrites will be a tool in engineering teams’ toolbox for certain classes of problems, rather than an exception.
Chad Fowler has been writing about a related idea he calls the Phoenix Architecture: designing systems so components can be thrown away and regenerated, rather than patched forever. I think it’s a really cool direction, and for certain use cases I can see teams rebuilding code on a regular basis instead of treating a rewrite as a once-a-decade event. It also ties back to my point above. You can only burn it down safely if your specs and validation are solid.
If an engineering leader is considering rebuilding a significant part of their stack with AI tomorrow, what questions should they answer before writing the first prompt? And what would make you tell them not to do it?
Like so many things in our work, it all comes back to “why.” Why are they considering making this change? What are their major drivers, and what outcomes do they expect? I’d recommend they answer those questions first, and make sure that the work is worth the effort. Then, I’d encourage them to answer how they’re going to validate the reworked system at each stage. Finally, I’d recommend they come up with a measurement approach for the outcomes. Even if coding and testing are faster and cheaper than it used to be, they still need to support and maintain this new code.
If they’re rebuilding to solve a clear business problem, and they’ve checked the plan and validation stages to make sure that it’s done properly, I’d say go for it. But if their case rests only on data about the current pain (e.g., time to fix bugs, number of features they couldn’t ship due to current architecture) with no clear, direct, and measurable connection between the rework and the pain, then I’d tell them to go back to the drawing board—er, agent prompt.
Top comments (0)