Most AI projects inside established companies start on software that already carries years of shortcuts: old integrations, inconsistent data, business logic in places nobody remembers. A conventional feature can usually avoid the worst of it.
An AI feature often can’t. A support copilot or an agent that updates records needs data from several systems, APIs into the core application, permission checks, infrastructure to run on, and a way to judge its own output. It crosses many parts of the stack at once and inherits the debt in each one.
Does AI increase technical debt? It can create new debt. The more immediate problem is that it raises the cost of technical debt already inside the system.
Why existing technical debt becomes more expensive
Not all technical debt is a mistake. Martin Fowler’s technical debt quadrant separates prudent, deliberate debt from debt taken on recklessly or by accident, and teams live with both. That gets harder once an AI feature depends on the indebted code. Five mechanisms show up most often.
Tight coupling
If refund rules live in a controller, a database trigger, and a nightly batch job, an assistant can’t answer “can I get a refund?” without duplicating that logic or calling code never meant to be called from outside. Duplicated logic drifts. Calling into shared code means a change for the AI feature can break checkout or reporting.
Weak APIs
When a core entity has no write endpoint, or two endpoints return different fields for the same customer, the AI project becomes an API project.
Thin test coverage
AI output is probabilistic, so you need evaluations to judge it. If the deterministic code around the model has no tests either, you can’t tell whether a bad answer came from the prompt, the retrieval step, the model version, or a legacy formatting function.
Weak observability
A traditional bug usually throws an error. An AI failure often returns a confident wrong answer with a 200 status. Without tracing through retrieval, the model call, and downstream systems, nobody can explain a wrong refund amount.
Permission shortcuts
Many internal systems rely on UI checks instead of real authorization. That’s survivable for employees clicking through screens, not for a model calling those systems for a user. The OWASP Top 10 for LLM Applications lists prompt injection and excessive agency (a model with more permissions than it needs) among its top risks, and both get worse when the systems behind the model have loose access controls.
Data debt becomes AI debt
In a conventional app, analysts know which columns to ignore, and each screen reads only the fields it needs. A retrieval system or agent reads far more broadly, without that tribal knowledge, so quiet data debt becomes visible product behavior.
If a customer exists three times across CRM and billing, retrieval returns all three, and the model picks one with full confidence. Inconsistent schemas add code: when “status” means order status in one system and payment status in another, somebody writes mapping logic that breaks whenever either source changes. Stale documents with missing metadata cause the quietest failures. A wiki with four versions of the travel policy and no reliable owner or update date will answer from whichever version ranks highest. And if no team owns a dataset, the same wrong answer keeps coming back.
This is the distinction teams miss: connecting an AI system to data is not the same as having data that is ready for AI. Connection can take a sprint. Readiness means the data is consistent, current, owned, and governed well enough that a model can read it on behalf of any user. That can take quarters.
Sculley and colleagues at Google made a related point in Hidden Technical Debt in Machine Learning Systems (NeurIPS 2015). The paper predates LLMs, and today’s retrieval pipelines look different. Its core observation still holds: the model is a small part of a larger system of data, configuration, infrastructure, and monitoring, and data dependencies are harder to detect than code dependencies.
AI-generated code can increase debt faster
AI coding tools speed up engineers, and AI-generated code isn’t automatically worse than human code. The problem is speed without matching review and test automation. An assistant sees a slice of the codebase and solves the local problem. It may not know that a utility already exists three directories away, or that the team agreed to stop calling the legacy pricing service. Each change is locally correct. Across hundreds of pull requests, you get duplicated logic and new callers of code the team wanted to retire. A tangled codebase gives it bad patterns to copy.
Google’s 2024 DORA report found that higher AI adoption was associated with better individual productivity, and also estimated that each 25% increase in adoption was associated with a 7.2% drop in delivery stability. That’s correlation, not proof of cause. DORA’s own reading is that AI makes larger changes easier, and large batches have long been linked to less stable delivery.
AI creates new maintenance obligations
Even on a clean foundation, AI adds components that need upkeep.
Model and provider dependencies. Models get deprecated, and providers change terms, so upgrades need testing.
Prompts and evaluation. Prompts change behavior the way code does. Without versions, owners, and an evaluation set with known answers, nobody can safely edit a prompt or swap a model.
Retrieval and data pipelines. Chunking rules, indexing jobs, and vector stores need upkeep. Changing the embedding model can mean embedding the whole corpus again.
Permissions, observability, and cost. Every tool an agent can call needs scoped access and audit logs. Teams also need to know which documents and model version produced each answer, and what each feature costs.
Each is something to version, test, monitor, or own, on top of the debt already there.
A hypothetical example
This scenario is hypothetical, a composite of common patterns.
A parts distributor wants an assistant that answers “where’s my order?” A two-week prototype that reads the orders table demos well.
Then the real requirements appear. Ship dates are calculated in a stored procedure nobody has touched in years. Backorder overrides live in a spreadsheet the warehouse manager updates by hand. Partial shipments sit in a separate service added after an acquisition, with no API.
The team now has to document the stored procedure, move the spreadsheet into an owned database, and build an API for the shipment service. None of that is AI work. All of it existed before the project, and now it’s the critical path.
The lesson: the prototype measured how easy the model call was, not how ready the system was.
What should companies fix before adding AI?
Not everything. Technical debt can’t be reduced to zero, most legacy systems don’t need a rewrite, and some AI features need almost no modernization.
Fix the debt on the AI feature’s critical path, not every piece of debt in the company.
Prioritize:
Security and permissions when the AI reads across systems or acts on a user’s behalf.
Data quality when answers need to be accurate about money, orders, transactions, or commitments.
Critical business logic when important rules are duplicated or hidden inside triggers, background jobs, or spreadsheets.
API reliability when the AI will update data or take actions, instead of only reading information.
Observability when an incorrect AI response could reach a customer, employee, or auditor.
Messy code in modules the AI never touches, old but supported frameworks, and systems scheduled for retirement can usually wait. If several rows apply to one use case, the project is really an application modernization effort with an AI feature at the end. Plan and fund it that way.
How technical debt management should change
Technical debt management has to move from the codebase to the use case. The question isn’t where the worst code is. It’s what this feature depends on.
- Map the dependencies. List every system, dataset, API, and permission boundary the feature touches.
- Separate critical debt from cosmetic debt. Aim technical debt remediation at items that affect correctness, security, or data access.
- Fix unstable interfaces and permission boundaries first. You don’t need to break up the monolith, only to put a stable, tested interface where the AI connects.
- Add tests and observability before scaling. Characterization tests record what legacy code does today, so you can change it safely.
- Assign owners for every dataset, prompt, and AI service the feature relies on.
- Review AI-generated code with normal engineering standards: same reviewers, same tests, same architecture rules.
Conclusion
AI makes technical debt more visible because it depends on more of the stack at once. An AI feature that reads across your data and acts through your APIs can’t route around old shortcuts, so they become part of its cost and risk.
Companies don’t need to eliminate all technical debt before adopting AI. They need to know which technical debt the AI system will depend on, and fix that part first.
The cheapest time to find out what your AI feature depends on is before the prototype impresses anyone.
Top comments (0)