DEV Community

Cover image for Are Coding Agents Getting Worse, or Have They Lost Their Teachers?
Mehdi Mohseni
Mehdi Mohseni

Posted on Originally published at dotnet-talk.com

Are Coding Agents Getting Worse, or Have They Lost Their Teachers?

When coding agents like Claude Code, Codex and Copilot's agent mode became part of the daily work of many teams, the first reactions were very positive, and a lot of developers were surprised how well an agent could add a feature to an existing project. Now, one or two years later, a different complaint is very common in discussions between developers: the code that the agents write seems to be getting worse. It has more duplication, the structure doesn't follow the architecture of the project, and fixes are added next to old code instead of improving it. The most popular explanation is that the models themselves are getting worse, because the vendors save money on compute, because of new safety restrictions, or because a newer model is simply less careful than the one before.

In my view the model is not the main reason in most of these cases, and the real reason is closer to us. When teams started to use agents, almost all of the code in their repositories had been written by humans and had passed through quality gates like pull requests, peer review and static analysis tools such as SonarQube. The agent was working inside a codebase of reviewed code, and that code was the example it learned from while it worked. Over time, more and more of the codebase has been written by the agent itself, and in many teams with fewer of those gates, so the agent is now learning more and more from its own unreviewed output. In this article I want to look at what the research actually says about this, both the part that supports this view and the part that doesn't, and then at what a team can do about it.

First, is the code really getting worse?

Before looking for a cause it's important to check if the effect is real, because developers are not always good at judging this from their own experience. The METR study from July 2025 is a good warning here: experienced open source developers expected AI tools to make them 24% faster, the measured result was that they took 19% longer, and after the study they still believed the tools had made them about 20% faster [14]. Our feeling about AI and productivity, and probably also about AI and quality, is not a reliable measurement.

There are, however, several measurements that point in the same direction. GitClear analysed about 153 million changed lines of code from 2020 to 2023 and found that the share of "moved" lines, which is their signal for refactoring, dropped from 25.0% in 2020 to 16.9% in 2023, while copy/pasted lines grew from 8.3% to 10.5% and churn, code that is changed again shortly after it was written, grew from 3.3% to 5.5% [1]. Their follow-up report with 211 million lines found that refactored lines fell to less than 10% in 2024 and that copy/pasted code was higher than moved code for the first time in their data [2]. It's important to say that GitClear itself only shows a correlation with the adoption of AI assistants, not a cause, but the direction is very clear.

Google's DORA research gives a similar picture from a different angle. The 2024 report estimated that for every 25% increase in AI adoption, delivery stability decreased by 7.2% and throughput by 1.5%, even though code quality as reported by the respondents went up by 3.4% [3]. In the 2025 report the relationship with throughput became positive, but AI adoption still had a negative relationship with delivery stability, and the authors summarised their finding in one sentence that is very relevant for this article: "AI doesn't fix a team; it amplifies what's already there" [4].

Reports that compare AI-written and human-written changes directly point the same way. CodeRabbit analysed 470 open source pull requests and found 10.83 issues per AI co-authored pull request against 6.45 per human one, which is about 1.7 times more, with readability issues more than three times as common [5]. Sonar evaluated leading models on more than 4,400 Java tasks and found that more than 90% of the issues were code smells, the kind of problem that doesn't break a test today but makes the code more difficult to change tomorrow [6]. So yes, there is a real problem, and it is mostly a problem of maintainability and structure, not of code that doesn't compile.

Is it the model?

Sometimes, yes, and it would be dishonest to write this article without saying that. In September 2025 Anthropic published a postmortem about three infrastructure bugs that degraded Claude's responses in August and early September of that year. One of them was a routing error that at its peak affected 16% of Sonnet 4 requests, and about 30% of Claude Code users had at least one message sent to the wrong server configuration [7]. Users who said that Claude had become worse in those weeks were right, although the reason was not the model weights but the serving infrastructure. Anthropic also wrote that they "never reduce model quality due to demand, time of day, or server load", and that their own evaluations "simply didn't capture the degradation users were reporting" [7].

The other well-known example is the 2023 paper by Chen, Zaharia and Zou, which reported that the share of directly executable code generated by GPT-4 dropped from 52.0% in March to 10.0% in June [8]. This result was shared very widely as proof that the models were getting worse, but the authors themselves explained that the June model had started to wrap its code in markdown code fences, which made it "not executable" for their test, and Narayanan and Kapoor pointed out that the study didn't check if the code was correct at all [9]. What changed was the behaviour and the format, not the ability to write code.

Both examples are useful, because they show what a model or platform problem usually looks like: it starts at a specific moment, it affects many users at the same time, and when it's found it can be fixed or explained. The complaint about agent code quality is different. It is a slow decline, it's described by teams working in their own codebases over months, and it continues across several model releases that are better than their predecessors on every public benchmark. That pattern is very difficult to explain with the model alone, so it makes sense to look at the other input the agent uses: the code it's working in.

The codebase is the agent's example

An agent doesn't write code in a vacuum. Before it changes anything it reads files around the place where it works, it searches for similar code, and it follows what it finds. Anthropic's own best practices for Claude Code recommend exactly this as a technique: "Point Claude to patterns in your codebase", with an example where the developer tells the agent which existing widget is a good example to follow [13]. The same document also says that CLAUDE.md should not contain "anything Claude can figure out by reading code", which assumes that the agent learns the conventions of a project from the code itself [13]. That is a strength when the code around it is good, but the same mechanism works in the other direction too.

Research shows quite clearly that the code in the context has a strong effect on what a model writes. Dinh and his colleagues tested code models with partial code that contained small potential bugs, for example a += that was changed to -=. With clean partial code, CodeGen-2B reached a pass rate of 54.9%, but with the buggy partial code it dropped to 3.1%, which was even worse than the 9.3% it reached with no partial code at all [10]. The authors' explanation is that the model "fails to react" to the problem in most of the failed cases, so it doesn't copy the bug on purpose, it continues the code it was given as if it was correct. These were small models from 2023, but the direction of the effect is what matters for this article.

For security the effect is similar. Snyk demonstrated in 2024 that GitHub Copilot suggested a parameterised SQL query in a clean project, but suggested string concatenation, which means SQL injection, after vulnerable code was placed in a neighbouring open file that Copilot uses as context [11]. That was a demonstration and not a measured study, but a peer-reviewed study by Asare, Nagappan and Asokan found that when Copilot was given the situations where human developers had introduced real vulnerabilities, it reproduced the vulnerable code about 33% of the time [12]. Their main conclusion was actually positive for Copilot, which was "not as bad as human developers", but it also confirms that the surrounding code shapes the suggestion.

So the code that is already in the repository works like a very large prompt that we never write explicitly. When that prompt was written by people and reviewed by other people, the agent continued good code. When the prompt is more and more the agent's own earlier output, the agent continues that instead.

The loop: when the agent learns from itself

Now let's examine what happens over time. In the first months of using an agent, almost all of the code it reads was written by humans, and every line had passed review, static analysis and the discussions in pull requests. The agent copies patterns that people agreed on. Every change the agent makes adds a small amount of its own code to the repository, and if that code is reviewed carefully, nothing changes. But in practice many teams review agent code less carefully than human code, because there is much more of it, because it looks clean and confident, and because the tests are green. Every unreviewed change becomes part of the example for the next change, so small deviations from the architecture are not corrected, they are copied and multiplied.

This is not a new idea in software engineering. Already in 1980 Lehman described the law of increasing complexity: "As an evolving program is continually changed, its complexity, reflecting deteriorating structure, increases unless work is done to maintain or reduce it" [15]. In 1997 he and his colleagues added the law of declining quality, which says that the quality of such systems "will appear to be declining unless they are rigorously maintained" [16]. Hunt and Thomas called the same thing software entropy and gave the advice "Don't live with broken windows", because one bad design that nobody fixes makes the next one easier to accept [18]. The agent doesn't change these laws. It only makes the changes much faster and in much larger numbers, so the decline that took a human team years can happen in months.

There is a related effect in model training that is often mentioned in these discussions. Shumailov and his colleagues showed in Nature that models trained again and again on data generated by models suffer "irreversible defects", and the rare parts of the original distribution disappear [17]. I want to be careful here, because that research is about training new weights, while in our case the weights don't change at all, only the code the agent reads changes. So model collapse is an analogy and not an explanation, but the shape is the same: a system that learns from its own output without an external reference slowly loses the quality of the original.

The most direct evidence for this loop at the level of a project is a 2026 preprint by Zhu, Tsantalis and Rigby, who analysed projects generated by LLMs and by a multi-agent system. In the multi-agent projects the total number of lines correlated with architectural smells at ρ=0.94, and the authors write that "neither functional correctness nor detailed prompting mitigates this decay" [19]. To be fair, the same study also found smells that humans produced and the models didn't, and it only tested one multi-agent framework on 90 programming problems, so it is not a final answer. As far as the published research goes, there is no study yet that measures exactly what this article describes: the share of agent-written code in a real repository over time and the quality of the next change. That is a gap in the research, and until somebody fills it, this explanation is a hypothesis that fits the evidence, not a proven fact.

A child who learns from the people around it

The picture I find most helpful is a child who is growing up. A child learns a very large part of its behaviour by looking at the adults around it, how they speak, how they solve problems and how they react when something goes wrong. When a child grows up with parents who show it good examples and correct it when it goes in the wrong direction, it learns those examples. When a child is left alone, it doesn't become less intelligent, but it has nothing to compare its behaviour with, and it develops its own habits, good and bad, without anybody noticing.

An agent in a codebase is in a very similar situation. At the beginning the reviewed human code was the parent: it showed which patterns to use, where each responsibility belongs and how errors are handled in this project. The pull request reviewer and the static analysis tool were the correction. When we stopped reviewing carefully and the agent wrote more and more of the code, we didn't make the agent less capable, but we took away its examples and its corrections, and then we were surprised that it developed its own habits.

What a team can do

If this explanation is right, the good news is that the solution is mostly in our hands and doesn't depend on the next model release. These are the steps that follow from the evidence above.

  1. Keep the same quality gates for agent code as for human code. A pull request written by an agent should be reviewed by a human with the same care, and static analysis such as SonarQube or the .NET analyzers should run on it in the build, not only on the human changes. Anthropic's documentation says it very directly: "If you can't verify it, don't ship it" [13].
  2. Write the architecture down where the agent reads it. Decisions that live only in the heads of senior developers can't be learned from the code, especially when the code has started to drift. A short CLAUDE.md or AGENTS.md file, architecture decision records and explicit rules about where things belong give the agent a reference that doesn't change with every commit.
  3. Point the agent to reference code on purpose. Choose a few modules that were written and reviewed by humans and are good examples, and tell the agent to follow them, as the Claude Code documentation recommends [13]. Protect these modules from casual changes, because they are the parents in the analogy above.
  4. Make the rules deterministic where you can. Instructions in a markdown file are advice, and Anthropic's documentation describes hooks as the deterministic alternative [13]. In .NET, analyzers with build errors, architecture tests and code style rules that are enforced in the build do the same job: the agent can't ignore them, and they don't get tired after the fiftieth pull request.
  5. Review agent code with a fresh context. The Claude Code documentation notes that a new session reviews code better because it "won't be biased toward code it just wrote" [13]. A second agent or a human reviewer who didn't see the conversation is a better check than the same agent reviewing itself.
  6. Measure the trend in your own repository. Duplication, churn and the share of refactoring changes are exactly the signals GitClear used [1][2], and you can measure them in your own repository over time. If you also track which changes were written by an agent, you can check this hypothesis on your own code instead of trusting my opinion or anybody else's.
  7. Plan time for refactoring. Lehman's law says that complexity grows "unless work is done to maintain or reduce it" [15]. Agents are very good at refactoring when they are asked to do it with clear rules, so the same tool that creates the drift can help to remove it, but somebody has to decide that it's needed.

Conclusion

Whether we like it or not, the share of code that agents write will continue to grow, and the question is not whether we use them but what we give them to learn from. The models are not perfect, and sometimes the platform really does get worse for a few weeks, as Anthropic's postmortem shows. But when the quality of agent code declines slowly over months in the same repository, while the models get better on every benchmark, it makes sense to look at the codebase first, because that is the example the agent copies every day. In short, if we want the agent to write code like a careful senior developer, the codebase has to keep looking like the work of careful senior developers, and that is still our job.

References

  1. Harding, W. and Kloster, M. Coding on Copilot: 2023 Data Shows Downward Pressure on Code Quality, GitClear, January 2024. https://www.gitclear.com/coding_on_copilot_data_shows_ais_downward_pressure_on_code_quality
  2. GitClear. AI Copilot Code Quality: 2025 Look Back at 12 Months of Data, 2025. https://www.gitclear.com/ai_assistant_code_quality_2025_research
  3. Harvey, N. and DeBellis, D. Announcing the 2024 DORA report, Google Cloud blog, 23 October 2024. https://cloud.google.com/blog/products/devops-sre/announcing-the-2024-dora-report
  4. Harvey, N. and DeBellis, D. Announcing the 2025 DORA Report: State of AI-Assisted Software Development, Google Cloud blog, 23 September 2025. https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report
  5. Loker, D. AI vs Human Code Generation Report, CodeRabbit, 17 December 2025. https://www.coderabbit.ai/blog/state-of-ai-vs-human-code-generation-report
  6. Sonar. The Coding Personalities of Leading LLMs, August 2025. https://www.sonarsource.com/the-coding-personalities-of-leading-llms/
  7. Anthropic. A postmortem of three recent issues, Anthropic Engineering, 17 September 2025. https://www.anthropic.com/engineering/a-postmortem-of-three-recent-issues
  8. Chen, L., Zaharia, M. and Zou, J. How is ChatGPT's behavior changing over time?, arXiv:2307.09009, 2023. https://arxiv.org/abs/2307.09009
  9. Narayanan, A. and Kapoor, S. Is GPT-4 getting worse over time?, AI Snake Oil, 19 July 2023. https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-time
  10. Dinh, T. et al. Large Language Models of Code Fail at Completing Code with Potential Bugs, NeurIPS 2023. https://arxiv.org/abs/2306.03438
  11. Degges, R. Copilot amplifies insecure codebases by replicating vulnerabilities in your projects, Snyk Labs, 22 February 2024. https://labs.snyk.io/resources/copilot-amplifies-insecure-codebases-by-replicating-vulnerabilities/
  12. Asare, O., Nagappan, M. and Asokan, N. Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?, Empirical Software Engineering, 2023. https://arxiv.org/abs/2204.04741
  13. Anthropic. Best practices for Claude Code, Claude Code documentation, accessed 19 October 2026. https://code.claude.com/docs/en/best-practices
  14. METR. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, 10 July 2025. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
  15. Lehman, M. M. Programs, Life Cycles, and Laws of Software Evolution, Proceedings of the IEEE 68(9), September 1980, pp. 1060-1076.
  16. Lehman, M. M., Ramil, J. F., Wernick, P. D., Perry, D. E. and Turski, W. M. Metrics and Laws of Software Evolution: The Nineties View, Proceedings of the 4th International Software Metrics Symposium, 1997, pp. 20-32.
  17. Shumailov, I. et al. AI models collapse when trained on recursively generated data, Nature 631, 755-759, 2024. https://www.nature.com/articles/s41586-024-07566-y
  18. Hunt, A. and Thomas, D. The Pragmatic Programmer, Addison-Wesley, 1999, section "Software Entropy". Tip list: https://pragprog.com/tips/
  19. Zhu, Tsantalis and Rigby. AI-Generated Smells: An Analysis of Code and Architecture in LLM- and Agent-Driven Development, arXiv:2605.02741, 2026 (preprint). https://arxiv.org/abs/2605.02741

Top comments (0)