AI has become very good at writing code.
On a clear and bounded task, it can produce code that is cleaner, faster, and more consistent than what many of us would write by hand. It does not get tired. It does not forget a closing bracket after a long day. It can follow a known pattern across many files in seconds.
AI can still produce ordinary coding mistakes, so compilers, tests, and code review are not going away. But those mistakes are no longer the most interesting part of the problem for me.
The harder failure often begins before the first line is generated.
We open a powerful AI agent and give it a prompt:
Add an archive feature.
That may be all it knows.
It does not automatically know what "archive" means in our business. It does not know which service owns the data, which users have permission, what the mobile app expects, why an old database rule exists, or which background jobs can still change the record.
Then the AI writes clean code for the incomplete world we described.
The code can be correct according to the prompt and wrong according to the system.
This is the idea I am trying to name. In the old workflow, we often found the bug in a condition, query, API call, or state change. In an AI-agent workflow, the bug may begin in the information we failed to provide.
I now debug the room around the agent too.
Did the agent know how this project works? Did it see the rules? Could it find the right documentation? Did it have the proper tools? Did it know what "done" meant? Could I inspect what it did?
Sometimes the failure is not in src/. It is in the prompt, instructions, skills, tools, permissions, retrieval, or feedback loop we built around the agent.
Imagine a skilled worker entering a silent factory
Picture a skilled worker arriving at a factory on Monday morning.
Nobody gives them a map. The rooms are not labeled. The safety rules are hidden in an old binder. Some tools are missing. Their keycard opens every door, including rooms they should never enter. The work order says only, "Fix the machine." There is no inspection checklist.
The worker is capable. The workplace is not ready for them.
If they repair the wrong machine, use the wrong part, or stop before the repair is safe, blaming only the worker misses half the problem.
This is how many teams use AI agents. They choose a powerful model, point it at a large repository, write a short request, and expect the agent to understand years of decisions that nobody gave it.
A coding agent is not only a model. It is a system made of a model, instructions, skills, tools, permissions, memory, repository context, and tests. Every part can help the agent succeed. Every part can also fail.
Anthropic calls this wider job "context engineering." The context can include system instructions, tools, external data, message history, and information retrieved while the agent works. Anthropic also warns that context is limited. Giving a model more text does not guarantee that it will use the right text well.[1]
The model matters. The room matters too.
One vague ticket, five different failures
Suppose I tell an agent:
Add an archive feature for customer projects.
The agent adds an archived field, hides archived projects from the main page, and writes a test. The code compiles. The test passes. The agent reports that the feature is complete.
Then I discover the missing pieces:
- Our mobile app still shows archived projects.
- An old background job can still modify them.
- The project uses soft deletion rules that the agent never saw.
- Only administrators should archive projects, but the API accepts any signed-in user.
- The database change has no rollback plan.
Was the code buggy? Parts of it may be.
But the first failure happened earlier. The agent never received the full meaning of "archive." It did not know the system boundary, the security rule, or the migration process. Its test proved only the small behavior it had invented.
Current research gives us a useful warning here. In SWE-Bench Pro, agents performed far better when task descriptions included human-added requirements and interface details. GPT-5 High resolved 25.9% of those tasks, but only 8.4% when those details were removed. Claude Opus 4.1 fell from 22.7% to 8.2%.[8]
Those numbers are not universal production bug rates. They come from one benchmark, and the benchmark has limitations. But the direction is hard to ignore: what the system tells the agent can change the result dramatically.
Agents need onboarding, not one giant prompt
Human developers do not learn a mature project by reading one ticket. They learn its language, boundaries, commands, habits, and history. They ask why a strange abstraction exists. They discover which rules are written down and which ones live in a senior developer's head.
Agents need a practical version of that onboarding.
OpenAI's Codex reads layered AGENTS.md files before it begins work. Teams can place general guidance at the repository root and more specific instructions inside subdirectories.[2] The open AGENTS.md format describes the file as "a README for agents," with setup commands, tests, conventions, and other project knowledge.[3]
Claude Code uses CLAUDE.md for a similar purpose. Its documentation contains an important warning: Claude treats these files as context, not as enforced configuration. It also recommends concise instructions because long or contradictory files reduce reliable adherence.[4]
That difference matters.
An instruction can say, "Never deploy without approval." A hard control prevents the deployment command from running without approval. The first guides behavior. The second enforces a boundary.
Good agent architecture knows when a written rule is enough and when the system needs a lock on the door.
The answer is not to paste the whole company into the context window
When teams notice that an agent lacks context, the first reaction is often to give it everything.
Every source file. Every design document. Every old discussion. Every log. Every policy.
That creates a different problem. Important details get buried under irrelevant details. Old instructions conflict with new ones. The agent spends time reading instead of working.
A better design gives the agent a small map and clear paths to deeper knowledge.
Aider's repository map is a useful example. It gives the model a compact view of important files, classes, functions, types, and call signatures. It selects what fits within a token budget instead of dumping the entire repository into the prompt.[7]
Skills provide another layer. Claude Code skills can package reusable procedures, scripts, templates, and reference material. The short skill descriptions remain available for discovery, while the full instructions load only when the skill is needed.[5]
MCP provides connections to external systems such as files, databases, APIs, and tools.[6] That matters because the repository is rarely the whole truth. The requirement may be in an issue tracker. The failure may be in monitoring. The approved design may be in a document. The current schema may be in a live database.
A fact existing somewhere in the company does not mean the agent knows it. The system needs to provide a safe, reliable path.
The prompt is a request, not the whole system
This is where I think teams misunderstand prompting.
A prompt tells the agent what we want right now. It should not be expected to carry the entire history and design of the product.
When I tell an experienced developer, "Add an archive feature," that short sentence works only because the developer already shares a large amount of context with the team. They know the product, the users, the architecture, the release process, and who to ask when something is unclear.
The same sentence given to a fresh agent is not the same assignment.
The agent may understand every word and still lack the knowledge behind those words. If it builds exactly what the prompt appears to request, clean code does not save us from the missing context.
That is why I think of this as an information bug or a context bug. The prompt reaches the model, but the meaning needed to implement it does not.
The solution is not a giant, perfect prompt. The solution is an agent-ready system: stable project instructions, discoverable skills, current documentation, useful tools, safe access, independent tests, and a way to ask for help.
My BRIEF check before I blame the agent
I now think about agent setup with five questions. Together they form a BRIEF.
Bearings: does it know where it is?
The agent needs a small map of the repository and the system.
Which service owns the data? Where do validations belong? Which terms have special meanings? Which old decisions must remain in place?
Architecture Decision Records are useful because they preserve why a meaningful decision was made, not only what the code looks like today.[13] An agent that sees only the current code may "clean up" something that exists for a reason.
Useful bearings include a concise AGENTS.md, a system diagram, a glossary, a repository map, and links to important decisions.
Rules: does it know how work is done here?
The agent needs the house rules.
That may include coding conventions, data-handling policies, migration steps, protected files, required reviews, and conditions that mean "stop and ask a human."
Keep these rules short and specific. If two instructions disagree, fix the instructions instead of hoping the model chooses the right one.
Where a rule must never be broken, enforce it with permissions, hooks, protected branches, policy checks, or approval gates. Do not rely on a paragraph alone.
Implements and identity: does it have the right tools and access?
A mechanic needs the correct wrench. A coding agent may need search, tests, build tools, logs, an issue tracker, or API documentation.
Missing tools force the agent to guess. Too much access creates a larger danger.
NIST defines least privilege as giving a user or process only the minimum access needed to perform its task.[11] The same idea belongs in agent design. Use read-only access by default. Separate development from production. Require approval for destructive actions. Give the agent a task key, not the master key.
Exit criteria: can it prove the work is done?
"Make it work" is not a finish line.
The agent needs checks it can run: tests, builds, type checks, security scans, expected screenshots, acceptance examples, or known outputs. The Scrum Guide's Definition of Done makes the same general point for teams: work needs a shared description of the quality state required for completion.[14]
But tests can be wrong or incomplete too.
SWE-ABS strengthened the tests for 11,041 patches that had already passed SWE-Bench Verified. The stronger suite rejected 2,184 of them, or 19.78%.[9] That does not mean one in five production patches is bad. It shows something narrower and still important: a weak evaluator can make an incorrect patch look successful.
The agent should not be the only author of its assignment, implementation, and proof.
Feedback: can we see what happened?
"Done" is a claim. I want evidence.
What files changed? Which commands ran? Which tools failed? What tests passed? Which assumptions did the agent make? Where did it ask for approval?
OpenTelemetry explains observability through signals such as traces, metrics, and logs.[12] Agent systems need their own version of this. Record tool calls, approvals, test results, errors, and important decisions. When something goes wrong, the team should be able to reconstruct the run instead of calling it a random hallucination.
Good feedback also helps the agent while it works. Anthropic recommends that agents receive ground truth from their environment, such as tool results or code execution, so they can judge progress. It also warns that autonomous agents can compound errors and should be tested with guardrails.[10]
Instructions, skills, and tools are part of the architecture now
We usually think of architecture as services, databases, queues, APIs, and deployment systems.
For agentic software development, that boundary is too small.
The files that instruct the agent are architecture. The skill library is architecture. The repository search method is architecture. Tool descriptions are architecture. Permissions are architecture. The test harness is architecture. The run history is architecture.
These parts do not replace good application design. They decide how the agent sees and changes that design.
This also changes how I diagnose failure.
If an agent edits the wrong package, I still review its reasoning. But I also ask whether it had a repository map.
If it breaks a security rule, I still reject the patch. But I also ask why the rule was hidden and why the environment allowed the action.
If it stops too early, I still hold the output accountable. But I also ask whether "done" was written as an executable check.
If it ignores a skill, I inspect the skill's name, description, trigger, and availability instead of assuming that installing it made it usable.
The point is not to excuse the model. The point is to debug the whole system.
The checklist I use now
Before I send an agent into a serious project, I ask:
- Where is the map? Can it find the relevant part of the system and understand the important boundaries?
- Where are the rules? Are they concise, current, and free of contradictions?
- Which skills and tools does it need? Can it discover and use them without receiving unnecessary power?
- What proves completion? Are the acceptance checks independent enough to catch a plausible but wrong result?
- What record remains? Can a human review the actions, evidence, assumptions, and approvals afterward?
A better model may improve the worker. It does not label the factory, write the safety policy, choose the keycard permissions, or define the inspection process for us.
Those are engineering responsibilities.
The new debugging question
Code bugs are still here. AI did not retire the compiler, the test suite, code review, security review, or architecture work.
It added another system that can be misconfigured.
So when an AI agent fails, I no longer ask only, "What is wrong with the generated code?"
I also ask:
What kind of workplace did we give the agent?
A talented worker in an empty, unlabeled factory will make avoidable mistakes. A capable agent with missing instructions, weak retrieval, the wrong tools, broad permissions, and no finish line will do the same.
The new bug is not always in the code.
Sometimes, the bug is the room we built around the agent.
Sources
[1] https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents — Anthropic: Effective context engineering for AI agents
[2] https://learn.chatgpt.com/docs/agent-configuration/agents-md — OpenAI: Custom instructions with AGENTS.md
[3] https://agents.md — AGENTS.md: A README for agents
[4] https://code.claude.com/docs/en/memory — Claude Code: How Claude remembers your project
[5] https://code.claude.com/docs/en/skills — Claude Code: Extend Claude with skills
[6] https://modelcontextprotocol.io/docs/getting-started/intro — Model Context Protocol: What is MCP?
[7] https://aider.chat/docs/repomap.html — Aider: Repository map
[8] https://arxiv.org/abs/2509.16941 — SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
[9] https://arxiv.org/abs/2603.00520 — SWE-ABS: Adversarial Benchmark Strengthening Exposes Inflated Success Rates
[10] https://www.anthropic.com/engineering/building-effective-agents — Anthropic: Building effective agents
[11] https://csrc.nist.gov/glossary/term/least_privilege — NIST: Least privilege
[12] https://opentelemetry.io/docs/concepts/observability-primer — OpenTelemetry: Observability primer
[13] https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions — Documenting Architecture Decisions
[14] https://scrumguides.org/scrum-guide.html — The Scrum Guide
Originally published at https://blog.jenuel.dev/blog/the-new-bug-isnt-always-in-the-code
Thanks for reading! If you enjoyed this article and like this kind of content, you're always welcome to buy me a little coffee, but only if you'd like to. No pressure at all, and either way I'm truly grateful you stopped by. ☕️
Top comments (0)