Context Engineering over Prompt Engineering: Architecting Codebases for AI Agents
TLDR: Agents don't fail because your prompt was badly worded. They fail because your codebase was never designed to be read. The engineers getting compounding returns from agents aren't the best prompt writers. They're the ones whose repos carry the architecture outside their heads: context standards in the repo, strongly typed contracts, and deliberately pruned retrieval. Prompt engineering is the visible 10%. Context engineering is the structural 90%.
The Prompt Ceiling
Somewhere around 2024, the industry's collective mental model of "working with AI" became a prompt box. The skill was asking well. Whole courses, cheat sheets, and prompt libraries flourished on the premise that the delta between a good and bad AI outcome lived in the wording of your request.
For single-shot, single-file tasks, that was roughly true. It also quietly set everyone's expectations, and the expectations are now the problem.
Because in 2026 the work moved. Mid-level and senior engineers aren't asking a chatbot for a function anymore. They're dispatching agents across repositories of hundreds of thousands of lines, agents that read code, edit files, run tests, and open pull requests. And in this regime something counterintuitive happens that every serious team eventually notices: prompt quality stops being the constraint.
You can rewrite your prompt five times. You can role-play the model into a "senior staff engineer persona." Doesn't matter. The agent will still confidently duplicate an internal utility that already existed three directories away, import a dependency your org banned two quarters ago, and couple two modules your architecture has spent three years keeping apart.
Not because the prompt was weak. Because the codebase was opaque.
What an Agent Actually Sees
To understand why, look at what the average large repository exposes to a reader:
- Implicit boundaries. "Don't call the data layer from the HTTP handler" exists as a norm, maybe a lint rule, maybe an ADR nobody reads. It does not exist anywhere a code reader encounters while working in the repo.
- Unwritten dependency policy. Which libraries are approved, which are banned, which need a security review. Usually a Slack thread. Sometimes a wiki page. Never in the path of work.
- Tribal domain models. What a "tenant" means in this system, which fields are PII, which service owns which concept. Spread across PR conversations, onboarding docs from 2019, and the memory of whoever's been here longest.
A new human engineer takes weeks to absorb all this, and that's with onboarding sessions and mentors and the ability to ask someone in the hallway. Then the industry hands an agent a repo checkout and a prompt, and acts surprised when the output violates constraints that were never written down anywhere the agent could read.
Here's the failure as it typically shows up, so common it's almost a template.
An agent is asked to "add a retry mechanism to the notification service." It produces a perfectly reasonable implementation. It also duplicates the exponential-backoff utility that already exists in an internal library, pulls in a third-party HTTP client the org specifically removed after a security audit, and places its new logic in a module that creates a circular dependency with the domain layer.
None of those are coding errors. All of them are context errors. And notice what makes them expensive: the code compiles, the tests pass, and the violations are only visible to someone who already knows the architecture. If that person exists, a senior just spent their verification budget (see the verification trap) catching things a document could have prevented.
Fix 1: Put the Architecture Where the Reader Is
The single highest-leverage change: stop storing architecture outside the repository.
The pattern that's emerged across serious AI-adopting orgs is an in-repo context standard, a known, versioned location that every agent (and every new human) reads first. An AGENTS.md or .context/ directory at the repo root holding the high-level domain model, the module boundaries and their rules, the explicit list of disallowed dependencies, the invariants ("never call X from Y," "all money fields are integers in minor units"), and pointers to the ADRs that matter.
Treated like code, not documentation. Reviewed in PRs. Updated when the architecture changes, in the same commit as the change.
The test for whether this is working is embarrassingly simple. Could a competent engineer who has never met your team make a boundary-respecting change using only what's in the repo? If your honest answer is "no, they'd need to ask someone in Slack," then your agent never had a chance. Neither did your next hire.
The corollary that surprises teams: this investment isn't really for the agent. It pays back twice. Once when the agent's first-draft PR respects the boundaries, and again every time a new human onboards in days instead of weeks. The agent didn't create the need. It just made the cost of not meeting it immediate.
Fix 2: Strongly Typed Contracts
The second fix is about what kind of surface your system presents to a reader.
Context files are necessary but they're advisory. Agents, like humans, follow probability, and prose is soft. Types are hard. The orgs getting the best agent behavior are the ones that make architectural constraints mechanically enforceable:
- Explicit interface definitions between modules: generated clients, protobuf schemas, OpenAPI specs for anything crossing a service boundary. An agent that hallucinates a function signature doesn't produce a subtly wrong call that compiles. It produces a type error, immediately.
- Strict types end to end. No
Anyescape hatches at the boundaries, no stringly-typed config. The failure mode shifts from "plausible code that violates the design" to "code that doesn't run," and that shift is the entire ballgame. An error the compiler catches at generation time costs seconds. The same error caught in review costs a senior's attention. Caught in production, it costs six hours or six figures. - Machine-checked policy where you can get it: lint rules and dependency analyzers that encode "this module may not import that one" as something the build fails on, not something the reviewer remembers.
The framing worth internalizing: every invariant you express only in prose is an invariant an agent will eventually violate, charmingly and with total confidence. Every invariant you express as a type or a lint rule is one less thing for the human reviewer to carry.
Fix 3: Prune the Context
The third fix is the counterintuitive one, because everything above has been "give the agent more context." But there's a failure mode on the other side, and mature teams hit it right around the time their context standards start working.
Context dilution. If your instinct is "the agent should know everything," you dump half the repository into the window, and the model now treats your critical domain rules as one equal voice in a choir of 40,000 tokens of boilerplate. The invariant you care about gets the same statistical weight as a vendored dependency's CHANGELOG. The agent saw the rules. It just didn't prioritize them, because you gave it no structure to prioritize with.
The working pattern is deliberate, selective context:
- AST-based indexing and retrieval, tools that navigate the code's structure (definitions, call graphs, usages) instead of streaming its text. The agent gets "here are the three functions relevant to this task and their signatures," not "here's 200KB of the module they live in."
- The in-repo standard provides the constants: domain model, boundaries, banned deps. The retrieval layer provides the variables: exactly the code relevant to this task, nothing else.
- Budget context like a designed API surface, not an all-you-can-eat buffer. Teams that do report the same thing: smaller, sharper contexts outperform complete ones, on cost and quality, almost every time.
The principle underneath all three fixes is one the prompt-engineering era obscured: an AI agent is the most literal-minded new hire your team has ever onboarded. It will not infer your conventions. It will not ask a clarifying question in the hallway. It will follow exactly what's in front of it, at full speed, with total confidence.
Every year your team spent frustrated at new engineers for "not getting it," when the real problem was undocumented architecture and oral-tradition onboarding, is now doubled. Because the agent executes the misunderstanding immediately, at scale, in code.
The Question Behind the Question
If your codebase needs a person to already know your architecture in order to work safely in it, is that an asset or a liability? For twenty years, "the architecture lives in our senior staff's heads" sounded like job security. In an agent-heavy 2026, it sounds like a single point of failure with a Slack handle.
Your Turn
I'd genuinely like to hear how other teams are drawing this line, because the tradeoffs are real and I don't think there's a settled answer.
- If a new human developer takes three weeks to understand your architecture, why do you expect an AI agent to infer it from raw files?
- Is your codebase structured to guide autonomous agents, or is your architecture stored purely in your senior engineers' heads? And if it's the second one: does that feel like security, or a countdown?
Comment with the ugliest thing an agent did in your repo that turned out to be your architecture's fault. Those stories are more useful than the prompts everyone keeps posting.
Next in the series: Deterministic Safety Gates for Agentic Workflows. What happens when agents stop writing code and start changing infrastructure, and why "the agent tested itself" isn't a safety argument.
Top comments (0)