Context Engineering Has Eaten Prompt Engineering

Day 3 of 7 — The Agentic Engineer’s Field Notes

In 2023 the hot job title was “prompt engineer.” Companies were hiring them at six-figure salaries to whisper magic incantations at GPT-4. Conferences had tracks on it. People wrote books about it. Some still do.

In 2026, prompt engineering is to agent building what HTML formatting is to web development — necessary, table-stakes, and not where any of the actual leverage lives.

The leverage moved. Andrej Karpathy gave the new discipline its name in mid-2025 with a now-widely-cited tweet: he prefers “context engineering” over “prompt engineering,” because “people associate prompts with short task descriptions you’d give an LLM in your day-to-day use,” whereas in every industrial-strength LLM app the work is “the delicate art and science of filling the context window with just the right information for the next step.”

The phrase stuck because it was already true. Every team running real agents in production had been groping toward this idea independently. By early 2026 it has crystallized into a coherent framework, with named strategies, named failure modes, and a body of practitioner literature you can actually refer to. Today I want to give you that framework.

The promise that didn’t quite deliver

Three years ago the answer to most LLM problems was “wait for the context window to grow.” We had 4K-token windows that felt cramped, then 8K, then 32K, then 128K, then 200K, then a million. Surely, the thinking went, when the model can fit the entire project into its context, all our problems go away.

The windows grew. The problems did not go away.

If you have actually used a million-token context window on a real task, you know what I mean. The model can technically see everything in there, but its attention — the thing that determines what it actually conditions on — is not uniform across the window. Performance degrades on information stuffed in the middle. The model becomes more sycophantic, more easily distracted, more prone to losing the thread on the original instruction. There is now a substantial research literature on “lost in the middle” effects, plus a louder practitioner literature on what teams call context rot: the slow decay of agent quality as the context fills up over the course of a long task.

The 2026 surveys make the impact explicit. 65% of developers now cite missing or rotted context as the leading cause of poor AI code quality — not model capability, not framework choice, not prompt phrasing. Context.

The lesson the field has finally internalized is that context is not free. Every token you put into the model’s input has three costs: dollars, latency, and attention. The first two are obvious. The third is the killer. You can have a 200K-token context window and your agent can still be effectively dumber than it would be at 20K, because what you put in there competes for the model’s finite attention.

Once you accept that, the entire framing of the problem changes. The job is no longer “stuff as much as you can into the prompt.” The job is to curate the context — surgically, dynamically, on every step of the loop — so that the model sees the smallest, most relevant slice that lets it make the next good decision.

That is context engineering.

The four ways your context can rot

Drew Breunig’s now-canonical taxonomy of context degradation gives us shared vocabulary for talking about what goes wrong:

1. Context poisoning — a hallucination or bad output enters the context early, and contaminates every subsequent reasoning step. The model is now confidently building on a false premise.
2. Context distraction — the context is so large that the model’s attention dilutes. Important signal is drowned in irrelevant noise.
3. Context confusion — superfluous information that looks relevant nudges the model toward decisions it wouldn’t have made with a cleaner context.
4. Context clash — different parts of the context contain contradictions. The model picks one. You don’t get to predict which one.

Memorize these. The next time your agent does something inexplicable, walk through this list. One of the four is almost always the cause.

The four-strategy framework

The cleanest framework I’ve seen for actually doing context engineering comes from the LangChain team, building on patterns that had already emerged across multiple production teams. It splits the work into four operations:

1. Write context

Save things outside the context window so they’re not eating attention budget.

This is the scratchpad pattern (the agent writes notes to itself during a task) and the memory pattern (the agent persists facts across sessions). Anthropic’s multi-agent research system writeup explicitly describes the agent saving its plan to memory because, in their wording, “if the context window exceeds 200,000 tokens it will be truncated.” If your agent has nowhere to externalize its working state, the context window itself becomes its only state, and that state evaporates the moment it exceeds limits.

The discipline: anything the agent might need later, but doesn’t need right now, gets written out. The context window is for the next decision, not for archival.

2. Select context

Pull the right things back in when needed — not before.

The 2023 instinct was “when in doubt, dump it in.” Got documentation? Paste it in the system prompt. Got a knowledge base? Embed the whole thing as RAG and concatenate the top 20 chunks. Got the user’s profile? Put it at the top.

The 2026 instinct is the opposite: just-in-time retrieval over upfront stuffing. Start with the smallest possible context. Give the model tools to fetch what it needs, when it needs it. Modern coding agents combine three retrieval mechanisms — grep/file search, knowledge-graph traversal, and re-ranking — and only pull the chunks the model has actively asked for. Even tool-selection itself is now a context-engineering decision: recent papers show that applying RAG to tool descriptions (selecting which tools to expose, not just which docs) improves selection accuracy by roughly 3x.

The mental shift: stop thinking of context as a place to store information. Think of it as a place where information passes through on the way to a decision. Most information should not be stored — it should be retrievable.

3. Compress context

When the context inevitably gets long, summarize aggressively at the seams.

This is the auto-compact pattern. Claude Code, for example, applies summarization once the context window crosses 95% capacity. The model is asked to produce a structured summary that preserves decisions, action items, and open questions — and to drop intermediate exploration. The rolling history is replaced with the summary.

What you are doing is teaching the agent to write its own meeting minutes. The summary is dramatically shorter than the raw history but preserves the load-bearing decisions. The model’s attention is freed to focus on the current task.

The trap: compaction is lossy. If you compact too aggressively, you lose the thread. If you compact too rarely, context rot wins. The right cadence is task-dependent and worth measuring. Recursive and hierarchical summarization (each summary itself gets summarized as the agent runs longer) is becoming the dominant pattern for long-running tasks.

4. Isolate context

Spawn sub-agents whose contexts never touch each other.

This is the most powerful strategy and the most underused. When a task has a clearly bounded sub-problem — “go read these ten files and tell me which ones contain a database connection” — do not run that in your main agent’s context. Spawn a sub-agent. Let it do the dirty work in its own context window, eat the noise, and return a clean one-paragraph summary to the parent.

This is not just a cost optimization. It is a cognitive optimization. Your main agent’s attention is the most precious resource in the system. Polluting it with the raw text of ten files when what it actually needs is one sentence (“the database connection is in db/client.py") is a self-inflicted wound. Sub-agents are how you keep the main loop's context dense and signal-rich.

The pattern generalizes. State schemas (e.g., LangGraph’s typed state objects) isolate context in specific fields, exposing only relevant information per step. Sandboxed code execution isolates token-heavy intermediate objects from the LLM while keeping them accessible to deterministic code. Any time you find yourself thinking “I wish I could let the agent do this without filling up context with the intermediate steps,” the answer is almost always isolation.

The OS analogy that finally makes this click

Karpathy frames LLMs as “a new kind of operating system, where the LLM is like the CPU and its context window is like the RAM, serving as the model’s working memory.” Once you internalize that frame, every context-engineering decision becomes intuitive.

You wouldn’t run a desktop operating system that loaded all of your hard drive into RAM at boot. You’d page in what you need, evict what you don’t, and treat memory as a precious managed resource. That is exactly what context engineering does. Write context = save to disk. Select context = page in. Compress context = swap. Isolate context = process boundaries.

The Letta team have run with this analogy explicitly, designing memory systems where the agent treats its context window as in-context core memory and external storage as recall and archival memory — three tiers, like RAM/cache/disk in a real OS. We’ll get into the specifics on Day 5. For today, the takeaway is that the OS analogy is not a stretch. The discipline is genuinely the same shape: managing a constrained, fast, expensive working surface against a cheaper-but-slower backing store.

The diagnostic question

When an agent misbehaves, ask: what was in the context at the moment it made the bad decision?

This question sounds trivial. In practice, almost no team can answer it without instrumentation, and the instrumentation almost never exists by default. The single most useful thing you can do for your team this week is build a viewer that lets anyone replay a failed agent run and see the exact context that was passed to the model at each step. The bug becomes obvious 80% of the time. The other 20% of the time it’s a model issue, and you can stop blaming yourself.

A debugging culture built around “show me the context” is a leading indicator that a team has matured past prompt engineering and is doing real agentic systems work.

Why this matters more than it looks

There is a moat being built right now, and it is not in models. The frontier model from any of the major labs will likely commoditize over the next two years. The moat is in the systems people are building around the model, and inside those systems, context engineering is the deepest piece of know-how. It is hard to articulate, hard to copy from a tutorial, and only really learnable by getting agents into production and watching them fail.

Teams that are good at it ship agents that hold up over long horizons, scale gracefully when the task gets harder, and survive model upgrades without falling apart. Teams that are bad at it ship demos.

If you want to be useful in this field for the next five years, this is the skill to invest in. Not the latest framework. Not the latest prompt template. This.

Tomorrow: the multiplier you’ve been ignoring

Go back to the loop diagram from yesterday. Pick the box labeled TOOLS.

Most teams treat tools as a thin wrapper around their existing API. Anthropic now publishes explicit guidance to the contrary — “more tools don’t always lead to better outcomes” is the opening principle of their tool-design playbook — and the teams that have internalized that point are quietly building agents that are far more reliable than their competitors using the same model.

Tomorrow I’m going to walk through what good tool design actually looks like in 2026. The schema choices. The error message philosophy. The MCP ecosystem (now powering 78% of enterprise agents). And — because we have to talk about it now — Simon Willison’s lethal trifecta, which has predicted four major production agent vulnerabilities in the first two weeks of 2026 alone.

See you then.

Day 3 of 7. Earlier: [_the mental shift](https://medium.com/@codewithrashid/why-most-agents-never-survive-production-and-the-mental-shift-that-fixes-it-ddffd932e1fb), then [_the anatomy of a working agent](https://medium.com/@codewithrashid/the-anatomy-of-an-agent-that-actually-works-c4dd996ea4b9)_. Tomorrow: tool design — the most undervalued lever in agentic engineering, plus the security framework you can’t ignore._

Sources & further reading:

• Karpathy on context engineering (@karpathy)
• LangChain, Context Engineering for Agents
• Drew Breunig, How Long Contexts Fail
• Letta, Agent Memory architecture
• Anthropic, How we built our multi-agent research system

---

First published at codewithrashid.com.