Context engineering for AI coding agents is the practice of deciding what goes into the model's context window, when it goes in, and what stays out. For coding agents that means setting a token budget for project memory, splitting what you push at session start from what the model pulls on demand, and ranking memories so the best ones fit. It is a different job from prompt wording. The prompt is a sentence. The context is everything the model reads before it answers.
Key takeaways: how to do it in three steps
Set a token budget. Pick a hard cap for the start-up briefing. At Open Agentic Brain & Memory (OpenAB), we use 2,000 tokens.
Split push from pull. Push the few things every session needs. Let the model fetch the rest through a recall tool.
Rank what makes the cut. Combine pins, recency and relevance, file memories by project, and measure the result on your own history.
Why more remembered context often makes an agent worse
The natural instinct is to keep everything. Every decision, every fix, every preference, pasted into the start of each chat. It feels safe. It usually backfires.
Models do not read a long context evenly. The "Lost in the Middle" paper (Liu et al., 2023) found that accuracy drops when the relevant fact sits in the middle of a long input, even for models built for long contexts. Practitioners call the broader effect context rot: the more you add, the harder it is for the model to find the line that matters. A large context window raises the ceiling on what fits. It does not make the model attend to everything equally.
Cost is the other half. Every token you inject is a token you pay for, and a token that counts toward your Claude, Cursor or Codex limits. We measured this on a store of 10,000 memories. Loading it as full context takes 1,131,367 tokens. Loading it as a briefing takes 1,325. Those are numbers from our own benchmark runs, and the section on measuring below explains how to reproduce the idea on your own data.
The LoCoMo long-conversation memory benchmark (Maharana et al., 2024) tells a similar story. With memory, we keep 97% of the full-conversation score while reading 18% of the tokens. Reading less and answering nearly as well is the whole point of the exercise. Bigger windows do not remove the need for this work, and the cost difference alone is a reason to care.
Context engineering for AI coding agents: briefing versus recall
Two mechanisms do different jobs, and mixing them up is the most common mistake.
The briefing: what you push at start-up
A briefing is a small block of context injected when a session begins, through a start-up hook. It should hold what almost every session on this project needs: the stack, the standing decisions, the rules you keep repeating. "We dropped Redis for SQLite." "Always use pnpm." "Tests live next to the source file."
Because it loads every time, it must stay small and cheap. Ours is capped at 2,000 tokens and is ready in 84 ms (p50). If a fact would only matter in one session out of twenty, it does not belong here.
Recall: what the model asks for
Everything else stays in the store and the agent pulls it when it needs it. Over MCP, our tools are brain_recall, brain_remember, brain_context and a load-my-brain call. Recall across 10,000 memories takes 28 ms (p50), so a lookup in the middle of a task costs almost nothing in time. It costs tokens only when the model actually asks.
The rule of thumb: push what you would regret the agent not knowing, pull what it can look up. When in doubt, pull. A missed recall is a cheap second query. A bloated briefing taxes every session.
Filing memories by project, found by walking up to the nearest .git
A briefing about the wrong project is worse than none. If your Rails monolith's conventions leak into a small Go service, the agent will confidently follow them.
So each memory is tagged with a project. We find the project by walking up from the current folder until we hit the nearest .git. Open a folder anywhere inside a repo and you get that repo's history first. There is no setup step, and monorepo subfolders resolve to the repo root.
Project tags do not wall memories off from each other. A short map of up to 20 projects lives in the global CLAUDE.md, AGENTS.md and GEMINI.md, so the agent knows what else exists. Recall can search one project or all of them. A preference like "I prefer small commits" is true everywhere, and it should be findable everywhere.
Pins, recency and relevance scores: choosing what makes the cut
Once memories are filed, you still have more than 2,000 tokens' worth. Something has to lose. Three signals decide it.
Pins. You mark a memory as must-include, and it goes in first. Use pins sparingly, for rules that would hurt if missed.
Recency. A decision from last week usually beats one from last year. Recent work is also the likeliest thing you will continue.
Relevance and outcome scores. Hybrid search combines keywords and meaning, so "database" can find the note about SQLite. Outcome scores track whether a memory has actually helped, so useful ones rise.
None of these is enough alone. Recency by itself buries a standing rule you set months ago. Relevance by itself needs a query, and at session start there is not one yet. That is why pins and recency carry more weight in a briefing, while relevance does most of the work in on-demand recall.
You stay in control throughout. A local control panel lets you read, edit, pin or delete every memory, so a wrong or stale one is something you can fix in a minute instead of working around.
Measuring it: 84 ms briefings and 28 ms recalls
Context engineering is easy to do by feel and hard to do well that way, so measure. Three things are worth checking.
Latency. Time the start-up hook and a recall call on your own machine. Our figures are 84 ms (p50) for the briefing and 28 ms (p50) for recall over 10,000 memories. Your hardware and store size will move them, so run it yourself and look at the median, not a single lucky run.
Token cost. Count the tokens in your briefing and compare it with the same memories loaded in full. The 1,325 versus 1,131,367 comparison came from this kind of count.
Recall quality. Run
brain eval, which scores recall against your own history. A public benchmark like LoCoMo shows the idea works in general. Your history shows whether it works for you.
If your briefing is full of things the agent never used, trim it. If the agent keeps asking for the same fact, pin it or move it into the briefing. It is a loop you will run a few times, then mostly leave alone.
If you want to try this on your own projects, OpenAB is free, open source (Apache-2.0) and self-hosted. Memory sits in a local SQLite database, encrypted with age, and the key stays in your OS Keychain. There is no account and no server of ours.
Frequently asked questions
What is context engineering?
Context engineering is choosing what information an AI model reads before it responds, and managing that against a token budget. Prompt engineering is about the wording of an instruction. Context engineering covers the instructions, memory, retrieved documents and tool output around it.
How many tokens should a memory briefing use?
There is no universal number, but small is better. We cap ours at 2,000 tokens, which is enough for a project's standing decisions and rules without crowding the window. Start there, then check with your own usage whether the agent needs more or never touches half of it.
Is it better to inject memory at start-up or retrieve it on demand?
Use both, for different things. Inject the small set of facts nearly every session needs, and retrieve the rest on demand so you only pay tokens when the model asks. If you can only pick one, on-demand recall wastes fewer tokens, but a short briefing saves the agent from not knowing what to ask for.
Does a bigger context window remove the need for memory?
Not in our experience. A larger window lets you fit more, but research such as "Lost in the Middle" shows models can miss relevant facts buried in long inputs, and you still pay for every token. Memory picks the right few thousand tokens instead of sending a million.
What is the difference between pushing and pulling context?
Pushing means the system injects context before the model asks, as a start-up briefing does. Pulling means the model requests context itself through a tool call, as with brain_recall over MCP. Pushing is predictable and always costs tokens. Pulling is cheaper when the information is rarely needed.
Sources: Liu et al., "Lost in the Middle: How Language Models Use Long Contexts" (2023); Maharana et al., "Evaluating Very Long-Term Conversational Memory of LLM Agents" (LoCoMo, 2024). Token and latency figures are from OpenAB's own benchmark runs.
Top comments (0)