Context engineering: the job prompt engineering grew into

Context engineering is deciding, for every request, exactly what the model gets to see: the instructions, the knowledge retrieved for this question, what the system remembers about the user, the tools it can call, the results of tools it already called, and the conversation so far. Prompt engineering is the part of that about wording the instructions. In production, it is rarely the part that breaks.

What is context engineering?

A model knows two things when it answers: what it learned in training, and what is in its context window for this request. You cannot change the first at request time. Context engineering is the discipline of controlling the second: choosing the sources, selecting from them, fitting the result into a budget, and putting it in an order and format the model uses well.

The term caught on as teams moved from single prompts to agents and retrieval systems, where most of the context is no longer written by a person. It is assembled by code, on every request, from data that changes.

How is context engineering different from prompt engineering?

	Prompt engineering	Context engineering
Unit of work	The instructions	Everything the model sees for a request
Written by	A person, once	Code, on every request
Main question	How do I phrase this?	What does the model need to know right now, and what should it not see?
Typical failure	Ambiguous instructions	Missing, stale, excessive or forbidden information
Where it lives	A prompt file	Retrieval, memory, tool and state layers, plus the prompt

Good instructions still matter. They are simply one input among six or seven, and usually the most stable one.

What goes into a model's context?

• Instructions: the system prompt, rules and output format.
• The request: what the user asked or the event that triggered the run.
• Retrieved knowledge: passages from your documents and data (see how RAG works).
• Memory: facts kept about the user or task from earlier sessions (see agent memory, planning and stopping).
• Tool definitions: the functions the model may call, with their schemas.
• Tool results and history: what earlier steps returned, and the conversation so far.
• Task state: where a multi-step process stands, such as which approvals are done.

Each source has its own owner, freshness and permission rules. Context engineering is where they meet.

Why does more context make answers worse?

Because a model does not read its context the way a database runs a query. Everything in the window competes for attention. Long contexts dilute the passages that matter, and models are measurably worse at using information buried in the middle of a long input than at the start or end. Every extra token also adds cost and latency, and every stale or irrelevant passage is one more thing the model can confidently repeat.

A large context window is a ceiling, not a target (see a context window is not memory).

How do you assemble context for each request?

Treat it as a build step with a budget, not as string concatenation:

context/assemble.py

BUDGET = {                       # tokens per section; the total stays well under the window
    "instructions": 1_500,
    "task_state": 500,
    "memory": 800,
    "knowledge": 4_000,
    "history": 1_500,
}

def assemble(req: Request, user: User) -> Context:
    ctx = Context(instructions=load_prompt(req.task, version=req.prompt_version))
    ctx.add("task_state", state_store.summary(req.run_id), budget=BUDGET["task_state"])
    ctx.add("memory", memory.relevant(user, req.text, scope=user.permissions), budget=BUDGET["memory"])
    passages = retrieve(req.text, scope=user.permissions)          # filtered before ranking
    ctx.add("knowledge", dedupe(passages), budget=BUDGET["knowledge"], cite=True)
    ctx.add("history", recent_turns(req.conversation), budget=BUDGET["history"])
    trace.record_context(req.id, ctx.manifest())                 # what went in, from where, how many tokens
    return ctx

Four habits carry most of the value:

1. Budget per section, so one source cannot crowd out the others.
2. Permission filtering before selection, so nothing reaches the model that the user could not see (see filter by permission before you rank).
3. Provenance on every passage, so the model can cite it and you can trace a bad answer to its source.
4. A manifest in the trace, so you can see what the model actually saw when an answer goes wrong (see trace and cost every model call).

How do agents change the problem?

An agent calls the model in a loop, and the naive design appends every tool result and every thought to the context. By step ten the window is full of old search results, half of them irrelevant, and the model starts losing the thread.

Three changes fix most of it:

• Keep task state outside the model. A small, structured record of what is done and what is next is more reliable than a long transcript the model has to re-read.
• Summarise tool results as they arrive. Keep the facts the task needs; drop the raw payload, or store it and keep a reference.
• Rebuild the context each step from state, memory and fresh retrieval, instead of growing one transcript forever. This is sometimes called compaction.

Decision record DR-37 (layer L3)

Context	A support agent handled long cases with many tool calls. After a dozen steps it repeated searches, missed earlier findings and cost several times more per case than short ones.
Options	Move to a model with a larger context window; Truncate the oldest turns when the window fills; Keep case state outside the model and rebuild the context at each step
Chosen	A structured case record (facts found, actions taken, open questions) updated after every step, with the context rebuilt from that record, the last two turns and fresh retrieval.
Rejected	A larger window delayed the problem and raised cost per step. Truncation dropped exactly the early findings the agent kept repeating searches for.

The model can only be as good as what you put in front of it. Context engineering is choosing that, on purpose, every time.

How do you know the context is right?

Measure it the same way as everything else, with an evaluation set (see ship a prompt change with an eval):

• Was the needed information in the context? Check this separately from whether the answer was right. If it was missing, the fix is retrieval or memory, not the prompt.
• Remove one section at a time and re-run the eval. Sections that do not change the score are cost without benefit.
• Watch tokens per request in your traces. A slow upward drift is usually context nobody decided to add.

Where to go next

The vocabulary used here, from tokens to memory, is defined in the glossary. For the pieces of the context in depth, read how RAG works, a context window is not memory and agent memory, planning and stopping.

---

First published on the Agensphere Journal. Agensphere builds production AI systems: agensphere.com.