Prompt engineering was one of the first major engineering disciplines around LLM applications.
Now AI agents are creating a larger problem:
How do we manage the information an agent needs while it is working?
That's context engineering.
Anthropic describes context engineering as managing the information available to an agent during inference, including instructions, tools, MCP, external data and message history.
A Simple Architecture
User
|
v
AI Agent
|
+-------+-------+
| | |
Memory Retrieval Tools
| | |
+-------+-------+
|
v
Context Builder
|
v
AI Model
|
v
Action/Response
Why Prompts Aren't Enough
A prompt might say:
You are a customer support agent.
But that's not enough.
The agent may also need:
Customer profile
Order status
Support history
Return policy
Inventory
Available tools
Business rules
The engineering problem becomes deciding which of these should be retrieved for each task.
Context Retrieval
A typical flow:
Request
↓
Classify task
↓
Retrieve relevant data
↓
Apply permissions
↓
Build context
↓
Run model
Don't blindly inject every available document into the context.
Memory
Long-running agents need state.
Separate:
Conversation state
Task state
Long-term memory
External knowledge
This makes the system easier to control and debug.
Tools
Tools are also part of context.
For example:
get_customer()
search_orders()
create_ticket()
update_crm()
The agent needs descriptions, parameters and authorization boundaries for each tool.
Coding Agents Need Context Too
A coding agent can read source files but may not understand:
Why an architectural decision was made
Which module owns a feature
Which APIs are deprecated
Which patterns the team follows
What a particular workaround is protecting against
Atlassian has been building broader code context capabilities around repositories and organizational knowledge.
Meta has similarly described using specialized agents to generate structured context across a large codebase.
Context Must Be Fresh
Some data should be retrieved live.
For example:
Inventory → live
Customer balance → live
CRM status → live
Company policy → versioned
Historical documentation → cached
The context layer needs to know the difference.
The Engineering Stack
Modern agentic software increasingly looks like:
LLM
+
Context
+
Retrieval
+
Memory
+
Tools
+
MCP
+
Business Logic
+
Permissions
+
Observability
The model is only one component.
Final Thought
The quality of an AI agent isn't determined only by the model it uses.
It is also determined by what the system allows that model to know, retrieve, remember and do.
That's why context engineering is becoming an important discipline for production AI.
Top comments (0)