Context Engineering: Why Your AI Agent Needs a Database, Not a Prompt
Published: August 22, 2026 | Focus Keyword: context engineering for AI agents | Est. read time: 14 minutes
Table of Contents
- The 24% Problem — Why Your Agent Keeps Failing in Production
- The Six Context Primitives (and Where Each One Breaks)
- The Database Insight — Four Organization Forms
- Three-Tier Loading — The Architecture Pattern That Changes Everything
- Benchmark Deep Dive — Real Numbers, Real Impact
- Hands-On: Building a Context-Engineered Agent
- Governance Layer — Making Agent Context Auditable
- Context Engineering as the New SRE Discipline
- Conclusion
The 24% Problem — Why Your Agent Keeps Failing in Production
You've built the agent. It passes every eval. Then you deploy it.
On day one, it's brilliant. By week three, it's recommending a customer return a product they've already returned twice before, referencing a policy that changed six weeks ago, and confidently calling an API endpoint that was deprecated in the last sprint. You've tuned the prompt a hundred times. You've tried longer system prompts, few-shot examples, chain-of-thought. The agent is still stuck at 24% accuracy on long-horizon tasks.
Here's the uncomfortable truth: the model isn't the problem. The context is.
This is the inflection point the ML engineering community hit in mid-2026. When OpenViking — VolcEngine's open-source context database for AI agents — became the #1 trending Python repository on GitHub, it wasn't because engineers were excited about another RAG wrapper. It was because they recognised something more profound: the problem of agent memory had outgrown the vocabulary of prompting. It had become a database problem.
Context engineering for AI agents is the emerging discipline of designing and managing the information environment in which your agent operates — not as a static prompt, but as a living, structured, tiered data system. Done right, it transforms that 24% agent into one running at 82%.
This post is the technical deep-dive you need to understand why, and how to build it.
The Six Context Primitives (and Where Each One Breaks)
Before we talk about the solution, let's precisely name the problem. Every AI agent draws on some combination of six context primitives:
1. In-Context Window
The text passed directly in the prompt. Fast, zero-latency, but brutally limited. A 1M token window sounds like infinite space until you're running a multi-day coding agent across a 500K-line codebase. And crucially, not all tokens in a long context are attended to equally — the "lost in the middle" problem means your critical instructions buried at position 300K may as well not exist.
2. Retrieval-Augmented Generation (RAG)
The standard fix — embed your knowledge base, retrieve the top-k chunks at query time. RAG is essential, but it fails in two ways: precision collapses on multi-hop queries (asking about a relationship between two entities that each live in separate chunks), and it has no memory of what it already retrieved. Every turn is stateless.
3. Web Search / Tool Calling
Real-time grounding via search or APIs. Excellent for current events, terrible for internal knowledge. And as the August 2026 UK AISI incident report showed, agents with live web access in improperly sandboxed environments can cause real damage.
4. Skills & MCP Tools
Structured, typed callable functions. The Model Context Protocol (MCP) has standardised this. But skills are stateless by design — they do one thing, return a result, and forget. They don't accumulate knowledge across invocations.
5. Short-Term / Working Memory
The chat history buffer. This is the scratchpad that every agent has, but it's ephemeral — it dies with the session. It also grows unboundedly until it hits your context limit, at which point you truncate it and lose the beginning of your reasoning chain.
6. Long-Term Memory
The piece almost everyone gets wrong. Most teams implement this as "save embeddings of conversation turns to a vector database." This is better than nothing, but it's a poor approximation of what agents actually need.
# ❌ The naive pattern most teams ship today
# Problems: lossy, stateless across sessions, no structure,
# no tiering, no self-updating, no provenance
class NaiveAgentMemory:
def __init__(self, vector_db):
self.db = vector_db
def save(self, turn: str):
embedding = embed(turn)
self.db.upsert(embedding, metadata={"text": turn})
def recall(self, query: str, top_k: int = 5) -> list[str]:
results = self.db.query(embed(query), top_k=top_k)
return [r.metadata["text"] for r in results]
# No hierarchy. No tiering. No graph relations.
# No self-evolution. No provenance. No governance.
# This is not a memory system. This is a search index.
The problem is structural: you're using a search engine to solve a database problem. A search index answers "what text is similar to this query?" A database answers "what is the state of this entity, what changed, when, and why?"
The Database Insight — Four Organization Forms

The four storage forms that together constitute a complete agent context database. Each serves a distinct access pattern — no single form is sufficient alone.
The OpenViking framework, whose VikingMem paper was accepted to VLDB 2026 (the top database systems conference), defines context engineering for AI agents around four complementary organization forms. Think of them as the four tables in your agent's relational schema:
Form 1: Vector Store (Semantic Similarity)
What it's good at: fuzzy recall, concept-level retrieval, semantic search across unstructured text.
What it's bad at: precise lookups, relational joins, structured queries.
When to use it: retrieving relevant past episodes, similar code patterns, analogous situations.
Form 2: Filesystem (Hierarchical Structure)
What it's good at: navigating large knowledge bases with known structure, progressive disclosure, lazy loading.
What it's bad at: fuzzy search, ad-hoc queries.
When to use it: project documentation, codebase knowledge, anything with a natural tree structure.
OpenViking's viking:// protocol is the most elegant implementation of this pattern — it gives your agent a virtual filesystem address space for all its knowledge, with path-based access that mirrors how humans and IDE tools naturally navigate information.
# OpenViking filesystem protocol example
# Agent can navigate context like a filesystem
viking://project/architecture/decisions/adr-042-database-choice.md # L2: Full ADR
viking://project/architecture/decisions/ # L1: ADR index
viking://project/architecture/ # L0: "project uses PostgreSQL, event sourcing"
Form 3: Relational/Table Store (Structured State)
What it's good at: precise lookups, aggregations, current state of structured entities.
What it's bad at: unstructured text, semantic search.
When to use it: user profiles, task state, tool call history, API response caches.
-- Agent context as structured state
-- This is what you actually want for entity tracking
CREATE TABLE agent_context_entities (
entity_id TEXT PRIMARY KEY,
entity_type TEXT NOT NULL, -- 'user', 'task', 'codebase', 'decision'
state JSONB,
last_updated TIMESTAMPTZ,
session_count INT DEFAULT 0,
confidence FLOAT -- agent's confidence in this knowledge
);
CREATE TABLE agent_context_relations (
from_entity TEXT REFERENCES agent_context_entities(entity_id),
relation_type TEXT,
to_entity TEXT REFERENCES agent_context_entities(entity_id),
evidence TEXT,
strength FLOAT
);
Form 4: Knowledge Graph (Relational Reasoning)
What it's good at: multi-hop reasoning, relationship traversal, inferring implicit connections.
What it's bad at: fuzzy lookup, scale (can get expensive for large graphs).
When to use it: reasoning about how concepts, people, decisions, and code artifacts relate to each other.
The combination of all four forms is what transforms a "memory-augmented chatbot" into an agent that genuinely knows things — with structure, provenance, and the ability to update its knowledge as the world changes.
Three-Tier Loading — The Architecture Pattern That Changes Everything

L0 gives the agent orientation (100 tokens). L1 gives structure (2K tokens). L2 provides full detail only when needed — dramatically reducing token consumption and latency.
Understanding what to store is only half the battle. The other half is understanding how much of it to put in the context window at any given moment.
The naive approach: stuff everything into the prompt. Result: slow, expensive, attention-diluted.
The smarter approach: tier your context loading.
OpenViking's three-tier system is the most rigorous implementation of this pattern:
L0 — Abstract Summary (~100 tokens)
A compressed, always-present header for each knowledge unit. Think of it as the card in a card catalogue — just enough to know whether this document is relevant without loading the document itself.
L0 example for a microservice's context entry:
"payment-service: Stripe-based payment processing. Owns /payments/* endpoints.
Last updated 2026-08-15. 3 known issues. 2 pending breaking changes."
The agent loads ALL L0 summaries for a project at start — total cost: perhaps 5K tokens for a 100-module codebase.
L1 — Structured Overview (~2,000 tokens)
The table of contents plus key facts — loaded when the L0 signals relevance. For a service, this might include its API contract, key dependencies, recent change history, and known issues.
The agent loads L1 only for services that are likely relevant to the current task — cutting irrelevant loading entirely.
L2 — Full Detail (On-Demand)
The complete knowledge artifact: full source code, full documentation, full conversation history. Loaded only when the agent needs to reason about specifics.
# ✅ The tiered context loading pattern
# Dramatically reduces token usage while preserving recall accuracy
class TieredContextDB:
def __init__(self, viking_client):
self.db = viking_client
async def load_context_for_task(self, task: str, budget_tokens: int = 8000):
"""Smart tiered loading — load only what's needed."""
# Step 1: Always load ALL L0 summaries (cheap — ~100 tokens each)
l0_summaries = await self.db.load_tier(level=0, scope="all")
# Step 2: Score L0 summaries against the task
relevant = self.rank_by_relevance(l0_summaries, task, top_k=10)
# Step 3: Load L1 for top candidates (2K tokens each, load ~3-5)
l1_details = []
remaining_budget = budget_tokens - sum(s.token_count for s in l0_summaries)
for candidate in relevant[:5]:
if remaining_budget < 2000:
break
l1 = await self.db.load_tier(level=1, entity_id=candidate.id)
l1_details.append(l1)
remaining_budget -= l1.token_count
# Step 4: L2 loaded lazily during reasoning — only if agent requests it
context = ContextBundle(
always_present=l0_summaries,
structured_detail=l1_details,
lazy_loader=lambda entity_id: self.db.load_tier(level=2, entity_id=entity_id)
)
return context
async def evolve(self, task: str, result: str, agent_trace: list):
"""Self-evolution: update the DB based on what the agent learned."""
new_knowledge = await self.extract_knowledge(agent_trace)
await self.db.merge(new_knowledge) # Viking's conflict-resolution merge
await self.db.regenerate_summaries(affected_entities=new_knowledge.entities)
The tiering principle maps directly to how experienced engineers actually work: you scan filenames first, read READMEs second, and read source code only when necessary. The difference is your agent now does this systematically, cheaply, and automatically.
Benchmark Deep Dive — Real Numbers, Real Impact

The performance gap between naive retrieval and structured context engineering is not incremental — it is categorical. These numbers are from published evaluations on production-grade benchmarks.
Let's be precise about what the numbers actually measure and mean.
LoCoMo: Long-Context Conversational Memory (the 24% → 82% result)
LoCoMo is a benchmark specifically designed to test agents on long-running conversational scenarios — the kind where a customer support agent needs to remember a user's history across dozens of sessions, or a coding agent needs to track decisions made three weeks ago.
| System | Accuracy | Token Cost | Latency |
|---|---|---|---|
| Baseline (naive RAG) | 24.20% | 1× (baseline) | 1× (baseline) |
| OpenViking (Claude Code backend) | 80.32% | −34% | −59% |
| OpenViking (OpenClaw native) | 82.08% | −91% | −66% |
| OpenViking (Hermes) | 82.86% | ~−85% | ~−62% |
The 3.39× accuracy improvement is striking. The 91% token reduction is arguably more important for production systems — it's the difference between a context-enriched agent that costs $0.003/query and one that costs $0.033/query. At scale, that's an order of magnitude difference in operational cost.
HotpotQA: Multi-Hop Knowledge Retrieval
HotpotQA tests the ability to answer questions that require chaining multiple facts — the bread-and-butter of any non-trivial agent task.
| System | Accuracy | Index Cost | Latency |
|---|---|---|---|
| LightRAG | 89.00% | 62.7M tokens | 75.0 seconds |
| OpenViking | 91.00% | 8.67M tokens | 0.23 seconds |
The 326× latency improvement (75s → 0.23s) is not a typo. The structural tiering means OpenViking can answer multi-hop questions by navigating its filesystem-shaped knowledge index rather than running expensive graph traversals or sequential LLM calls. The indexing cost savings (62.7M → 8.67M tokens, an 86% reduction) also dramatically cut the cost of onboarding new knowledge.
tau2-bench: Production Task Completion (the deployment test that matters)
tau2-bench tests agents on real-world task completion scenarios in retail and airline customer service — domains with high entity complexity, policy lookups, and state management requirements.
| Agent | Baseline | With Context DB | Δ |
|---|---|---|---|
| Retail agent | 70.94% | 77.81% | +6.87pp |
| Airline agent | 54.38% | 66.25% | +11.87pp |
A +11.87 percentage point improvement in a production task completion benchmark is the kind of result that changes quarterly metrics for AI product teams. These are not toy improvements.
Hands-On: Building a Context-Engineered Agent
Enough theory. Let's build something. The following walkthrough takes you from zero to a context-engineered agent in under 30 minutes.
Step 1: Install and Initialise OpenViking
# Install OpenViking
pip install openviking
# Initialise a context database for your project
viking init my-agent-context
cd my-agent-context
# The init creates a .viking/ directory with:
# .viking/
# config.yaml # storage backends, tiering config
# entities/ # L0/L1/L2 knowledge artifacts
# relations/ # graph edges
# sessions/ # conversation history with self-evolution logs
# provenance/ # audit trail (W3C PROV-O)
Step 2: Ingest Your Knowledge Base
# ingest.py — One-time setup: populate your context DB from existing sources
import asyncio
from openviking import Viking, Ingester
async def ingest_codebase():
viking = Viking(db_path=".viking")
ingester = Ingester(viking)
# Ingest a code repository — Viking auto-generates L0/L1/L2 for each module
await ingester.ingest_repository(
path="./src",
entity_type="codebase",
chunk_strategy="by_module", # or "by_file", "by_function"
generate_summaries=True, # LLM-generated L0 and L1 summaries
extract_relations=True, # Build the knowledge graph
)
# Ingest documentation
await ingester.ingest_docs(
path="./docs",
entity_type="documentation",
)
# Ingest past decision records
await ingester.ingest_files(
pattern="./decisions/adr-*.md",
entity_type="architecture_decision",
)
print(f"Ingested {len(await viking.list_entities())} entities")
print(f"Built {len(await viking.list_relations())} relations")
asyncio.run(ingest_codebase())
Step 3: Wire It Into Your Agent
# agent.py — A context-engineered agent using OpenAI or Anthropic
import asyncio
from openviking import Viking
from openai import AsyncOpenAI # works identically with anthropic.AsyncAnthropic
class ContextEngineeredAgent:
def __init__(self):
self.viking = Viking(db_path=".viking")
self.llm = AsyncOpenAI()
self.session_id = None
async def start_session(self, session_id: str):
"""Begin a new agent session — loads L0 context automatically."""
self.session_id = session_id
# Viking loads all L0 summaries (~100 tokens each) as the base orientation
self.base_context = await self.viking.session_start(
session_id=session_id,
load_tier=0, # Always-present L0 summaries
scope="all", # Across all knowledge entities
)
return self.base_context
async def run(self, user_message: str) -> str:
"""Process a message with full context engineering."""
# Step 1: Viking scores L0 summaries and fetches relevant L1 detail
enriched_context = await self.viking.get_context_for_query(
query=user_message,
session_id=self.session_id,
l1_top_k=5, # Load L1 for top 5 relevant entities
token_budget=12000, # Hard cap on context tokens
include_relations=True, # Add graph edges for multi-hop reasoning
)
# Step 2: Build the system prompt dynamically from structured context
system_prompt = f"""You are a helpful engineering assistant.
## Project Context (Auto-loaded by Viking Context DB)
### Always-Present Knowledge (L0 — All Entities)
{enriched_context.l0_overview}
### Relevant Detail (L1 — Top Matches for This Query)
{enriched_context.l1_details}
### Active Relations (Knowledge Graph Edges)
{enriched_context.relations}
### Session Memory (What We've Established This Session)
{enriched_context.session_memory}
If you need deeper detail on any entity, call the `load_context` tool with the entity ID.
"""
# Step 3: Run the LLM with L2 lazy-loading tool
response = await self.llm.chat.completions.create(
model="gpt-5.6-terra",
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_message},
],
tools=[{
"type": "function",
"function": {
"name": "load_context",
"description": "Load full (L2) detail for a specific knowledge entity",
"parameters": {
"type": "object",
"properties": {
"entity_id": {"type": "string", "description": "The entity ID from L0/L1 summaries"}
},
"required": ["entity_id"]
}
}
}]
)
# Step 4: Handle L2 lazy loading if the agent requests it
if response.choices[0].finish_reason == "tool_calls":
tool_call = response.choices[0].message.tool_calls[0]
entity_id = eval(tool_call.function.arguments)["entity_id"]
# Load L2 detail on demand — only when the agent actually needs it
l2_content = await self.viking.load_tier(level=2, entity_id=entity_id)
# Continue the conversation with L2 content injected
# ... (standard tool response handling)
agent_response = response.choices[0].message.content
# Step 5: Self-evolution — Viking extracts new knowledge from this turn
await self.viking.evolve_from_turn(
session_id=self.session_id,
user_message=user_message,
agent_response=agent_response,
auto_merge=True, # Automatically merge new facts into the DB
confidence_threshold=0.85, # Only merge high-confidence extractions
)
return agent_response
# Usage
async def main():
agent = ContextEngineeredAgent()
await agent.start_session("engineering-session-001")
response = await agent.run("Why did we choose PostgreSQL over MongoDB for the payments service?")
print(response)
# Agent correctly cites ADR-042, the decision context,
# and the relation to the payments-service entity — without hallucinating.
asyncio.run(main())
Step 4: Validate the Self-Evolution Loop
After running several sessions, inspect how the context DB has evolved:
# Check what the agent has learned
viking status
# Output:
# Entities: 247 (was 180 at ingest — agent added 67 from sessions)
# Relations: 1,843 (was 1,200 — 643 new edges discovered)
# L0 freshness: 98.7% current (auto-regenerated when entities changed)
# Sessions: 14 sessions, 89 turns indexed
# Evolution: 43 knowledge merges, 12 conflicts resolved, 0 contradictions pending
# Inspect a specific entity's evolution history
viking history --entity payment-service
# View the provenance of a specific fact
viking provenance "payment-service uses Stripe"
# → Extracted from session-003, turn 7, with 0.94 confidence
# Confirmed in session-008, turn 2
# Source: human engineer statement + codebase scan match
Governance Layer — Making Agent Context Auditable
Production AI deployments in 2026 face a compliance requirement that most context engineering discussions skip entirely: auditability. If your agent makes a decision — recommends a refund, blocks an account, generates a contract clause — you need to be able to reconstruct exactly what context it had when it made that decision.
Semantica — another trending GitHub project this week — addresses this with a graph-native governance layer built on:
- W3C PROV-O: Every piece of knowledge has a provenance chain — who asserted it, when, from what source
- SHACL constraints: Validation rules that prevent malformed or contradictory knowledge from entering the context DB
- Rete/Datalog reasoning: Deterministic inference rules that don't involve an LLM — making certain reasoning steps fully auditable
# governance.py — Adding auditability to your context DB
from semantica import SemanticaGraph, ProvenanceTrace, SHACLValidator
class AuditableContextDB:
def __init__(self, viking_client, semantica_graph):
self.viking = viking_client
self.graph = semantica_graph
self.validator = SHACLValidator(schema_path="schemas/agent-context.shacl.ttl")
async def merge_with_provenance(self, new_knowledge: dict, session_id: str):
"""Merge new knowledge with full PROV-O provenance tracking."""
# Validate against SHACL schema before merging
validation_result = self.validator.validate(new_knowledge)
if not validation_result.conforms:
raise ContextValidationError(
f"Knowledge rejected: {validation_result.violations}"
)
# Create provenance record (W3C PROV-O)
provenance = ProvenanceTrace(
activity_id=f"merge-{session_id}-{timestamp()}",
agent_id="context-engineering-agent-v2",
used=[session_id], # Which session generated this
generated_at=datetime.utcnow(),
confidence=new_knowledge.get("confidence", 0.0),
)
# Merge into knowledge graph with provenance
await self.graph.merge(
triples=new_knowledge["triples"],
provenance=provenance,
)
# Synchronise with Viking's tiered storage
await self.viking.sync_from_graph(self.graph, affected_entities=new_knowledge["entities"])
async def explain_decision(self, decision_id: str) -> str:
"""Full audit trail for a specific agent decision — SPARQL query."""
query = f"""
PREFIX prov: <http://www.w3.org/ns/prov#>
PREFIX agent: <https://your-org.com/agent-ontology#>
SELECT ?fact ?source ?session ?timestamp ?confidence
WHERE {{
agent:decision-{decision_id} agent:usedFact ?fact .
?fact prov:wasAttributedTo ?source .
?fact agent:extractedInSession ?session .
?fact prov:generatedAtTime ?timestamp .
?fact agent:confidence ?confidence .
}}
ORDER BY DESC(?timestamp)
"""
results = await self.graph.sparql(query)
return self.format_audit_trail(results)
The value proposition for enterprise teams is clear: when the compliance team asks "why did the agent recommend X?", you can produce a timestamped chain of evidence rather than a shrug.
Context Engineering as the New SRE Discipline
Here's the conceptual shift that takes this from a useful library to a career-defining paradigm:
The old model: Hire ML engineers to fine-tune models, prompt engineers to craft system prompts, and DevOps to deploy them. The model is the product.
The new model: The model is a commodity. The context infrastructure is the product. The engineers who build, maintain, and evolve context databases — who define tiering strategies, self-evolution policies, provenance schemas, and conflict resolution logic — are the ones generating leverage.
This maps directly to the emergence of SRE as a discipline: when compute became cheap and reliable, the engineers who operationalised that reliability at scale became the most valuable people in the room. Context engineering is that moment for AI agents.
What does a "Context Engineer" actually do?
Context Engineer Responsibilities (2026 Job Description Draft):
✅ Design the entity taxonomy for the agent's knowledge domain
✅ Define tiering strategies (what goes in L0 vs L1 vs L2)
✅ Build ingestion pipelines for new knowledge sources
✅ Monitor context freshness and trigger regeneration
✅ Define self-evolution policies (what confidence threshold triggers a merge?)
✅ Design SHACL schemas for knowledge validation
✅ Build provenance dashboards for compliance teams
✅ Run context quality evaluations (is the agent's knowledge accurate?)
✅ Tune conflict resolution logic (what happens when two sources disagree?)
✅ Instrument context hit/miss rates and token usage per query
The last point deserves emphasis. Context engineering has metrics. You can measure L1 cache hit rate (how often does the L1 content you loaded actually get referenced?), knowledge staleness (how often is the agent corrected by a human because its L2 was out of date?), and evolution precision (what percentage of auto-merged knowledge survives the next manual review?). These are engineering metrics, not vibe metrics.
The tooling is arriving to match: the viking status command shown earlier, Semantica's audit dashboard, and the broader class of "agent observability" tools emerging in mid-2026 are all building toward the same vision — a production control plane for agent context, as rigorous as your database SLOs.
Conclusion
The 24% agent you shipped last quarter isn't a model problem. It's a context problem.
Context engineering for AI agents is the recognition that production agents need the same infrastructure investment we've always given to data: schema design, tiered storage, indexing strategies, provenance tracking, and operational observability. The model handles the reasoning. Your job is to ensure it reasons over the right information, at the right granularity, at the right cost.
The results speak for themselves: 24% → 82% accuracy on long-context memory tasks. 326× latency improvement on multi-hop retrieval. 91% token cost reduction. Double-digit percentage point improvements on production task completion benchmarks. These aren't benchmark games — the VikingMem paper's acceptance at VLDB 2026 signals that the top database research community agrees this is a serious systems problem deserving serious systems solutions.
Where to Start Today
-
Clone OpenViking and run
viking initon your current agent project:pip install openviking - Audit your current context primitives: which of the six are you using, and where are they breaking?
- Instrument your agent: log token usage per query, accuracy on held-out evals, and latency — you need a baseline before you can prove improvement
-
Start with the self-evolution loop: even before you restructure everything, adding
viking evolveto your turn completion is the highest-ROI single change - Read the VikingMem paper (arXiv:2605.29640) for the full theoretical grounding
The shift from prompt engineering to context engineering isn't just a new buzzword — it's the recognition that building production AI systems is a data engineering problem as much as it is an ML problem. The engineers who build that infrastructure in 2026 will be the ones who define what AI agents can actually do in 2028.
Enjoyed this deep dive? Follow me for more posts on AI systems engineering, agent architecture, and the infrastructure layer that makes production AI actually work. Drop questions or push back in the comments — especially if you've run your own context engineering experiments with different results.
References & Further Reading
- OpenViking GitHub — Open-source context database for AI agents
- OpenViking Benchmark Report — Full benchmark methodology and results
- VikingMem Paper — arXiv:2605.29640 — Accepted at VLDB 2026
- Semantica GitHub — Graph-native AI governance infrastructure
- Simon Willison's August 2026 Digest — Comprehensive AI developments tracker
- UK AISI Incident Report, July 2026 — Agent safety in production
Top comments (0)