Modern models can accept more context than ever. That helps developers build richer prototypes, but it can also encourage a dangerous assumption: if an agent has a large enough context window, it can keep working indefinitely.
Long-running agents fail for a different reason. The problem is not only how much information the model can technically receive. It is whether the runtime can keep the right information available, preserve execution state, control tool output, and recover after the working window has been reorganized.
This is why context engineering is becoming runtime engineering.
A long task is a sequence of model calls
A chatbot may answer after one or two model calls. An agent completing a real task can make dozens or hundreds. It searches, reads files, calls APIs, inspects results, revises its plan, requests approval, and tries again.
Every step adds new material:
- user instructions and business constraints;
- model decisions and intermediate plans;
- tool calls and their results;
- retrieved documents and database records;
- approval states, checkpoints, and generated artifacts.
Tool results are usually the fastest-growing part. A single repository scan, document extraction, or analytics query can return more tokens than the conversation that preceded it. Replaying all of that material on every inference call increases cost and latency while making the current task harder to see.
Google has described the same failure mode in ADK: long-running workflows accumulate irrelevant conversation, old tool output, and duplicated instructions. OpenAI also notes that tool-driven loops fill the context window quickly and now provides native compaction for extended workflows. Anthropic frames the underlying discipline as context engineering: selecting the smallest set of high-signal tokens that gives the model the best chance of taking the right next action.
The common conclusion is simple: a larger physical window does not remove the need for an actively managed working window.
References:
- OpenAI: From model to agent
- Google: Build long-running agents that pause and resume
- Anthropic: Effective context engineering for AI agents
Separate the model window from the task state
A reliable agent should not treat its prompt as the only copy of reality. At minimum, the runtime needs to distinguish four forms of state:
- Stable instructions — the agent’s role, policies, tool contracts, and the user’s current goal.
- Durable task state — checkpoints, approvals, pending actions, workflow position, and structured variables.
- Evidence and artifacts — original documents, tool outputs, generated files, and other material that may need to be inspected again.
- Working context — the compact, task-relevant view sent to the model for the next decision.
The working context is a view, not the database of record. It may be much smaller than the model’s maximum input window. That is intentional: the runtime needs space for the next tool result, the next model response, and unexpected branches in the task.
This also changes the meaning of memory. Memory is not “keep every message forever.” It is the ability to reconstruct the information required for the next correct action.
Govern context before every model call
In ZGI, context is treated as runtime working memory. Before a model call, the runtime estimates the request, preserves non-compressible state, reduces low-value history, rebuilds the working view, and validates it again.
Conceptually, the loop looks like this:
collect current task state
-> estimate the next request
-> protect instructions and active tool rounds
-> project oversized results into references
-> compact consumed history when necessary
-> rebuild the working context
-> validate budget and protocol integrity
-> call the model
Doing this before every inference is important. Agent context can grow suddenly: several parallel tools may finish in the same round, or one tool may return an unexpectedly large payload. A policy that runs only when the hard model limit is reached reacts too late.
Use layered compaction, not one destructive summary
The safest reduction is usually the least semantic one. ZGI therefore applies context reduction in layers.
1. Project oversized results
Large tool results are stored outside the model window. The model receives a bounded preview, metadata, and a reference to the complete result. The source remains available if the agent later needs a precise passage or value.
2. Replace consumed output with a receipt
Once the model has already used a tool result, the runtime does not need to replay the raw payload forever. A compact receipt can preserve the source, size, summary, content hash, and artifact reference.
3. Compact older complete rounds
If deterministic reduction is not enough, older execution history can be summarized. The boundary matters: a model response and all tool results produced from it form one complete API round. The runtime should never keep the tool call while dropping its result, or keep the result while losing the call that produced it.
4. Recover safely
If the rebuilt request still exceeds the hard limit, the runtime needs a controlled recovery path. It can perform a more aggressive final compaction, revalidate the result, and continue only if the task state remains legal. Otherwise it should checkpoint and stop clearly instead of retrying a request that cannot succeed.
Test continuity, not just token reduction
A context system can reduce token counts while quietly damaging the task. Useful tests therefore need to check more than input size.
ZGI’s validation scenarios include a single task running through 100 tool-call rounds with multiple compactions, parallel tool results remaining correctly paired, oversized artifacts being projected and reloaded, repeated summary failures converging safely, and complete execution history being restored across user turns.
The important questions are:
- Can the agent still identify the current goal?
- Are tool calls and results structurally valid?
- Can original evidence be recovered?
- Are approvals and checkpoints preserved?
- Does failure stop in a known state?
- Can developers inspect why compaction happened?
This is the difference between prompt trimming and runtime context management.
Context is now part of the execution layer
Long-running agents need more than larger windows. They need explicit task state, bounded working context, recoverable evidence, complete tool-call history, and a failure path that does not invent progress.
Models will continue to improve. Context windows will continue to grow. Neither change eliminates the runtime’s responsibility to decide what the model should see at each step.
ZGI is being built around that responsibility: helping agents keep working within finite windows while preserving the state and evidence required to finish real tasks.
The source is available on GitHub: [github.com/zgiai/zgi]
Top comments (0)