Your Agent Used 6% of Its Context Window and Still Lost a Third of Its Tool Output

By Igor Eduardo · Austin, TX · https://igoreduardo.com

A short new paper, Context Is a Pipeline, Not a Window: Information Attrition in LLM Agent Harnesses (arXiv 2610.36114, posted 2026-10-26), measures something most agent dashboards never show: how much of a tool's output is lost before the model ever sees it.

The authors instrumented a real tool-using agent that diagnoses injected faults by reading Linux kernel traces. Their point is simple. An agent never reads its context window directly. It reads whatever the harness, the code wrapped around the model, decided to put there. Between "the tool produced a result" and "the model saw it", the harness serializes, cuts to a budget, keeps or drops history, and passes findings between workers. Each of those steps can lose information.

What they report:

• The window was nearly empty. Across 720 runs, the single largest prompt used 5.75% of the window.
• A third of tool results were cut anyway. 3,827 of 11,189 tool results were truncated before reaching the model, and every cut result became broken JSON.
• The loss followed position, not importance. The harness sliced each result's text at a fixed length, so whatever sat near the end disappeared. One list of containers reached the model in 4.7% of calls; a second list, in the very same payloads, reached it in 99.3%.
• Accuracy split along the same line. Faults that showed up in the surviving list scored 40–55 out of 60; faults that showed up only in the deleted list scored 0–24 out of 60. The authors are explicit that this field split is correlation, not proven cause.
• On a production-scale trace, a whole evidence class vanished. Replaying the shipped cut on one real trace delivered 0 of 128 records of the fault's evidence.
• The first cost was extra work, not wrong answers. In a controlled experiment, when the agent had another route to the answer it still found it, but tool calls rose from 53 to 76 (the paper's "43% more"), computations roughly doubled (18 to 36), and yield, findings per computation, fell from 0.61 to 0.31.
• It hides well. Only 2.5% of calls were affected in their replay of 864 recorded results, and in traces from two production coding agents, waiting on the model took a median 84% of wall-clock time, so latency dashboards barely move.

The authors also surveyed seven agent frameworks and found two that cut tool output by position in the same way. I am not naming them here; the paper does, and the point is the pattern, not a scoreboard.

Scope, in their words: one harness, one domain, one model, small samples, and the causal link to the accuracy drop is still an open question. Those are their numbers, not mine. I have not re-run the experiments.

The stake

Teams will read "context window: mostly empty" as "the model saw everything".

Context size and final accuracy are the two numbers agents are usually judged on. Neither tells you whether the evidence the model needed was delivered intact. A log that records what the harness sent cannot show what the tool produced, so this class of loss is invisible in the place people look for it.

And it fails quietly. When the agent has a second route to the answer, the loss shows up as more calls and more tokens. When it does not, as with a quiet fault that lives in exactly one field, the evidence is simply gone and the agent answers anyway.

Opinion (one sentence)

For any agent that acts on tool output, "what reached the model" should be a measured, reported number next to accuracy, because a harness can delete the evidence that matters while every capacity and latency metric looks healthy.

That is a preference with a limiter, not a benchmark result. If your tools return small, flat outputs that never approach a budget, this may cost you nothing. Once tools return lists, logs or traces that can grow, the delivery step becomes part of the system under test.

What a delivery-aware agent eval looks like (menu, not recipe)

Column	Question it answers	Failure if skipped
Produced vs. delivered	Did each field of the tool result reach the model?	Evidence lost upstream looks like model error
Payload validity	Did the delivered result still parse?	Broken structure passed off as "the data"
Loss by position	Do late fields disappear more than early ones?	Importance decided by key order
Work per finding	How many calls and computations per useful finding?	A silent cost rise read as "harder task"
Hand-off between workers	Did computed findings reach whatever decides?	Silence read as "nothing found"
Quiet-case accuracy	Accuracy on cases with only one route to the evidence	Redundant routes masking the loss

None of that requires a private harness or a new model. It requires logging the tool's full result next to what was actually sent, and refusing a release that only reports window size and final accuracy.

Why "the answer was still right" is not a pass

The finding I would steal first is the compensation tax. In the controlled runs, accuracy on the easy, loud cases stayed flat while the agent did far more work to get there. That is exactly the kind of result that passes a pass-rate gate and then fails in production on the one case where the second route does not exist.

The authors' own prediction was wrong in an instructive way: they expected that deleting the culprit's list would drop accuracy, and the agent recovered the name from another table. Their sharper rule is that the cut hurts only where there is no second route, and that its cost appears as extra work before it appears as errors. If your eval set is full of loud cases, you will measure the work and miss the risk.

Where this connects to habits I already keep public

My public work is mostly retrieve-first and evaluation-as-contract systems, and the habit is the same: grade each stage on its own before letting the final answer vouch for the pipeline. In retrieval, that means checking that the right evidence came back before grading the answer. For agents, this paper isolates the next stage: whether the evidence that came back actually reached the model. A system can retrieve correctly and still never show the model what it retrieved.

Steal this check

Pick one agent that reads tool output and add four things before your next release:

1. Log produced and delivered side by side. Keep the full tool result and the text the model actually received, per call.
2. Count what didn't arrive. Per field, per run: how often was it cut, and did the payload still parse?
3. Track work per finding. Calls and computations per useful result, next to accuracy.
4. Keep a quiet-case slice. Cases where the evidence lives in exactly one place. Grade accuracy there separately.

Then ask whether the release is still an upgrade once delivery is visible.

What I am not claiming

• I am not reporting this paper's results as my own, and I have not reproduced them.
• I am not ranking frameworks, models or vendors.
• I am not claiming truncation caused the accuracy split. The authors say that is still open.
• I am not publishing budgets or cut rules to use. Those depend on your tools and what a missed field costs you.

The point

A mostly empty context window tells you there was room. It does not tell you what arrived. Measure the path from tool to model, report loss next to accuracy, and keep a slice of cases where the agent has only one way to find the truth.

---

Igor Eduardo builds retrieve-first and evaluation-as-contract systems. More at igoreduardo.com.