Your AI agent isn't hallucinating- it's reading garbage context

Ashish Bagri — Mon, 20 Jul 2026 12:39:51 +0000

Your agent isn't hallucinating. It's reasoning correctly over the wrong inputs.

Here's a failure pattern every team running agents in production has hit:

An alert fires. The agent investigates: pulls metrics, checks recent deploys, scans logs, proposes a fix. The fix is confidently, articulately wrong. Instinct says blame the model (bad reasoning, needs a better prompt, maybe a bigger model).

Then someone reconstructs what the agent actually saw. The metrics query returned a 5-minute-old cached aggregate. The deploy list was fetched before the relevant deploy landed. The log window was truncated at 1,000 lines and the line that mattered was #1,014. Given those inputs, the agent's conclusion was reasonable. It just wasn't debugging the incident that happened.

It's garbage in, garbage out, with a twist that makes it worse for agents than for any previous software. A dashboard shows you the garbage. A human sees a stale chart and might notice the timestamp. An agent consumes the garbage silently and acts on it, with fluent reasoning layered on top. The output doesn't look like garbage; it looks like a confident, well-argued investigation. That confidence is what makes it dangerous.

Why is this surfacing now

For chatbots, context was mostly a retrieval problem over documents; mediocre retrieval meant a mediocre answer a human would shrug at. Three things changed with production agents:

Agents act. A stale metric doesn't produce an off paragraph; it restarts the wrong service or pages the wrong team at 3AM. The cost went from cosmetic to operational.
The input surface exploded. A production agent reads metrics, logs, deploy history, tickets, and chat from separate systems, each with its own latency, rate limits, caching, and clock. The "world" it reasons over is stitched together from partial snapshots, inside the model's reasoning loop, where nobody can inspect it.
Errors compound. A single wrong input gives a slightly wrong answer. A 20-step investigation where step 3's conclusion becomes step 4's premise turns a small input error into a confidently wrong remediation.

Teams see the failure, file it as a reasoning problem, and reach for a bigger model or a longer prompt. The failure was upstream of the model the whole time.

Five ways context goes bad

Stale: true when fetched, then the world moved. Caches and polling intervals manufacture staleness silently.
Partial: pagination limits, truncated log windows, retries that quietly gave up. The agent reasons over what arrived, not what didn't.
Unordered: events from six systems, six clocks, interleaved wrong. Causality inverts: the deploy looks like it followed the errors it caused.
Lossy: every transformation in the pipeline throws something away on purpose. Embeddings drop exact identifiers and word order. Summarization drops whatever seemed unimportant; the exact judgment the agent exists to make. Stack a few of these and the model ends up reading a paraphrase of a summary of a sample.
Unattributed: the agent can't tell where a fact came from or when it was captured. Two contradictory data points arrive with equal authority, and the model just picks one.

None of these are model problems. A better model reasons more elegantly over the same bad inputs and arrives at the same wrong answer with nicer prose.

"Just wait for bigger context windows"

Context windows are in the millions of tokens now. Why not pipe everything in raw and let the model sort it out? Three problems:

Long context degrades. The "lost in the middle" effect (Liu et al.) and the more general context rot (Chroma's study, tested across 18 models) both show accuracy dropping as input length grows. A million-token window isn't a million tokens of reliable attention.
Cost scales with tokens. Every token is paid for on every reasoning step. For an agent running hundreds of investigations a day, "send everything" is a bad unit-economics decision.
The window can't fix what never arrived. Freshness, completeness, ordering, and provenance are properties of the systems collecting the data. No model recovers a log line truncated upstream or un-stales a cached metric.

Context engineering is a systems discipline

"Context engineering" has displaced "prompt engineering" for a reason. Anthropic's own guidance treats context as an engineering problem, not a wording problem. But even that's often applied narrowly, just packing the window better. The real discipline looks more like data infrastructure than writing.

Good context is:

Complete over a defined window — "everything about service X in the last 30 minutes" is a guarantee, not a hope.
Fresh, with staleness measured, not accidental.
Ordered, on one clock, so causality survives.
Lossless where it counts. Structured and typed for cheap querying, but with original records preserved.
Attributed: traceable to a source and a moment.
Replayable: the property that matters most after something breaks. The postmortem question isn't "why did the model say that," it's "what did the model see." If context was assembled from live API calls, you can't answer that. Τhe caches have rotated and the APIs have moved on. Treating agent context as a durable, ordered record turns forensics from guesswork into a replay.

The takeaway

The industry has spent two years making agents better at thinking and almost no time making sure they're thinking about the right thing. The models are already good enough to be dangerous when fed garbage and fluent enough that the garbage stays invisible until the postmortem.

Next time an agent fails in production, run the cheap experiment before reaching for a bigger model: reconstruct exactly what it saw, and ask if you'd have done better with the same inputs. Usually the answer is no. What looks like a reasoning failure is usually a context failure, and context failures are fixable with engineering today.