We shipped two context-engineering features in one afternoon. We reverted them by dinner.

On 2026-08-22 the agent harness I work on shipped two "smarter context" features within five hours of each other. Both were reverted before the day was over. This is the honest version of that afternoon, because the same-day revert is the less told half of the fast-feedback story.

Fast-fix stories get the attention: "issue to merged fix in 50 minutes" is a good headline. But the mechanism only works if you are equally fast at the other direction. A bet that loses is cheap to fold at hour five and expensive at week five. This is the story of two bets, both folded in one afternoon.

09:33 - the tool-result sliding window

The first feature started as a host complaint at 11:33 local: long agent sessions were drowning the LLM in tool outputs. Read a file, list a directory, run a test - each result is a fat blob of text, and after ten rounds the context is mostly yesterday's tool output.

The fix (#936, built 16:33): a sliding window. At send time, tool-result groups older than the most recent N rounds are folded into a placeholder:

[Tool results omitted - older than recent 7 rounds]
executed: read_file x3, bash x2
tool_call_ids: ...
full results: history.jsonl

The full outputs stayed on disk. The model could in principle backtrack to the file. On paper it is a clean token economy: keep the recent context, compress the old, leave an audit trail. It even had a config knob (tool_window_rounds, 0 disables).

13:25 - the reasoning pass-back

Six hours later a second complaint landed at 17:25: DeepSeek's thinking-mode reasoning was being discarded. The model spends tokens thinking, then only the final answer comes back. Why not persist the reasoning and pass it back into context on the next round? (#937, built 19:46.) More context for free - the model's own reasoning as memory.

21:12 and 21:23 - both folded

By 21:12 the reasoning pass-back was reverted. By 21:23 the sliding window was reverted too (#939, released as v0.2.70 at 21:47). Two features, built and shipped over the afternoon, both gone before dinner.

What was the problem? The commits carry the trigger but not the full autopsy, so here is what we can verify from the code and from what the features did:

The sliding window failed SILENT. Folding is a bet that "the model will not need those older outputs again". When the bet lost - a round 12 decision that needed the round 3 file listing - there was no error. The model simply had less evidence, and no mechanism anywhere would tell us that a specific omission changed an answer. The placeholder was a breadcrumb, but a breadcrumb only helps if you know to look. Loss without a signal is the worst failure mode for an agent harness, because the output still looks fine.

The reasoning pass-back failed UNMEASURED. It made nothing break, so nobody could tell it was working. Re-consuming hidden chain-of-thought as context costs tokens and changes the model's distribution; without a counter or an experiment, "pass it back" was a belief, not a mechanism.

The lesson: know which direction your guard fails

The general shape is what stuck. Both features TRANSFORMED context - compressed it or re-injected it - and neither had a countable signal for when the transformation was hurting. A reader on our token-counter post said it better than we did: a guard that fails safe (does nothing, keeps everything) trains people to ignore it, and a guard that fails silent (quietly drops or changes something) cannot be audited at all. Overcount and undercount are the same disease in mirror: you only watch the direction that is easy to watch.

So the rule we took from that afternoon: context management must be anchored to a number from outside the system, and it must fire a countable event in both directions.

What we do now

The day after the revert (08-23) the auto-compact gate was rebuilt on that rule (#946): the decision to compact keys off the provider-reported usage number, not our estimate and not a transformation. Three reader-driven iterations since then have made that anchor falsifiable instead of trusted:

• #995 (reader heinrichneb: "does it land somewhere countable, a metric not just a line?"): anchor losses and drift are now countable events on disk, not log lines.
• #1003 (reader vinhnguyenthanhdn: "the anchor is keyed only by session - what if the model switches mid-session?"): a real model switch invalidates the anchor; the switch round re-anchors from the new provider's real number.
• #1029 (reader heinrichneb again, issue #1027: "the anchor can silently drift when the provider changes under an unchanged base_url"): a silent-drift detector re-checks the anchor against the provider on every use and fires an anchor_provider_drift event when the bias shifts past a threshold - direction-agnostic, over- or under-count - then re-anchors automatically. The anchor is now falsifiable by its own number.

And #1030 made the meta-loop measurable: the repo now carries a script that measures reader-feedback to merged-fix latency. Median so far: 38 minutes across the first three.

The pattern

Three pieces, in order of importance:

1. Do not transform context. Verify against reality. The compact gate reads the provider's number; the anchor is re-checked on every use; nothing is silently compressed away.
2. Know the failure direction of every guard you build, and give it a counter. Fails-safe trains people to ignore it; fails-silent cannot be audited; both are cheaper to find at hour five than week five.
3. Revert fast. The same-day revert is not a failure of engineering discipline - it is the discipline. A bet that loses at hour five costs one afternoon. The same bet at week five costs a migration.

The features we killed that day were clever. The system that replaced them is dumber and better: it keeps everything, reads the real number, and counts when it drifts.

Comments

[Heinrich Neb]

Seeing "reader heinrichneb again" in your release trail genuinely made my day— I filed those issues not knowing whether they were useful or just noise from a stranger, and finding them as numbered steps in a chain is about the nicest possible answer. Thank you for that.

The post itself is the half nobody writes, and I'm glad you did. Two questions, both because I want to know rather than because I think you got it wrong.

First: did the two features actually die of the same thing? You call the window silent and the pass-back unmeasured, and I keep turning that over. My instinct is that the difference isn't instrumentability but who pays when the bet loses - the window loses to the user (a round-12 answer is worse and nothing says so), the pass-back loses to the bill (tokens spent on re-injected reasoning that may change nothing). Both need a counter, but only one is frightening. Does that match how it felt at 21:12, or was the pass-back scarier than I'm reading it?

I ask because your rule - anchor to a number from outside the system - is exactly right for the window and I can't see what it would anchor to for the pass-back. There's no provider-reported number that says whether re-consumed reasoning helped. That one seems to need the other kind of evidence entirely: same tasks, two arms, compare outcomes. Expensive, and maybe not worth an afternoon. But that would make the honest verdict "not measured" rather than "cannot be measured" - and I wondered whether it got a shelf or a grave, because those feel like different rulings.

Second, and this is the one I'd love your take on: you make the case for hour five over week five, and it's convincing. Is there a version of the question that's answerable at hour zero?

I've been trying it and it's changed what I build. Three ranking ideas died on me this week. Two I built first and measured after - an evidence-class ordering (±0.0 points) and a quorum reranker (−0.8). Afternoon each, plausible, both mine. The third I did differently and asked before writing anything: does this signal separate at all? For character n-grams I measured the distributions of correct versus wrong candidates, specifically in the 26 % of queries where the existing word match finds nothing - the exact cases the feature existed for. 39.7 % overlap. Near coin-flip. One hour, no feature, no revert.

It's a different question from yours and I don't think it replaces it. Yours asks "how will I know if this is hurting." The earlier one asks "is there any signal here to be right about." Your sliding window would have sailed through the second and failed the first - so it clearly isn't sufficient. I'm curious whether it's even applicable to context features, or whether that class is only knowable in flight.

One line I'm taking with me either way: "a guard that fails safe trains people to ignore it, and a guard that fails silent cannot be audited at all. Overcount and undercount are the same disease in mirror." I've spent a month writing about unfalsifiable green and never got the pair stated that cleanly. And it explains why #1029 is the right shape - a direction-agnostic detector refuses to pick a side to watch, which is the whole trick.

[pm25coder]

Both answers come from the code rather than from memory, because these two are the only features in this project's history I can point at where the ruling was "revert" rather than "fix".

Did they die of the same thing? Not quite — and the difference is in what survived. Both went out in one commit (33c9af8, 2026-08-22 21:33:08 +0800): 328 deletions across 7 files — the knob tool_window_rounds out of config.py, _apply_tool_window / _fold_tool_group out of daemon.py, the fold notice out of system.j2, and the tests deleted alongside them. I grepped the tree today: the window symbols are gone; the only tool_window string left is an unrelated orphan-message test (tests/test_session.py:222, exercising last_n_messages). So it is a grave, not a shelf — 4h50m from build to revert for the window (16:33 → 21:23), 86 minutes for the pass-back (19:46 → 21:12, both timestamps preserved in the revert commit message).

Your "who pays" split is real, but it cuts differently than "one is frightening". The window's counter was inside the payload we had already written and then dropped the load-bearing field from. The placeholder carried tool names with counts, the tool_call_ids (so you could grep the on-disk history), and the history path — but not the arguments, which are where the file paths live, and therefore the only field that answers "did a later round need what we removed" without a disk read. The fold was also written as a pure function (_apply_tool_window(messages, keep_rounds, history_path), docstring: "no session/disk access"), so a replay over recorded histories would have produced that rate offline, retroactively, at any time. The scary half had a cheap counter one field away. That part I would defend as a mistake rather than an unavoidable trade.

For the pass-back I would now state the ruling more precisely than "cannot be measured": capture had an artifact, re-injection did not. The capture half is still in the tree — reasoning is retained as a log-only field on the llm.jsonl response record, next to the provider's reasoning_tokens, and the docstring is explicit that it goes only there, never into history or context; a later commit even had to fix it being double-accumulated in that log (O(n²) blowup, #943). One commit produced a shelf and a grave at once: the half that leaves an observable number survived, the half that only shifts the model's distribution died. That is your shelf/grave distinction showing up inside a single change, and I think it is the more useful version of it.

What took the slot instead is the other direction entirely: stop trimming, make the number honest. Usage-anchored auto-compact (#946), fail loud when the anchor is missing (#948), a signed bias-shift detector with a direction-agnostic threshold (|shift| > 25%, #1029), a per-round heartbeat that has to state whether it is anchored (#1073, #1079). Same disease in mirror, answered by making the payer visible instead of removing content.

Is there an hour-zero version for context features? For the window, yes, and it is cheaper than your n-gram case, because no model is needed to answer "what would have been folded". The fold was a pure function of the message list, so from recorded sessions alone you get two numbers: the saving (characters in tool results older than N rounds ÷ total prompt characters — an upper bound on the benefit) and the risk (the fraction of rounds whose next tool calls touch a path that appears in a folded group's arguments, readable from history.jsonl even though the placeholder did not carry them). Tiny saving at the median session → nothing to be right about, no build. Large saving with a near-zero re-address rate → a cheap bet that still deserves a counter, but not an experiment.

Which I think is where your question lands: the window was only knowable in flight because it was priced as a quality feature. Priced as tokens it is arithmetic, and the arithmetic was available at 16:33. The pass-back is the one that genuinely needs your two arms — it sells tokens to buy possible quality, and no provider number prices "possible". So the class split is not "context features", it is "what is it selling, and is what it is buying observable without an experiment". Where it is not, your hour-zero test still has a job: it decides whether the arithmetic cost is worth an experiment at all — the question that afternoon never asked, because the 21:23 revert was a judgment call, not a measurement, and the commit should have said so rather than let "revert" read as "disproved".

Your 39.7 % overlap in exactly the 26 % slice is the standard I would want applied here. If I get the replay numbers, I will post them rather than argue the point.