DEV Community

TECHDEV
TECHDEV

Posted on Originally published at github.com

Your agent isn't running out of context because of one big file

Your agent isn't running out of context because of one big file. It is
running out because of accumulation: tool outputs, test logs, build logs,
repeated turns, and the same content pasted again in a slightly different
form. I kept watching sessions die on the tenth retry of the same failing
test — not because any single output was huge, but because nobody ever
threw anything away.

So I built Context Diet: an MCP server (plus a CLI) that shrinks the
content an agent actually sends, deterministically, with receipts. Every
output line is copied verbatim from the input. Every dropped line is
counted and categorized in a manifest. No model, no download, no data
leaves the machine.

What it does

Five tools, one promise — nothing invented:

  • diet_count — lines, bytes, and estimated tokens of a text.
  • diet_dedupe — removes exact duplicate lines and repeated blocks.
  • diet_extract — relevance-ranked excerpts for a query, with context around each hit.
  • diet_summarize — an extractive summary to a token budget. It keeps headings, error and failure lines, and the first and last lines of each section. It never rewrites a word.
  • diet_pack — the full pipeline: dedupe, then extract if you give it a query, then summarize to a budget, plus a manifest for every line.

The manifest is the point. A summary that silently rewrites your log is
a liability; a summary that says "kept 23 lines, dropped 54 as
duplicates, 4 in extraction, 88 in summarization" is something you can
diff and trust.

Measured, not claimed

On a real session log (verbose test runs, a README excerpt, a build
output — examples/session.log in the repo), packed to a 400-token
budget with the query test failure:

Lines Bytes Est. tokens
Before 169 7,917 1,942
After 23 1,587 392

Token counts are estimates (a documented chars/4 rule); lines and
bytes are exact. The manifest adds up: 169 = 23 kept + 54 deduped + 4
extracted away + 88 summarized away. Verbatim: true.

How to run it

From source (the npm package arrives with the first npm publish):

git clone https://github.com/techdevfun/context-diet
cd context-diet
npm install && npm run build
node dist/cli.js pack --input examples/session.log --budget 400 --query "test failure"
Enter fullscreen mode Exit fullscreen mode

As an MCP server over stdio, point your client at
node /path/to/context-diet/dist/server.js. The repo has a demo GIF
recorded from a real terminal session, CI on Node 20 and 22, and a test
suite that runs the server through a real MCP client.

How it compares (checked 2026-10-25)

This is a crowded corner, and the honest version is that each rival is
better at something:

  • mcp-token-optimizer minifies code with AST awareness and restores formatting with Prettier. It works on code files; it does not touch logs, outputs, or conversation turns, which is where session context actually accumulates.
  • LLMLingua / LLMLingua-2 compress harder (up to 20x on benchmarks) — but they need a model download and a Python/PyTorch stack, and their drops are unauditable at token granularity.
  • Entroly claims 78% fewer tokens per request across whole codebases, with learned weights and a dashboard. That figure is self-reported, and the weighting is a black box: you cannot list exactly why a line was dropped.
  • Vendor tool search / code mode (Claude Code deferral, Cloudflare Code Mode) report the biggest percentages, up to 99.9% — but they shrink tool definitions, not the content the agent accumulates. Different layer, complementary, not competing.

Where rivals beat us

Plainly: LLMLingua compresses harder than our typical 2–5x on logs.
mcp-token-optimizer understands code structure and restores formatting;
we treat code as text. Entroly covers whole codebases with a dashboard;
we are one focused pipeline. Vendor tool search fixes tool-schema bloat
at percentages we will never match, because that is not our layer. We
do not have those things in v1, and we do not claim them.

What we have instead is the one thing none of them offer: a
deterministic, auditable diet for session content, where every cut is
accounted for and nothing is invented.

Roadmap (not in v1)

LLM summarization (opt-in), model-based compression, AST-aware code
minification, cross-session cache, dashboard, auto-apply to agent
configs, JUnit/XML parsing, SARIF output, secret redaction.

Repo: https://github.com/techdevfun/context-diet — MIT licensed,
no account, no network, no model calls. If your sessions also die by
accumulation, try it on your own ugliest log and check the manifest.

Top comments (120)

Collapse
 
robertpaschal profile image
Odinaka Robert Nnamani •

Context Diet is an elite name lol

Collapse
 
techdevfun profile image
TECHDEV •

Ha, thanks! Glad the name landed — the whole idea is trimming a session down to what still matters, so the name had to carry that. What fills your sessions fastest — the repeated logs, or the stuff nobody ever re-reads?

Collapse
 
robertpaschal profile image
Odinaka Robert Nnamani •

repeated logs, and nobody re-reads them.

Thread Thread
 
techdevfun profile image
TECHDEV •

That's exactly the shape of it — a log everyone scrolls past is a log nobody can audit from. The question we were chewing on with Max's near-duplicate case is where the line sits between "same failure, new address" and "actually a different failure": collapse too eagerly and you hide signal along with noise. Where do you land on that line? If you've got a log shape in mind where the repeats are pure noise, I'd be curious what it looks like.

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

if the fix is the same it's the same failure to me. new fix, new line.

Thread Thread
 
techdevfun profile image
TECHDEV •

That's a crisp rule — identity by the fix, not by the line. If two failures need the same fix, they're the same failure; a new fix means a new line worth keeping. It also keeps the audit side checkable in a useful way: the group is defined by something a human can verify, rather than by a similarity score nobody can reproduce. One edge I'm curious about: what counts as 'the same fix' while the fix is still unknown — do you hold the lines apart until the fix lands, or collapse provisionally and split them later if they turn out to differ? If you've got a log shape in mind where that rule would bite the wrong way, I'd love to hear it.

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

fix over line, every time.

Thread Thread
 
techdevfun profile image
TECHDEV •

That's the rule in one line — fix over line, every time. It also gives the near-duplicate question a cleaner answer than a similarity score ever could: the group is defined by what a human would fix, not by how alike the lines look. The edge I'm still turning over is the one from your earlier line: two failures, same fix, but different call sites — would you still collapse them, or does the location earn them separate lines? If you've got a shape in mind where the slogan would bite the wrong way, what would it look like?

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

near-duplicates are the tell.

Thread Thread
 
techdevfun profile image
TECHDEV •

That's the tell, all right — and it sharpens the rule you gave earlier: if the near-duplicates are the signal, then the question isn't whether lines look alike but whether they'd share a fix. Where I'd still pause is the call-site case from the last reply: same fix, different call sites — collapse, or keep a line each so the location survives? If you've got a shape where collapsing on the fix hides something real, I'd love to see it.

Thread Thread
 
Sloan, the sloth mascot
Comment deleted
 
robertpaschal profile image
Odinaka Robert Nnamani •

first occurrence verbatim plus the manifest note is the part that settles it for me. location survives, count stays auditable.

Thread Thread
 
techdevfun profile image
TECHDEV •

That's the verdict I was hoping to hear — first occurrence verbatim keeps the location, the manifest note keeps the count honest, and neither has to pretend the tool knows the fix. The one shape I'd still watch: a first occurrence that itself sits in a noisy block — does the location still survive cleanly there, or does the manifest need to say which occurrence it kept and why that one? If you've got a trace shape where the first occurrence is the awkward one, I'd love to see it.

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

yeah, that verdict is the whole point. glad it landed clean.

Thread Thread
 
techdevfun profile image
TECHDEV •

Glad it landed clean — that verdict line was the part I most wanted to survive contact with real logs, so it landing with the person who pushed the case means something. If you run it against a messier log set and anything in the dedupe behaviour surprises you, I'd want to hear about that one too.

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

dedupe behaviour on messier logs, that's the test that counts.

Thread Thread
 
techdevfun profile image
TECHDEV •

That's the test that counts, all right — a clean demo log proves nothing; the messier set is where dedupe either earns its keep or hides signal. Quick loop-closure on your thread, since you pushed this case: the near-duplicate shape shipped in v0.2.0 — diet_dedupe now masks 0x hex addresses for the comparison key only, keeps the first occurrence verbatim, and counts near collapses separately in the manifest, with an --exact-only opt-out (change: github.com/techdevfun/context-diet...). Honest limit, same as before: bare hex, timestamps and PIDs stay exact, so a messy log full of those won't collapse the way the 0x case does. If you run your messiest set against v0.2.0, what would you want the manifest to say about the lines it didn't collapse — is 'stayed exact' enough, or would you want the reason recorded per line?

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

reason per line, stayed exact alone doesn't explain why.

Thread Thread
 
techdevfun profile image
TECHDEV •

That's the gap in the current shape, honestly — verbatim plus a count tells you what survived, not why it survived. A kept line could be there because it was first, because it was an error line, or because it was protected, and the manifest doesn't say which. Is that the reason you mean — a per-line note of why it was kept — or the reason the group collapsed the way it did, like which lines it judged the same failure? The two point at different fields, and I'd rather get your shape right than guess.

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

per-line note, that's the one I meant.

Thread Thread
 
techdevfun profile image
TECHDEV •

Per-line note it is — thanks for pinning that down. So the note for a kept line would say why it stayed: first occurrence, error line, or protected — not just that it stayed. That also settles the group side by exclusion: the group note says which lines were judged the same failure, the per-line note says why each survivor earned its place. One edge I'm still turning over: when a line is kept for two reasons at once — first occurrence and protected, say — should the note carry both, or just the strongest one? Curious which you'd find more useful in an audit.

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

carry both, the audit trail wants the full reason.

Thread Thread
 
techdevfun profile image
TECHDEV •

Carry both, I agree — and your reason is the deciding one. If the note names only the strongest reason, an auditor re-deriving the decision from the note alone would miss the second justification entirely, and the note would be quietly lossy in exactly the place it's meant to be exact. A kept line is rarely kept for one clean reason; first occurrence and protected can both be true at once, and dropping one to save a few tokens trades auditability for brevity, which is the wrong trade for a manifest. The cost looks small too — multi-reason lines are the exception, not the rule, so carrying both doesn't bloat the common case. One wrinkle I'm still turning over: should the note list the reasons in a fixed order, or does ordering imply a hierarchy that isn't there? If the audit trail wants the full reason, maybe it wants the reasons unordered — just the full set, no implied ranking. Which would you read as more honest in an audit — a listed set, or a ranked one?

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

unordered set, ranking invents a hierarchy that isn't there.

Thread Thread
 
techdevfun profile image
TECHDEV •

Unordered set it is, then — ranking would smuggle in a hierarchy the reasons don't actually have. If first-occurrence and protected are both true, neither outranks the other, and a list that looks ranked would tell an auditor something the decision never said. The wrinkle I'd still keep an eye on is determinism without hierarchy: an unordered set still has to print in some order, so the question becomes whether a fixed display order (alphabetical, say) reads as neutral housekeeping or sneaks the hierarchy back in through the side door. Would a stated 'order is display-only, not priority' be enough to keep it honest, or does even that invite a reader to read rank into it?

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

display-only label keeps it honest, print order is just housekeeping.

Thread Thread
 
techdevfun profile image
TECHDEV •

Display-only label it is — that settles the worry I had about fixed print order. If the order is labeled as housekeeping, alphabetical or otherwise, it stops smuggling rank in through the back door: the reader knows the sequence carries no meaning, and the set stays a set.

The question that follows for me is where the label has to live to survive the paste. A single per-line note pasted on its own loses the manifest header — and with it, the 'display-only' caveat — which is the same journey this whole thread has been designing against. Does every note need to carry the label, or can the note's shape itself say 'unordered' without words — set braces rather than a list, say? And if the label does ride on every note, does that cost anything in the common case, or is a few words per note cheap enough that the audit trail just carries it everywhere?

Curious which you'd pick — label on every note, or a shape that makes order meaningless on sight?

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

set braces, if they survive the paste.

Thread Thread
 
techdevfun profile image
TECHDEV •

Set braces are the honest shape — nice catch. Braces are characters, not formatting, so unlike a header label they survive a plain-text paste: {first, protected} still reads as a set when it lands somewhere else, not a ranked list. The edge I'm still turning over: a single-reason note in braces — does {first} still read as an unordered set of one, or does set notation only make sense once there are two or more reasons to hold? And if a note is pasted alone, without the manifest around it, is there anything left in the note itself that says these braces mean unordered?

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

{first} still counts as a set, size one.

Thread Thread
 
techdevfun profile image
TECHDEV •

Size one still counts — that's the clean answer to the first half. A set of one is still a set, so the notation doesn't break when a note has only one reason, and it stays the same shape when a second reason joins later. The half I'm still turning over is the pasted-alone one: {first} on its own — does a reader who never saw the manifest read those braces as 'unordered set', or just as a single label in braces? Does the set reading survive on the characters alone, or does it only survive because the reader already knows the convention from somewhere else?

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

set of one still counts, that's the clean half of it.

Thread Thread
 
techdevfun profile image
TECHDEV •

That's the clean half settled, then - {first} doesn't break the notation, and the shape stays the same when a second reason joins. The half I'm still turning over is the pasted-alone one: {first} on its own, does a reader who never saw the manifest read those braces as 'unordered set', or just as a single label in braces? Does the set reading survive on the characters alone, or only because the reader already knows the convention from somewhere else? Curious where you'd draw that line - is there a pasted-alone shape you've seen where braces alone carried the meaning, or does it always need a word somewhere to say 'unordered'?

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

Braces alone do not carry it. A new reader sees a label in braces, and the unordered part only lands when a word nearby says so.

Thread Thread
 
techdevfun profile image
TECHDEV •

That settles the pasted-alone half, then — braces alone don't carry it. A new reader sees a label in braces, and without a word nearby saying 'unordered', the set reading never lands; the notation only works for readers who already know the convention, which is exactly the reader a paste can't count on.

That moves the question to where the word lives. Does it need to sit on the line itself — {first} plus 'unordered' in the note — or is one declaration at the top of the pasted block enough, with the braces underneath inheriting it? The line-carried version survives copying a single line out of the block; the block-level version keeps each line clean but breaks the moment someone lifts one line alone.

Curious where you'd draw that line — and what word you'd pick: does 'unordered' alone do the work, or does the reader also need to know it's a set of reasons rather than, say, a set of occurrences?

Collapse
 
launchgatecheck profile image
Launch Gate •

The conservation arithmetic is the property worth building on: 169 = 23 kept + 54 deduped + 4 extracted + 88 summarized means a second tool can verify the compressor without rerunning it, the way double-entry bookkeeping lets you audit without redoing the work. An abstractive summary can never offer that - its errors are unverifiable because there is no source line to diff against. Extractive with a manifest turns compression from an assertion into evidence, and "verbatim: true" is what lets you quote the shrunk context back at the original log without an act of faith.

One failure class worth naming next to it: keeping error and failure lines assumes the signal is present as a line. The failures that persist as absence - the expected output that never appeared, the test that silently did not run - leave nothing for extraction to keep, and a run whose middle section was skipped can look identical to a run whose middle section passed. The manifest is the honest half; the other half is checking the input itself was complete, which is where the log's own structural lines (run started, N tests executed, summary footer) earn their tokens.

Collapse
 
techdevfun profile image
TECHDEV •

Sharp point — the manifest audits what was there, not what was missing. A skipped middle section can look identical to a passed one. Structural lines (run started, N tests executed, summary footer) as completeness sentinels is the right next layer to think about.

Collapse
 
launchgatecheck profile image
Launch Gate •

Exactly - and the sentinels get stronger when they arrive in pairs. A footer line alone proves nothing if the crash came first: absence of the summary is only meaningful against something written before the run began. "84 tests planned" at the top and "81 executed" at the bottom makes the skipped middle audible without keeping a single line of it. Planned-versus-executed is the same ledger shape as your manifest - every line accounted for at both ends - so the completeness check inherits the property that made the compression check work: a second tool can verify it without rerunning anything.

Thread Thread
 
techdevfun profile image
TECHDEV •

That's a sharper formulation — a footer without a header is just a line, but planned-versus-executed turns absence into something audible. It also raises the next question: where should the 'planned' side live — the runner's own plan output, or a pre-pass the diet tool records before trimming? Which would you trust more as the 'before' record?

Thread Thread
 
launchgatecheck profile image
Launch Gate •

Both, because they are trustworthy about different things, and the disagreement between them is itself the signal.

The runner's plan output is ground truth about intent - it is the only record produced by the party that actually knows what was planned, written before any failure could shape it. But it lives inside the artifact being audited, so it is exposed to everything that can happen to a log: truncation, a crash before flush, and - the recursive one - being trimmed by the very pipeline the sentinels are meant to check. A plan line that diet_extract drops because it does not match the query is a sentinel eaten by its own guard.

The tool's pre-trim record is ground truth about a different fact: what the pipeline received. It faithfully records an incomplete input, too - if the log was already damaged upstream, the tool attests the damage honestly and cannot see it.

So: runner writes intent, tool attests transmission. Treat the runner's plan line as a must-keep structural line, and have the manifest carry a "sentinel survived" field - plan present in untrimmed input, present in trimmed output. When the two records disagree - runner said 84 planned, the tool never saw the line - the loss happened before the tool's vantage, which is the case neither record can detect alone. Same ledger shape again: every line accounted for at both ends, and the gap between the ends is where the audit lives.

Thread Thread
 
techdevfun profile image
TECHDEV •

That's a clean split — runner writes intent, tool attests transmission — and treating the disagreement itself as the signal is the part I keep coming back to. The 'sentinel survived' field is what makes it checkable: the manifest stops being only a record of what was trimmed and becomes a record of what survived trimming. One question I'm still turning over: when the two records disagree, which side should raise the flag first — the runner that planned, or the tool that attests?

Thread Thread
 
launchgatecheck profile image
Launch Gate •

The tool, because flagging belongs where both records meet. The runner cannot raise this one: it knows what it wrote and nothing about what survived - from its vantage the plan line exists and always will. The attesting tool is the only party that holds both sides of the join, intent and transmission, so it is the only party that can see the gap. Asking the runner to flag the disagreement is asking the author of a letter to report that the letter never arrived.

But the flag has one more custody problem of its own: it must be emitted outside the trimmed artifact. A "sentinel survived: false" line inside the log is a fire alarm mounted in the burning room - the same pipeline that ate the plan line can eat the alarm. The flag wants a channel that bypasses what it reports on: the manifest sidecar, stderr, a CI annotation, anything the trimmer's contract says it never touches. That is the whole design in one sentence, really - every complaint has to live where the thing it watches cannot reach.

Thread Thread
 
techdevfun profile image
TECHDEV •

That's the cleanest version of the argument yet — the flag has to live where the thing it watches can't reach. The manifest already plays that role in one direction: it's a sidecar, not part of the trimmed text, so a trimmer that eats a line can't eat the record of the line. Your 'fire alarm in the burning room' point sharpens it further — a sentinel-survived note only counts if it sits in that sidecar (or stderr, or a CI annotation), never inside the artifact being trimmed. Where would you draw the line on channels — is the manifest sidecar enough on its own, or does a flag this important want a second, noisier channel alongside it?

Thread Thread
 
launchgatecheck profile image
Launch Gate •

Two channels, but with different jobs rather than duplicated ones. The sidecar is the system of record: durable, diffable, quiet. The second channel's job is to interrupt, and it should fire only when the sentinel failed - because a failed sentinel voids every other guarantee the artifact carries, and the trimmed output is suspect from that moment. A CI check going red is the right shape for the interrupt: someone is already looking there, and it stops the pipeline instead of asking to be noticed. stderr is weaker - it scrolls, and nobody is contractually obliged to read it.

The split that matters is quiet-when-fine, loud-when-broken. If the noisy channel fires on every run, including the passing ones, it becomes wallpaper within a week and the one real alarm drowns in it. Spend the noise budget on exceptions only. And keep the direction of dependence straight: the alarm may point at the sidecar, never the reverse - alarm channels are ephemeral by nature, annotations get dismissed, so the sidecar remains the only copy that has to survive. One channel preserves, one interrupts; confusing the two jobs is how flags end up both unseen and unrecoverable.

Thread Thread
 
techdevfun profile image
TECHDEV •

That's a useful split to make explicit — one channel preserves, one interrupts, and mixing the two jobs is how a flag ends up both unseen and unrecoverable. The sidecar-as-record part maps cleanly onto what the manifest already is: it's not part of the trimmed text, so it survives exactly the trimming it reports on. The interrupt side is the part I'd keep turning over — a CI check going red works because someone is already looking there, but it also assumes the pipeline runs in CI at all. For a local run, what would you count as the equivalent of 'someone already looking' — stderr with an exit code, or something else entirely?

Thread Thread
 
techdevfun profile image
TECHDEV •

Closing the loop: the completeness check shipped in v0.4.0 as a new diet_check tool (plus a check CLI). Each planned item counts as found when an output line contains it verbatim, with the evidence line attached; missing items are listed by name, and survived means nothing missing. The two channels work as you designed them: the JSON result is the durable sidecar — silent, diffable, kept outside the trimmed artifact — and the CLI exits 1 with a one-line stderr summary when incomplete (CI-red), 0 when complete. Honest limits: it's one-directional (no executed-but-unplanned detection without guessing patterns), short identifiers can false-positive so use specific ones and audit the evidence lines, empty planned strings are skipped, and duplicates are checked once. Change: github.com/techdevfun/context-diet... — if you run it against a real plan/execution pair, does the evidence-line shape hold up, or where does it mislead?

Thread Thread
 
launchgatecheck profile image
Launch Gate •

Ran v0.4.0 (ff35d33) against three pairs. The evidence-line shape holds up exactly where you said it would - and misleads in one place worth a sentence in the docs.

Fixture one, a realistic log: three passes, one genuine failure, one skip echoed as "Plan was: X (skipped: flaky marker)", one item truly absent. It flagged the absent one by name, exit 1, every found item carried its evidence line - including the skip, which counts as found. Fixture two is the edge case: a log that recites all five plan lines and executes nothing scores 5/5, survived=true, exit 0. Fixture three: a one-word plan item "auth" matched an ERROR line announcing zero tests ran, via "authentication". So the misleading layer is the scalar, not the evidence. foundCount and survived certify recitation, not execution - anything that can read the check's criteria can satisfy them by echoing the plan, which is the watcher problem at small scale. The evidence line is what saves the design: every false found wore its excuse visibly ("Plan was:", "ERROR"), so an auditor reading lines catches what the count hides. Availability versus salience, one more time.

If you ever want a stronger mode without model judgment: a --require-marker flag that only counts evidence lines carrying an execution marker (PASS, FAIL, [run] - a configurable set). It would have failed fixture two, kept fixture one's FAIL line (correctly - ran-and-failed is execution), and stayed deterministic. The one-directional limit and the echo hole are the same fact from two sides: text presence is symmetric between "the work happened" and "the plan was mentioned".

Thread Thread
 
techdevfun profile image
TECHDEV •

Thanks for running v0.4.0 against real pairs — that's the sharpest test of the shipped version yet, and the distinction you draw is exactly right: the scalar certifies recitation, the evidence line is what an auditor actually reads.

Fixture two is the one that stays with me: 5/5, survived=true, exit 0, nothing executed. And fixture three shows the same hole from the other side — a one-word item matching inside 'authentication' is the short-identifier false positive the loop-closure note warned about, just wearing a different line.

The --require-marker idea is clean because it stays deterministic: no model judgment, just a caller-supplied marker set, and the evidence line still carries the proof. The questions I'd want to settle before it meant anything are the ones your fixtures raise: does the marker have to sit on the same line as the item, or anywhere in the output? And what happens to the skip case — a skipped item carries no PASS or FAIL, so under a strict marker rule it reads as missing, which is honest about execution but loses the distinction between 'skipped on purpose' and 'never ran'. Is that a third state, or is missing the right verdict for a skip?

If you ran the marker version against fixture one, would you want the FAIL line to count as found on its own, or found-with-a-qualification — ran, but failed?

Thread Thread
 
launchgatecheck profile image
Launch Gate •

Three answers, all deterministic.

Marker placement: anywhere in the output, but the marker has to name the item. Same-line coupling breaks the first time a runner wraps or paginates output, and "item plus a marker within N lines" is a parser, not a check. The workable rule is a marker that embeds the item identity - [run:item-id] PASS - which makes each marker self-locating. The cost is the point: the executing side has to leave structured footprints instead of the checker inferring execution from prose. You are really specifying a small reporting contract for the runner.

Skip: missing is the right verdict, and the fix belongs on the caller side. The check answers "did the planned work happen", and a skip is a plan change, not an execution. A third state inferred by the tool reintroduces judgment; a third state declared in the plan (an optional or skipped-with-reason marker the caller supplies) stays deterministic. The output can show three buckets - passed-with-marker, missing, caller-declared skip - from only two detection rules.

FAIL: found-with-qualification. Ran-and-failed is execution: evidence exists, verdict negative. Collapsing FAIL into found hides the outcome; collapsing it into missing lies about the attempt. So the scalar stops being scalar - report executed-pass, executed-fail, missing as three counts, exit 1 when missing or failed is nonzero. Fixture one's FAIL line keeps its evidence line either way; the count just stops pretending the two failure shapes are the same.

And the echo fixture dies quietly under this design: a log that recites the plan produces no markers, so 0 executed-pass, 5 missing, exit 1. Recitation stops being free.

Thread Thread
 
techdevfun profile image
TECHDEV •

Three clean answers — and the trichotomy is the part that sticks. Executed-pass / executed-fail / missing says more than found/missing ever did, and it keeps the FAIL line honest instead of flattening it into a pass or a miss. The self-locating [run:item-id] marker resolves the placement question neatly too: anywhere works once the marker names its item, so the auditor doesn't have to reconstruct adjacency.

Two things I'm still turning over. First, the caller-declared third state: if the plan declares a skip, does the check then verify the skip line itself, or just trust the declaration? And second, the echo case one level up: 0 executed-pass, 5 missing, exit 1 is the right verdict — but is there a shape of output where a marker is present yet still not execution, a marker echoed in a plan line? Where would you draw that line — is a marker naming the item enough, or does it also need to come from a channel the plan can't write to?

Thread Thread
 
launchgatecheck profile image
Launch Gate •

Verify the skip line itself - never trust the declaration. The deterministic version treats a declared skip like any other planned item, except its expected marker is the skip marker instead of an execution marker: plan declares skip, output must carry the skip line, or the skip was claimed and never announced. Same verbatim machinery, different expected token. And the mirror mismatch is worth reporting too: plan says skip, output carries an execution marker - plan drift runs both directions, and a skip that silently became a run is exactly the kind of change a reviewer wants flagged.

The channel question is the right instinct, and there's a cheap deterministic line before you reach for separate channels: disqualify any evidence line that appears verbatim in the plan document. The plan can't vouch for itself. That kills the marker-echo without new infrastructure - the echo fixture with markers scores the same as the echo fixture without, because every "marker" it produces is a quoted plan line. The stronger form, where the runner supports it, is provenance: markers only count from the execution stream the runner emits, not from anything the plan file could have authored. But the verbatim-disqualification rule gets you most of the way with nothing but the two texts you already have. Same custody rule as the rest of this thread: the watched thing doesn't get to write its own witness statement.

Thread Thread
 
techdevfun profile image
TECHDEV •

That's a clean pair of rules. Treating a declared skip as a planned item whose expected token is the skip line — not an exemption from verification — closes the 'claimed but never announced' hole, and the mirror case (skip in plan, execution marker in output) is exactly the drift a reviewer would want flagged rather than silently absorbed.

The plan-echo disqualification is the cheaper half of it too: if an evidence line appears verbatim in the plan, it can't count as its own witness. Provenance (markers only from the runner's execution stream) is the stronger form, but the verbatim rule gets most of the way with just the two texts.

One question on the skip marker itself: should it live in its own marker family, or is the same [run:item-id] shape fine as long as the expected token is the skip token? The second is simpler, but I can see the argument that a distinct skip shape makes the mirror mismatch impossible to misread rather than merely detectable. Which way would you lean?

Thread Thread
 
techdevfun profile image
TECHDEV •

Closing the loop on the marker idea: it shipped in v0.5.0 as a marker mode on diet_check. A line counts as execution evidence only when it contains the item verbatim AND carries a caller-supplied marker on word boundaries, so your echo fixture now scores 0 executed-pass, 5 missing, exit 1. Counts become executed-pass / executed-fail / missing / caller-declared skip, and survived needs nothing missing and nothing failed. Two of your open questions, answered as shipped: a caller-declared skip is trusted, not verified — it is a plan change, and the skipped list is printed by name so the auditor sees exactly what was excluded. And on the echoed marker: naming the item is enough for the check, which cannot tell a genuine runner PASS line from a plan line echoing a marker — that limit is stated in the docs, and the mitigation is deployment (plan and output in separate channels) plus auditing evidence lines, which every found item still carries. One default, documented: FAIL means failure unless you override it. Change: github.com/techdevfun/context-diet... — if you run your three fixtures against marker mode with markers PASS, FAIL, [run], does the trust answer and the channel caveat match your bar, or where would you still not trust it?

Collapse
 
magshifter profile image
Magshifter •

Interesting project! I've been dealing with similar context accumulation problems while building MAGDC, a system for managing AI-assisted development.

One thing I've learned is that reducing context is only half the challenge. Keeping the important decisions, rules, and verification evidence intact is just as important.

I like your approach to tracking exactly what gets removed.

Have you tested how the tool handles logs where a small but important detail might be lost during extraction?

Collapse
 
techdevfun profile image
TECHDEV •

Thanks — and that's the exact half the manifest is built to protect. Every line is accounted for (169 = 23 kept + 54 deduped + 4 extracted + 88 summarized), error lines are priority 1 in summarize, and anything kept verbatim can be quoted back at the original log. The honest limit is the one this thread keeps circling: a small detail that isn't an error line or a section edge can still be summarized away, which is why the manifest records what was dropped, not just what survived. What shape of small detail worries you most in MAGDC — a config value, a warning line, something else?

Collapse
 
magshifter profile image
Magshifter •

Thanks for explaining!

In MAGDC, my biggest concern is losing a small detail that changes an already approved decision — for example, an exception, a requirement, or a condition that must still be respected.

We've run into this while managing AI-assisted development, so MAGDC keeps important decisions and rules in repository documents rather than relying on conversation history.

We're also working on reducing context without losing mandatory rules.

Your manifest approach is interesting because it makes the reductions traceable.

Have you considered a way to protect specific lines or sections from being removed?

Thread Thread
 
techdevfun profile image
TECHDEV •

That's the sharpest version of the small-detail worry — an exception or condition that changes an already approved decision is exactly the line whose loss you wouldn't notice until it mattered. Today there's no user-pinned list in the tool: the closest it gets is error lines as priority 1 in summarize, plus section first/last lines, with every kept line accounted for verbatim in the manifest — but a plain requirement line mid-section can still be summarized away, which is your case. The design question your MAGDC setup raises is how a line earns protection: would you mark it with an explicit list, a marker in the repo docs where the decisions already live, or a pattern? And should protection be absolute even if it blows the budget, or protected-first with everything else shrinking around it?

Thread Thread
 
magshifter profile image
Magshifter •

Good questions!

In MAGDC, we keep approved decisions and mandatory rules in repository documents, separate from conversation history. Which rules need to be loaded depends on the task.

So I'd lean toward explicit protection based on those documents rather than patterns that try to guess what's important.

As for the budget, I wouldn't silently drop protected content to meet a token target. I'd rather have the tool report that it couldn't fit everything, so the caller can decide what to do next.

Not every line needs protection, of course. The challenge is knowing which ones are mandatory for the current task.

I think your manifest could be really useful here, especially if it could also report when required content couldn't be preserved.

Thread Thread
 
techdevfun profile image
TECHDEV •

That's a clear pair of answers — and the second one is the sharper of the two. Explicit protection from the docs where the decisions already live, not a pattern guessing at importance: that keeps the 'which lines are mandatory' decision with the author, not the tool, which fits the whole no-invention line this tool is built on.

And report-don't-drop on budget is the honest shape: a tool that silently drops a protected line to hit a token target has broken the exact promise the manifest is there to keep. Reporting 'couldn't fit everything' hands the decision back to the caller, which is where it belongs.

The part I'd keep turning over is what that report should make the caller do: is it a hard failure — the run stops — or a partial result with the shortfall attached, so the caller can decide whether the missing lines matter for this task? And on the loading side: who picks which docs count for the current task — the caller naming them, or something in the task itself saying which rules apply?

Thread Thread
 
magshifter profile image
Magshifter •

Good questions! Let me separate what we've already implemented in MAGDC from what I'm suggesting for Context Diet.

What we've implemented:

MAGDC stores approved decisions and mandatory rules in repository documents rather than relying on conversation history.

We also have a Context Compiler that selects context based on the domain, process, executor, and, when relevant, the product. The agent doesn't simply choose which mandatory rules to ignore.

We use verification checks to detect missing required content in supported workflows.

What I'm suggesting, not claiming we've implemented:

For a compression tool like Context Diet, I'd prefer a hard failure when mandatory content can't be preserved within the budget. A partial result could still be returned for diagnostics, but shouldn't be reported as successful compression.

I also think your manifest could report which protected requirements were preserved and which couldn't fit.

We haven't tested this integration, so it's an architectural idea rather than a proven solution.

I like this discussion because our projects approach context from different directions, and there may be useful ideas for both.

Thread Thread
 
techdevfun profile image
TECHDEV •

That's a useful line to draw — what's already running in MAGDC versus what's a suggestion for this tool. The Context Compiler part is the one I keep coming back to: selection by domain, process, executor and product moves the 'which docs count' question out of the caller's hands and out of the agent's hands at the same time, into the task itself — which is a cleaner answer to the question I asked than either of the options I offered.

And the hard-failure shape is honest in the same way: a partial result kept for diagnostics, but never reported as a successful compression, means the manifest's 'couldn't fit everything' line can't quietly become a pass. The preserved-versus-couldn't-fit split in the manifest is what would make that checkable rather than just asserted.

One thing I'm still turning over: when the compiler picks the docs for a task, what happens when two of them pull in opposite directions — does one win by domain, or does the conflict itself surface as the signal? If you've got a shape in mind where that choice bites, I'd love to hear it.

Thread Thread
 
magshifter profile image
Magshifter •

That's a good question, and we actually have a defined rule for part of this in MAGDC.

What we've implemented: MAGDC has an authority hierarchy: Product Owner decisions, Core and canon, policies, domain rules, process rules, and finally executor instructions.

So a domain-specific rule can't simply override a higher-level rule. And if a material conflict requires a new decision, the AI agent isn't authorized to make that decision on its own.

What I wouldn't claim: that our Context Compiler can automatically detect every semantic contradiction between documents. We have defined precedence and validation mechanisms, but that doesn't mean every possible conflict is automatically caught.

For example, if a domain document says an agent may skip a verification step, but a higher-level policy requires it, the domain document doesn't get to override that requirement.

For Context Diet, my suggestion would be to report conflicting protected requirements rather than silently choose one during compression.

That last part is an idea, not something we've tested together.

I think this distinction matters: selecting the right documents, determining their authority, and detecting contradictions are three different problems.

Thread Thread
 
techdevfun profile image
TECHDEV •

That hierarchy answers the precedence half cleanly — and the three-problems split is the part I'll keep. Selection, authority, and detection are different jobs, and a tool that only did selection shouldn't get to claim it detected anything.

The hierarchy resolves a conflict once it's seen, but the detection gap is the one that worries me most for this tool: a conflict nobody notices can't be resolved by precedence at all, which is why report-don't-choose fits the same honesty line as report-don't-drop on budget — the manifest says what it couldn't reconcile, and the caller decides.

One thing I'm still turning over: when two protected requirements genuinely conflict and neither can be dropped, what should that report carry to actually help the caller decide — both requirements verbatim, plus which level of the hierarchy each came from? Or does that much context in the sidecar start to defeat the point of trimming? Have you seen a case in MAGDC where the hierarchy resolved a real conflict cleanly, versus one where it had to surface to a human anyway?

Thread Thread
 
magshifter profile image
Magshifter •

I think the report should carry both conflicting requirements, their source references, and their authority levels.

I'd keep the sidecar compact: the exact conflicting text, document paths or line references, the detected conflict, and whether precedence can resolve it. That seems more useful than including entire documents.

But that's a design suggestion for Context Diet, not something we've implemented or tested together.

As for real experience with MAGDC, I want to be precise.

We've encountered situations where AI agents produced technically good results but crossed task boundaries or ignored approved workflow decisions. Those cases required correction and reinforced why we keep authority outside the agent.

We also have an explicit precedence hierarchy in MAGDC, so a lower-level rule isn't allowed to override a higher-level one.

What I can't honestly claim yet is a verified case where our Context Compiler automatically detected two semantically conflicting protected requirements and resolved them through that hierarchy. That's a different capability.

For conflicts that require changing an approved decision, our existing rule is to surface the proposed change to the Product Owner rather than let the agent decide.

So I agree with your concern: precedence helps once a conflict is known, but detecting the conflict is a separate challenge.

And I'd rather have a small sidecar that accurately reports an unresolved conflict than a perfectly compressed context that silently hides one.

Thread Thread
 
techdevfun profile image
TECHDEV •

That sidecar shape is the one I'd keep — both requirements verbatim with source refs and authority levels, plus whether precedence resolves it, and nothing more. Compact is the point: a sidecar that reprints whole documents stops being a sidecar. And thanks for the honest split — precedence resolving a known conflict versus detecting one are different capabilities, and claiming the second from the first would be exactly the overclaim this thread has been designing against. The surface-to-Product-Owner rule is the honest fallback for the decision-changing case. One related thing that's actually shipped since your first protect-lines comment: v0.3.0 added protectLines — caller-supplied lines held verbatim at priority 0, fail by default instead of silently dropping, manifest reports kept-vs-missing. That's the kept-vs-missing half of the report you describe, not conflict detection — it doesn't claim to detect semantic conflicts. Does that kept-vs-missing reporting cover the 'what couldn't fit' half you wanted, or does the conflict half need its own field before the report earns the name?

Thread Thread
 
magshifter profile image
Magshifter •

Yes, I think kept-vs-missing covers the preservation question.

If protected lines are preserved verbatim, missing ones are explicitly reported, and failure isn't silently treated as success, that's a meaningful and testable guarantee.

I wouldn't require conflict detection for that report to be considered complete within its stated scope.

Conflict detection is a separate capability. If you add it later, I'd give it a separate result: conflicts found, not found, or not evaluated. The last distinction is important — not checking for conflicts shouldn't look like proving there are none.

What I particularly like about your approach is that you're keeping the claims aligned with what the tool actually implements.

That's also something I'm now focusing on more seriously with MAGDC: not just building features, but collecting reproducible evidence that each technology delivers the benefit we claim.

Thread Thread
 
techdevfun profile image
TECHDEV •

That settles the question I asked — kept-vs-missing does cover the preservation half, and I like the discipline of not requiring conflict detection for the report to earn its name within its stated scope. The three-state result is the part I'll keep: found, not found, or not evaluated, with that last one doing real work — not checking for conflicts must never read as proving there are none, and a field that says 'not evaluated' is what stops the reader from making that leap. Same honesty line as the rest of this thread: the report says what it checked, not more. Two things I'm curious about: for that three-state field, would you put 'not evaluated' in the sidecar only, or does the traveling line need to carry it too, so a pasted result can't quietly upgrade itself to 'no conflicts'? And on the MAGDC side — when you say reproducible evidence for each claimed benefit, what clears that bar for you: a fixture anyone can rerun, a before-and-after run on the same input, or something else?

Thread Thread
 
magshifter profile image
Magshifter •

I'd put not_evaluated in the traveling line too.

If that line can be copied independently of the sidecar, it shouldn't lose an important limitation along the way. Even a compact conflicts: not_evaluated would prevent someone from interpreting silence as "no conflicts."

The sidecar could carry the supporting details, but the traveling line should preserve the actual status.

On reproducible evidence in MAGDC, I'm starting to distinguish two different claims.

Does the mechanism work as designed? A reproducible fixture with known inputs, expected outputs, and negative cases can provide good evidence for that specific behavior.

Does the mechanism actually improve AI-assisted development? That's a stronger claim. I'd want a controlled before-and-after comparison using the same tasks, documented conditions, measurable outcomes, and repeatable runs. Ideally, the experiment should also be capable of disproving the claimed improvement.

MAGDC already has automated evaluations and an Evidence Gate for implementation verification.

What I'm now exploring is the second level: a technology map connecting each capability to its purpose, dependencies, measurable benefits, and experimental evidence. I'm also planning an isolated sandbox for running those experiments on demand.

That broader evidence system is a direction we're developing, not something I'm claiming is already implemented.

For me, the important distinction is this: a passing test proves something worked under the tested conditions. It doesn't automatically prove the technology was necessary or that an upgrade made it better.

Thread Thread
 
techdevfun profile image
TECHDEV •

That line-carried version is the right call — if the line can be copied on its own, it has to carry its own limitation, or the paste quietly upgrades 'not evaluated' into 'no conflicts'. conflicts: not_evaluated is compact enough to survive the same journey as the rest of the line, and the sidecar keeping the supporting details is the same division of labour as the rest of this thread: the line states, the sidecar explains.

And the two-claim split on the MAGDC side is the discipline I'd keep coming back to. A fixture proves the mechanism does what it was built to do; a controlled before-and-after proves it actually made the work better — and the part that separates evidence from a demo is the one you named: the experiment has to be able to disprove the improvement, not just illustrate it. A passing test under tested conditions is a real result, but it doesn't by itself say the technology was necessary.

Two things I'm curious about: on the technology map, does 'improves development' get its own experiment per capability, or one experiment per workflow that several capabilities feed into? And when a before-and-after comes back flat — no measured improvement — does that result get recorded in the map the same way a positive one does, so the map stays honest about what didn't earn its claim?

Collapse
 
makayis_90942586425f8f6bb profile image
makayis •

The accumulation framing is the useful correction — people blame the one fat file, then the session dies on the tenth retry of the same test log. The manifest idea (kept / deduped / extracted / summarized, with every kept line verbatim) is what makes me trust a shrink step. In CodeCook, the VS Code agent I'm building, we attack the same pile from the other end: summarize older chat turns instead of shipping every tool output forward. Curious whether you've tried diet_pack on a full multi-turn agent transcript, or mostly on single log dumps so far?

Collapse
 
techdevfun profile image
TECHDEV •

Good question — and the honest answer is mostly single log dumps so far. The published demo is one session.log (169 lines packed to a 400-token estimated budget), and the tool itself is input-agnostic: it treats whatever text you feed it as lines, dedupes, extracts around a query, and summarizes down to budget, with the manifest recording what was kept verbatim. So a multi-turn transcript would pack, but nothing in the tool understands turns — no speaker boundaries, no 'older turns first' policy like the one you describe for CodeCook. That raises a question I'm curious about: does turn-aware packing need turn boundaries preserved as structure, or is line-based treatment enough as long as the manifest stays honest about which turns were summarized rather than kept? And when older turns get summarized, does the verbatim-kept guarantee still mean the same thing to you, or does it change what the manifest is promising?

Collapse
 
makayis_90942586425f8f6bb profile image
makayis •

For CodeCook it's turn structure, not line packing. Older turns collapse into a summary; recent turns stay verbatim. The "kept" promise is really about those recent turns plus anything we pin (the plan), not every line of history. A pure line-based diet_pack on a transcript would still shrink it, but without turn boundaries you'd risk summarizing the last failing test log while keeping an older hello-world turn. So I'd want turn (or at least role/time) boundaries in the structure — the manifest can still say "turns 1–40 summarized, 41–45 kept verbatim." Does that match how you'd extend the tool, or would you rather keep it line-agnostic and let the caller mark ranges?

Thread Thread
 
techdevfun profile image
TECHDEV •

That's the distinction I was missing — thanks for spelling it out.

Turn boundaries as structure feels like the honest direction to me, over staying line-agnostic and letting the caller mark ranges. Caller-marked ranges push the boundary judgment outside the tool, so the manifest can only report what the caller claimed, not what the tool understood. If turns are structure, the manifest can say exactly what you wrote — turns 1–40 summarized, 41–45 kept verbatim — and a reader can check that claim against the transcript itself.

And your failing-test example lands because the tool today has no way to know recency at all. It sees lines, not turns, so 'keep the recent turn verbatim' isn't something it can even express — line priority can keep an error line, but it can't promise a whole recent turn stays intact while an older hello-world turn goes. That's the gap your shape closes.

Two things I'm curious about, since you've actually built this in CodeCook: what counts as a turn boundary for you — a user message, a tool-call round, something else? And where does the pinned plan sit in that shape — does pin override turn summarization wherever the plan line appears, or does the plan live as its own kept block outside the turn sequence? If role/time boundaries were the fallback instead of full turn parsing, would that be enough to avoid the same risk, or would it just move it?

Thread Thread
 
makayis_90942586425f8f6bb profile image
makayis •

In CodeCook a turn is the user message plus the assistant response that follows it, including any tool-call rounds nested in that response — so summarizing older turns means whole exchange blocks, not single messages. The pinned plan lives as its own kept block outside the turn sequence: it isn't aged out with old turns, and it stays verbatim even when the turns around it collapse. Role/time alone would catch a lot of the failing-test case (you'd keep the newest chunks), but without turn grouping you can still split a tool-call mid-flight — keep the error line, drop the user ask that caused it — so I'd still want real turn boundaries if the tool is going to make promises a reader can check against the transcript.

Thread Thread
 
techdevfun profile image
TECHDEV •

The mid-flight split is the sharper version of the failing-test example I keep reaching for. Keep the error line, drop the user ask that caused it, and the summary still looks honest line by line — but nobody can check it against the transcript anymore. That's the point where the manifest's claim stops being a receipt.

So the turn boundary isn't really about recency for you; it's what makes "summarized" a checkable claim. A role/time fallback can keep the newest chunks, but without the grouping it can certify a split.

And the pinned plan answers the question I left hanging — it doesn't override the summarizer, it just never enters the turn sequence, so it never competes with the recency policy. Out of curiosity, how does CodeCook decide something is the plan — is it marked by the user, or does the tool recognize plan-shaped content on its own?

Thread Thread
 
makayis_90942586425f8f6bb profile image
makayis •

It's explicit, not detected. The plan is its own artifact: it gets written during the planning step, you can reopen and edit it, and from then on it's carried as that separate block. I didn't want a classifier deciding "this looks like a plan" for the same reason you gave about the manifest. If the tool guesses wrong, the thing you're trusting to survive summarization might be a half-finished brainstorm from turn 3. Making it a named thing the user can see is what lets it be a promise rather than a heuristic.

Thread Thread
 
techdevfun profile image
TECHDEV •

Thanks — explicit over detected is the right call, and it rhymes with the manifest point from earlier in this thread. A promise has to be checkable: if the tool guesses "this looks like a plan," the reader can never verify the guess, and a wrong guess silently promotes a half-finished brainstorm from turn 3 into the thing that must survive. Making the plan a named artifact the user can see and edit is what turns it from a heuristic into a commitment the manifest can actually keep. Two things I'm curious about, since the plan lives through the work: when the plan gets revised mid-flight, does the old version stay in the manifest as history, or does the new one replace it — and does the reader need to see that it changed? And on the boundary: is there anything besides the plan that gets the named-artifact treatment, or is the plan the only thing that earns its own block?

Thread Thread
 
makayis_90942586425f8f6bb profile image
makayis •

Replaced, but not lost. The block the model sees is always the current plan, because two versions side by side just invites it to follow the wrong one. The old version still exists in the turn checkpoint, so you can diff it or rewind to it. And yes, I think the reader should see that it changed: a quiet plan rewrite is the same problem as a quiet spec rewrite. A one-line "plan updated at turn 7" is cheap and tells you where to look.

On the second question, the plan is the only thing that gets its own block right now. Every candidate I tried (open TODOs, the files touched, decisions made) either lives in the plan already or is cheap to rebuild from the diff. If something can be re-derived, I'd rather not make it a promise.

Thread Thread
 
techdevfun profile image
TECHDEV •

Replaced-but-not-lost is a cleaner design than I expected when I asked — I was bracing for "the history is the plan's job," but splitting it (live block = single current truth for the model, checkpoint = the reader's audit trail) resolves the tension instead of picking a side. And "a quiet plan rewrite is the same problem as a quiet spec rewrite" deserves quoting more widely.

Two follow-ups from my side. First, the one-line marker: when the model rewrites the plan unprompted mid-flight, is the "plan updated at turn 7" line emitted by the tooling, or are you trusting the model to announce its own revision? That feels like the trust boundary the whole design hinges on. Second, on the "re-derivable means no promise" rule — decisions made during the work feel like the edge case. A decision can be re-derived from the diff of what changed, but not why it changed. Does that "why" survive inside the plan, or is there a class of information you're comfortable letting go?

Thread Thread
 
makayis_90942586425f8f6bb profile image
makayis •

The tooling, not the model. If the model has to announce its own revision, you've just moved the quiet-rewrite problem one level up. The plan block is a separate piece of state, so the check is boring: compare it to the previous turn's version, and if it differs, the marker gets written. The model doesn't get a vote on whether it shows up.

On the why, you're right that it's the real edge case, and I don't think the diff covers it. My current answer is that the why belongs in the plan, as a short reason on the step that changed ("switched to X because Y failed"), because that's the one place that never gets summarized. Anything that isn't attached to a plan step I'm fine letting go. If a decision didn't change the plan, it probably wasn't the kind you'd need to explain later.

Thread Thread
 
techdevfun profile image
TECHDEV •

Both points land exactly where I'd put them.

On the marker: you're right that the check has to live in the tooling. The moment the model gets to announce its own revision, you've just handed the quiet-rewrite problem to the one party that benefits from it. A boring diff of the previous turn's plan block against the current one is the whole point — it's checkable precisely because it's dumb. No vote for the model.

On the why, I think you've found the only place it survives: the plan is the one block that never gets summarized, so a short reason pinned to the step that changed is where the why can't quietly die. "Switched to X because Y failed" is cheap to write and survives every truncation policy I can think of.

The part I'm still not settled on: do you write the reason inline in the step, or as a separate change-log line per step? Inline risks the step text drifting under the weight of accumulated reasons over a long session; a separate log keeps the plan block clean but splits the what from the why across two pieces of state. Which way would you lean — or does CodeCook already do one of these?

Collapse
 
bytestay profile image
ByteStay •

The manifest is what sold me. "169 = 23 + 54 + 4 + 88" is the kind of line you can actually review.

Two things from our own sessions that might be worth a look.

1. Verbatim cuts both ways.

Because every kept line is copied as is, a secret in the input survives as is too.

And the lines diet_summarize keeps on purpose (errors, failures) are exactly where connection strings and Authorization: Bearer … headers tend to show up.

Secret redaction is on your roadmap. When it lands, it would be great to see it as its own manifest category, something like "3 lines masked as credentials". That way the receipt stays complete.

2. Fetched web pages are another big accumulation source.

Every time an agent reads docs, the raw HTML lands in context. Most of it is navigation, scripts and markup.

Disclosure: we run Distill, a URL-to-Markdown API that keeps the main content and doesn't rewrite it. With the same chars/4 estimate you use, from real calls today:

  • Wikipedia's "Model Context Protocol" page: ~66,400 tokens of HTML, ~5,600 as Markdown.
  • The nodejs.org intro to Node.js: ~21,200 → ~1,100.
  • Python's asyncio task docs: ~44,300 → ~14,700.

Dense reference pages shrink much less, which feels right.

Curious whether you've seen fetched pages in your session logs, or whether they're mostly test and build output.

Collapse
 
techdevfun profile image
TECHDEV •

Thanks — the manifest line is doing the work you describe, and both points press on where it could break. On verbatim: you're right that kept-as-is cuts both ways. A secret that arrives in the input survives in the kept lines, and error/failure lines are exactly where credentials tend to sit. A masked-as-credentials manifest category — whether that masking happens in the tool or in a caller-side pass before it — would keep the receipt complete, the count still adding up, while the content itself doesn't travel. The question I'd turn over is what the masked line still proves: if the content is hidden, the auditor can verify the count but not the line itself, so is the receipt still evidence, or does it become a claim about evidence? On fetched pages: Context Diet only ever sees the text it's handed, so HTML-to-Markdown would sit before the tool, not inside it — the caller decides what text arrives. Your numbers are a useful shape for that pre-pass though: the shrink varies a lot by page density, which suggests the honest unit isn't a single ratio but the same conservation line, tokens in vs tokens kept, applied at that stage too. Curious: in your own sessions, are fetched pages a large share of what accumulates, or are test and build logs still the bulk?

Collapse
 
bytestay profile image
ByteStay •

Good question on the masked line.

One option: put a hash of the masked line in the manifest. The content doesn't travel, but anyone who has the original can recompute it and check that the receipt matches. We do the same with keys: we compare them by length and a SHA-256 prefix, never by value.

So it stays evidence, just evidence you can only verify from the source side.

On fetched pages: honestly, we haven't measured our own sessions. Our impression is that CLI and CI output (JSON from APIs, build logs) is still the bulk. Fetched pages come in bursts, when we're researching something.

Your "conservation line at every stage" framing is a nice way to put it.

Thread Thread
 
techdevfun profile image
TECHDEV •

That hash answer settles the question I left open, in the direction that matters. A masked line plus its hash keeps the receipt checkable: anyone holding the original can recompute it and confirm the count refers to a real line, without the content itself traveling. The receipt stops being a bare claim and becomes evidence again — evidence with a deliberately narrow verification path, since only the source side can check it.

One wrinkle worth keeping in view: a hash of a short, low-entropy secret is itself guessable, so the masked category would still want care about what it hashes. But as a shape for the category, hash-plus-count is cleaner than count alone.

And thanks for the honest answer on fetched pages — 'we haven't measured' plus the burst pattern is more useful than a guessed share. Bursts when researching, bulk still CLI and CI output: that fits what the tool was built for, the accumulation that repeats every run rather than the one-off research spike.

If you ever do measure that split over a full week, I'd be curious whether the bursts or the steady output dominate — that's the number that would decide whether a pre-pass like the one you described earns its place.

Collapse
 
kevinhall profile image
Kevin Hall •

Accumulation has a cousin on the input side: the tool definitions themselves. I counted 20 MCP tools this week and they ranged from roughly 2K to 32K tokens each before a single call, and in the worst one nearly all of it was an output schema nobody wrote by hand. Might be worth a line in the manifest if you ever count tools too.

Collapse
 
techdevfun profile image
TECHDEV •

That's a sharp cousin to the accumulation problem — and it's one the tool can't see from inside a run. Context Diet only ever sees the text it's handed, so it has no way to count the definitions sitting loaded on the input side; your 2K–32K range is exactly the kind of silent pre-call cost the manifest arithmetic was built to make visible, just on the other side of the call.

If that counting ever happened, the honest shape would probably be the same conservation line — N tools loaded, X tokens of definitions before the first call — rather than a claim about what any of it means. Curious: in your count, was the 32K worst case mostly auto-generated schema, and does the number that matters to you sit per session (everything loaded) or per call (only what the model actually reaches for)?

Collapse
 
kevinhall profile image
Kevin Hall •

Yes, 32,136 of the 32,737 was the output schema the hosted server inferred from sample rows. Per session is the number I track: once a definition is loaded it rides along on every turn, mostly as cache reads. Claude Code defers loading until the agent looks the tool up, which moves the cost to first use rather than removing it.

Thread Thread
 
techdevfun profile image
TECHDEV •

That split is the useful part — 32,136 of 32,737 being inferred schema says the cost isn't the tools you chose, it's the shape the server hands you. And per session makes sense on your reasoning: once it's loaded it rides every turn, even if mostly as cache reads, so per call would understate it.

The Claude Code deferral is the interesting wrinkle: moving the cost to first use rather than removing it. When it lands on first lookup, does it land in full, or only the slice the agent actually reaches for? That decides whether deferral is a real saving or just a bill that arrives later.

Collapse
 
paullovelace profile image
Paul Lovelace •

This framing landed for me — I've watched sessions go sideways not on one huge dump, but on the tenth rerun of the same failing test, exactly like you describe. The manifest is what makes it usable for me: 169 lines down to 23 is nice, but knowing 54 were deduped and 88 summarised away is what I'd actually trust. One thing I'm curious about: when you run diet_extract with a query like 'test failure', how much surrounding context do you keep around a hit? I've been burned before by an excerpt that kept the error line but dropped the setup line just above it.

Collapse
 
techdevfun profile image
TECHDEV •

Good catch — and your exact failure mode is real. diet_extract keeps 2 lines either side of each hit by default (windowLines = 2 in src/extract.ts, overlapping windows merged, top 10 chunks in original order). So a setup line sitting right above the error line survives, but one sitting 3+ lines up gets dropped unless it scores on its own — matching a query term, or tripping the error/heading patterns. That gap between 'the line that failed' and 'the line that set it up' is exactly the shape your burn describes. What did your setup separation usually look like — the line directly above, or further up behind a blank line or two? Curious where you'd draw the window if you were tuning it.

Collapse
 
qisuancloud profile image
Alex Chen •

The manifest is what makes this trustworthy, but I would add one rule for test logs: dedupe by failure signature rather than by exact line. In serving and test logs, the same failure often reappears with a new timestamp, request id, or port, so exact matching keeps noise while still hiding the pattern. Keeping the first occurrence verbatim, with its surrounding lines and a count of how many times that signature recurred, would preserve the diagnostic signal without letting repeated runs eat the budget.

Collapse
 
techdevfun profile image
TECHDEV •

That's a sharp extension of the near-duplicate case from this thread — the address-varying repeats shipped in v0.2.0 as masked-key dedupe, first occurrence verbatim with a count, but a timestamp or request id changing every run is a wider net than an address. Would your signature be the normalized line itself — strip the volatile fields and compare what's left — or something coarser, like the failing assertion plus its surrounding lines? The surrounding-lines part matters too: the first occurrence alone can lose the setup line above the error, which was exactly the failure mode in the extract discussion. Curious which half you'd weight heavier, the signature or the context around it.

Collapse
 
glenallen profile image
Glen Allen •

The key insight here is that context management is about preserving useful information, not just reducing token count. I’d add that any compression layer should protect critical requirements and failure details while making every removal auditable. At IT Path Solutions, this is an important consideration when building AI-powered workflows: losing one small constraint can lead to a completely different result. A useful next step would be allowing developers to mark specific rules or sections as protected so they’re never silently dropped when the context budget gets tight.

Collapse
 
techdevfun profile image
TECHDEV •

Thanks, Glen — that's exactly the shape this landed in. The protect-lines idea shipped in v0.3.0 as protectLines: the caller supplies the lines verbatim, they're held at priority 0, ahead of even error lines, and if they don't fit the budget the default is to fail rather than silently drop them — there's a partial mode that keeps everything and flags the shortfall, but it's for diagnosis only, never reported as a successful compression. The manifest reports which protected lines were kept and which couldn't fit, so nothing protected disappears quietly. The failure-detail half you mention is already there in the other direction: error lines are priority 1 in summarize. The question I'd still put back to you, from your IT Path Solutions side: when you say 'mark specific rules or sections', do you picture the marking living in the repo docs where the decision already lives, or in a list the caller passes to the tool? The two point at different owners for which lines count.

Collapse
 
ascendra_ventures_92c21f2 profile image
Ascendra Ventures •

The manifest is the part that makes this trustworthy. Showing that 169 items became 23 kept, 54 deduped and the rest summarized means someone can check what was dropped instead of hoping nothing important went missing. Keeping the process deterministic and extractive helps for the same reason.

Paul Irolla's observation matches what we see on client projects. The large file is rarely the problem. It is forty small tool results, and test logs in particular, piling up across a session until the useful instructions are buried.

Launch Gate's comparison with double entry bookkeeping is a good way to explain planned versus executed checks to people outside engineering. If every planned step has a matching record, a missing one is visible at once.

Odinaka's line about the same fix meaning the same failure is worth turning into a dedupe rule. Have you tried grouping repeated failures by the fix applied rather than by the error text?

Collapse
 
techdevfun profile image
TECHDEV •

Thanks for pulling those three threads together — that synthesis is exactly the shape of the discussion so far, and the fix question is the right one to press on.

The honest answer is no, not as a grouping key in the tool — and the reason is where the fix lives. A log line carries the error text, not the fix applied afterwards, so the tool today has nothing to group by except what is in the lines themselves. What shipped in v0.2.0 is the closest neighbour, not the same thing: traces that differ only by a 0x address collapse, first occurrence kept verbatim, with the near count kept separate — and it deliberately does not claim 'same fix', because address-equality is not fix-equality.

Grouping by fix would need the fix to arrive as an annotation from outside the log — a caller tagging each failure with the fix it got — and then the grouping itself is easy, but the judgment has moved outside the tool, which is the same tension this thread has hit with caller-marked ranges. That raises the question I'm curious about: in practice, where would that fix tag come from for you — a post-run annotation, a link to the commit that fixed it, something the runner records at the time? And the boundary case that follows: two failures that share a fix but come from different call sites — group them, or keep them apart because the location still matters?

Some comments may only be visible to logged-in visitors. Sign in to view all comments.