Every retrieval stack prints context utilisation — how full you got the window. It is cheap, bounded, and looks like a measure of how well you used what you paid for.
Measured across 10,000 packing problems (2,000 corpora × 5 budgets, six packers each, exactly one document per corpus carrying the answer span):
| spread across all six packers | |
|---|---|
| utilisation | 2.45% |
| whether the answer survived | 56.56% |
23.1× wider, on the same runs. Read the utilisation column alone and they are the same system.
Run it: https://dev48.infy.uk/ai/days/day73-context-window-packing.html
The control
truncate is greedy-by-score plus one line: when the next document does not fit, put in as much of it as fits. It is what most hand-rolled context builders actually are.
| greedy_score | truncate | |
|---|---|---|
| utilisation | 98.33% | 99.88% |
| answer kept | 48.28% | 43.44% |
| runs ending in a cut fragment | 0% | 98.46% |
It fills the window best and keeps the answer worst, because a cut fragment still counts toward utilisation. The metric actively rewards the packer that destroys the payload.
Exact optimisation is not the fix
knapsack is a 0/1 dynamic programme over the budget — provably optimal on total relevance, checked here against brute force over all 4,096 subsets.
| knapsack | greedy_density | |
|---|---|---|
| mean score | 8,126 | 8,075 |
| answer kept | 54.52% | 55.59% |
On the 643 problems where the two disagree about the answer document, density is the one that keeps it 58.32% of the time. Maximising a total is not a proxy for retaining any particular sentence.
The ceiling is exactly 100%
An oracle that places the answer document first keeps it on every single problem. So the budget never binds here, and every miss by every other packer is a choice it made, not a window it ran out of. Without that row you would blame the budget.
And the arithmetic underneath
Counting tokens as chars/4 overflows the real window on 39.40% of these problems, while the packer believes it fit.
// the bug this page shipped in its first version, caught by its own verifier:
// the admission test must run on the ESTIMATE and the running total on the
// REAL cost. Adding one to the other reported 18.31% where the truth is 39.40%.
if (estRun + estimate <= budget) { estRun += estimate; realRun += trueTokens; }
Report the unstable thing — whether the span you needed is present — not how full the window got. And put an oracle row in the table so you can tell a bad packer from a small budget.
113 independent verifier asserts, 25 in-page checks, 0 failures.
Top comments (0)