Counting tokens is the easy part. Every library does it.
The hard part is what happens after you are over budget. You have to drop something — and most tools drop silently. They truncate the tail of the prompt, or they keep adding files until they hit the limit and stop. Both leave you with an agent that answered from a prompt you never saw.
So I built roomy: it packs under an explicit budget and returns, for every candidate, either the reason it is in or the reason it is out.
👉 Repo (MIT): https://github.com/dev48v/roomy
👉 The write-up, live: https://dev48.in/agentlab/vol5-02-roomy.html
$ python roomy.py ../mcpfromzero --budget 6000 --reserve 1000
budget 6000 reserve 1000 used 4820 headroom 180
------------------------------------------------------------------------
IN README.md 2871 fits (priority 2)
IN docs/NOTICE.md 412 fits (priority 1)
IN python/mcpfromzero/__main__.py 27 fits (priority 0)
...
out conformance/run.py 3694 no room left: 3694 needed, 180 free
out typescript/src/tools.ts 4371 no room left: 4371 needed, 180 free
out python/mcpfromzero/tools.py 4666 no room left: 4666 needed, 180 free
------------------------------------------------------------------------
9 in, 14 out
The rule of thumb is wrong in the dangerous direction
Four characters per token is an average over English prose. Agent prompts are made of code, where identifiers split, punctuation is its own token, and every newline costs.
Measured over one real repository:
| measure | tokens |
|---|---|
| characters | 206,658 |
| the 4-chars-per-token rule | 51,664 |
| roomy's estimate | 65,859 (+27%) |
An over-count wastes a little budget. An under-count overflows the window, and that failure arrives as a truncated reply rather than as an error. roomy is deliberately biased toward the first.
It ships no tokenizer, on purpose — vendoring one ties the tool to a single model family and is wrong for every other.
The reserve is not optional
The window is shared with the output. A prompt that "fits" in 6,000 tokens does not fit if the model then needs 1,000 to answer:
| reserved for the reply | usable | files in | tokens used |
|---|---|---|---|
| 0 | 6000 | 11 | 5917 |
| 1000 | 5000 | 9 | 4820 |
| 2000 | 4000 | 9 | 3959 |
| 4000 | 2000 | 7 | 1949 |
A required item that does not fit raises
plan = pack(candidates, budget=8000, reserve=1000) # raises WontFit
It does not return a plan with the thing missing. A caller who ignores that exception has built a prompt without something they declared they could not work without — and that failure comes back as a confident, wrong answer rather than a crash.
Smallest-first is a heuristic, and the gap is measured
Packing a budget optimally is knapsack. roomy sorts by priority, then size, then name. Rather than claiming that is fine, the page measures it against an exhaustive 2^16 search:
| budget | greedy | optimal | greedy / optimal |
|---|---|---|---|
| 1500 | 1403 | 1498 | 93.7% |
| 3000 | 2721 | 3000 | 90.7% |
| 6000 | 4966 | 6000 | 82.8% |
| 12000 | 11809 | 12000 | 98.4% |
| 24000 | 14381 | 14381 | 100% |
It gives up real window. That is the trade: a packing you can explain line by line and reproduce exactly is worth more here than a few percent, because the output is read by a human deciding what to include — not by a bin.
The sort ends on the name so the result never depends on the order the caller happened to build the list in.
Tested cross-language
The write-up ships a JavaScript port of the packer. A port checked only against itself is not checked, so the verifier runs the Python package as a subprocess and demands the two agree, decision for decision, on the same inputs — 38 randomised cases plus deliberate edges (empty lists, duplicate names, a reserve that swallows the budget, a zero-token candidate).
36 unit tests. 909 verifier asserts including the cross-language comparison. 0 failures.
The four invariants worth guarding, and the only ones the tests really care about: the budget is never exceeded, every candidate gets a decision, the result does not depend on input order, and a required item that does not fit raises rather than vanishing.
Top comments (0)