A frozen 12-billion-parameter model answered 180 out of 180 fresh problem instances while generating no tokens at all. In a technical report posted to arXiv on July 26, author Sietse Schelpe describes a system that leaves a language model's weights untouched and instead grows a persistent memory of verified solution procedures beside it. Once a problem family has been solved and the procedure has passed a verification step that never consults an answer key, every later instance of that family is answered by executing the stored procedure -- deterministically, bit-exact, at zero generation tokens.
Key facts
- The headline number: 180 out of 180 fresh instances across nine problem families, at zero generation tokens per answer.
- The control that matters: with the memory empty, the same system solves nothing -- the capability lives in the stored procedures, not the 12B weights.
- Who and when: single-author industry report by Sietse Schelpe, submitted July 26, 2026, describing a proprietary system.
- Primary source: A Frozen 12B Beats Frontier Models on Verified Work on arXiv.
The usual way to make a language model better is to retrain it: more compute, a new opaque set of weights, and output that still varies run to run. This report takes the opposite route. The model stays exactly as it is. What grows is a registry of parameterized methods -- procedures that solve a whole family of problems rather than one instance -- each admitted only after passing an independent check that does not peek at the correct answer. Present a new problem from a family already in the registry, and the system selects and runs the stored method instead of thinking about it again.
The test battery covered nine mathematical and algorithmic families with unseen parameters, and four models from four different vendors, both dense and mixture-of-experts architectures, each scored 180 out of 180 with no generation tokens spent on the answers. The paper's framing is that execution-bound capability has been decoupled from parameter count.
The most useful review of this work is not in the paper. On the Hugging Face discussion page, the sharpest comment grants that the engineering is reasonable while calling the headline overstated: this is closer to executable retrieval-augmented generation, a verified function registry, or a tool library than to an improved base model. That reading is correct, and the paper half concedes it. It never runs frontier APIs against its own battery, comparing instead with published benchmark results and list prices. It states plainly that frontier models remain ahead at raw, from-scratch reasoning. And the empty-memory control proves the point: strip the registry and the system is just a 12B model again.
So the honest description is a cache with a quality gate. That is less thrilling than the title and more interesting than the dismissal deserves. Think of an experienced accountant who, the first time a novel filing situation appears, works it out slowly and writes the method into the office manual -- with a colleague checking the method rather than the answer. Every later occurrence takes minutes, produces the same result, and costs nothing extra. What makes this version notable is that the verification step never sees the answer key, which is what separates a genuine procedure cache from simply memorizing responses. Our explainers on agent memory and tool use cover the two halves of that design, and the related idea of compiling knowledge directly into a small artifact showed up earlier in program-as-weights.
There is also a property here that no sampling-based model can offer at any size: determinism. The same input produces the same bits, every time, with a verification record attached. For pricing engines, compliance calculations, tax logic or anything auditable, that is worth more than a few points of benchmark score. It is the difference between an expert who is usually right and a spreadsheet formula that is checkable.
The caveats are substantial. "Forever" means only as long as the memory persists and the world does not change -- a stored procedure encodes assumptions that can silently expire when a rule, a rate or a schema changes, and the paper does not describe how a procedure gets invalidated. The registry has to be populated by solving each family once, so there is prior compute and prior system state, not free capability. The whole result is scoped to problems that decompose into parameterized families with automatic verification, which excludes most open-ended reasoning. And reproducibility is essentially nil: a public testbench and benchmark repository exist, but the engine, configuration, raw artifacts and reproduction environment are withheld under a non-disclosure agreement. Read the title as marketing. Read the mechanism as a real question worth asking -- what does it mean to cache reasoning, and who verifies the cache?
Originally published on Ground Truth, where every claim is checked against the primary source.
Top comments (0)