CodeSmith Meta-Ability: Let Code Think for the Model

Meta-Ability: Let Code Think for the Model

Source version of CodeSmith: v0.5.0 (commit 3a74c82f). All paths are relative to the repo root; line numbers refer to this version.
Intended audience: readers who have watched a model get an arithmetic problem wrong and want to know why "having it write code" is the right answer after all.
Series index: README.

The main line of the previous twelve installments was how the harness constrains and manages the model. This one shifts the angle: how the harness arms the model — not by handing it more prefabricated tools, but by giving it the ability to create tools on the spot.

What Is "Meta-Ability"?

An ordinary capability is the Agent being able to do some concrete thing: answer a question, call a specific API, generate a passage of text. A meta-capability is "a capability that can create other capabilities": with it, the Agent writes new tools, new constraints, new forms of expression on the spot to get the job done, rather than having every capability prefabricated in advance. Code generation is precisely such a meta-capability — exact, executable, composable — so it can produce new tools (scripts, probes, API call sequences), new constraints (assertions, validation rules), and new forms of expression.

What this capability can act on lines up, from innermost to outermost, in at least six directions:

1. Thinking itself — replacing error-prone natural-language reasoning with code (this installment);
2. Business rules — encoding vague policy as executable constraints;
3. Content presentation — generating slide decks, video frames, visualization artifacts;
4. System interfaces — bridging heterogeneous APIs, adapting to evolving data formats;
5. User interfaces — dynamically constructing forms and interactive interfaces;
6. The Agent itself — creating or repairing new Agents in code, bootstrapping in the process (Article 14).

This series picks the two ends of the lineage: the innermost, thinking, and the outermost, the self — the four directions in between, readers can unlock for themselves with the same key. When Article 6 introduced RLM I called it "opening a private kitchen for itself", but that was only the operational layer. This installment takes the principle all the way down: why "letting the model write code" is not gilding the lily but a patch for a fundamental shortcoming in the way the model thinks.

A Probabilistic Brain Cannot Do Deterministic Sums

LLMs are astonishing at natural-language understanding and generation, but fundamentally weak at exact arithmetic, symbolic manipulation, and strict logical derivation. The reason, in one sentence: the model thinks probabilistically and approximately, while mathematics and logic demand deterministic, exact answers.

One concrete comparison:

Problem: "A class has 40 students. 60% are taking math, 45% are taking
          physics, and 25% are taking both. How many take physics but not math?"

Natural-language reasoning (error-prone):     Code reasoning (exact, verifiable):
"60% math = 24 students,                     math = int(40 * 0.60)    # 24
 45% physics = 18 students,                  phys = int(40 * 0.45)    # 18
 25% both = 10 students,                     both = int(40 * 0.25)    # 10
 physics only = 24 - 10 = 14 students"       only_phys = phys - both  # 8
→ subtracts from math count; wrong answer    → print(only_phys)  # 8 ✓

The left side is not the model "being stupid"; it is the thing's native mode of work: every step of the reasoning is a sample from a probability distribution, and chained across many steps the error snowballs. The right side has no such problem — code either runs right or throws; there is no "roughly correct". Let the LLM understand the problem and write the code, and let the interpreter do the exact computation — each sticks to its own trade.

This idea did not wait for the Agent era. Stephen Wolfram, the founder of Mathematica, made the point long ago: symbolic computation systems (systems that work on expressions in mathematical symbols rather than approximate numbers) are exact but cannot understand plain human speech — Wolfram Alpha depends on a built-in syntax parser, and the moment the phrasing of the question shifts a little, parsing fails. The LLM patches exactly that hole, while putting its own weak spot on display at computation. And so the collaboration pattern settled into its final shape:

User's natural language ──LLM──▶ formal expression (code) ──interpreter──▶ exact result
(fuzzy, open-ended)              (clean structure, unambiguous)           (deterministic)

In plain words: the LLM is the translator, the interpreter is the calculator — never make the translator do the arithmetic.

Back to the Code: The RLM Prompt Writes That Sentence into a Contract

With this perspective in hand, go reread the RLM system prompt in crates/agent-runtime/src/rlm/prompt.rs and you will find it is precisely the engineering of the division-of-labor principle above. The opening paragraph (prompt.rs:14):

You are the root of a Recursive Language Model (RLM). The input is loaded into a long-running Python REPL. You hold a live context handle, not the raw body. Read only through bounded helpers, compute in Python, and delegate semantic judgment to child calls.

Translated, that is three sentences: you (the model) may not look at the raw material directly — you are handed a live handle; exact work (slicing, searching, counting) is invariably done by writing Python; semantic judgment (what this code means, whether it should be changed) falls to you and the child LLMs alone. The module header comment (rlm/mod.rs:15-18) states the invariants even more sternly:

Invariants:
- `P` is held only as a REPL variable (`context` / `ctx`); never
  appears in the root LLM's window.
- The root LLM receives small metadata messages — length, preview,
  helper list, prior-round summary.
- Code rounds and sub-LLM calls travel over a single stdin/stdout
  pipe to a long-lived Python subprocess. No HTTP sidecar.

In plain language: the input is forever only a variable living in the REPL, and only metadata (length, preview, the previous round's summary) ever enters the model's window — this is the "virtual memory" idea of VarHandle (Article 8) recreated for the reasoning scene; code rounds and child calls travel a single stdin/stdout pipe into a resident Python subprocess, sparing even the HTTP boundary.

The REPL helper functions that live under this prompt are worth a look too, because each one is the "translator/calculator" division of labor made concrete: context_meta() returns bounded metadata; peek(start, end) takes a bounded slice by offset; search(pattern, max_hits=100) is regex search with a cap on returned hits; sub_query(prompt, slice) delegates semantic judgment to a child LLM and can confine its scope to a single slice. The most delicious part is the contract clause (verbatim from prompt.rs):

Contract: every turn, output exactly one `repl block of Python and nothing else. No prose-only turns. No "I will do X"; emit the code that does X.

Every turn, exactly one repl block of code — no talking instead of doing. "Next I will count..." is banned in so many words; if you want to say it, write the code that does the counting. This is what turns "let code think for the model" from slogan into protocol.

The recursion's budget is policed by real numbers too: in rlm/bridge.rs, every child call has a 120-second timeout (CHILD_TIMEOUT_SECS), default max_tokens 4096, and a cap of 16 prompts per RPC batch (MAX_BATCH); recursion depth is counted by depth_remaining, and when it runs out, execution degrades to an ordinary LLM completion. A translator may delegate to translators layer upon layer — but every layer has a clock and a purse.

The Harness Comes Before the Capability

One more detail must be confessed honestly: RLM's Python subprocess is an ordinary subprocess — it does not pass through an OS-level sandbox. The Seatbelt/bwrap apparatus covers shell commands only; repl/runtime.rs directly spawns whatever python it finds on PATH. Its cage is a different one: bounded exits (stdout past 1,000 characters becomes a handle, Article 6), child calls pinned to Flash (the model parameter confiscated), nesting depth capped. A meta-capability amplifies not just capability but destructive power, so the harness must be in place before the capability is — even if this harness is not a sandbox but a contract over exits and child calls. It is the continuation of the series' "distrust" throughline: we dare to give the model the power to write code precisely because every layer's exits are fenced.

Summary

1. Model inference is probabilistic; exact computation and strict logic are its weak spot;
2. LLM as translator, interpreter as calculator — the natural complementarity of the two systems (neural and symbolic);
3. RLM writes that division of labor into a contract: the model sees only metadata; every exact operation goes through Python;
4. The "one repl block per turn" contract turns "thinking in code" from a suggestion into a protocol;
5. Recursion has a budget (120s timeout, 4096 max_tokens, batch cap 16, depth counting), and the capability has a sandbox.

Trade-off: writing code plus executing it costs far more latency and token spend per turn than simply "having a think" — for simple problems it is a sledgehammer to crack a nut — so RLM is designed to open only when the model decides it needs it, not on by default.

The other face of meta-capability is defense: code is not only a tool for solving problems; it is also the goalkeeper that constrains behavior.