CodeSmith Tool Internals: House Rules for the Model

Tool Internals: House Rules for the Model

Source version of CodeSmith: v0.5.0 (commit 3a74c82f). All paths are relative to the repo root; line numbers refer to this version.
Intended audience: anyone who has written a tool's description, or been driven up the wall by a model misusing tools.

Article 16 covered the slot — how the tools are brought under one management. This one is about the manuals of the weapons themselves. Start with the single most typical tool description (crates/tui/src/tools/file.rs:32):

Read a UTF-8 file from the workspace. Use this instead of cat, head, tail, or sed -n '..p' in exec_shell — it's faster, sandbox-aware, and skips the approval prompt.

"Use me, not cat/head/tail/sed inside exec_shell — faster, sandbox-aware, and no approval prompt." Across the repo's tool descriptions there are ten of these "Use this instead of ..." anti-pattern lists: write_file talks you out of heredocs, edit_file out of sed -i, fetch_url out of curl, grep_files out of grep -r.

These sentences are not documentation written for humans — they are house rules written for the model. The model has a native bias toward the shell (cat and curl are everywhere in its training data), and the shell is the most expensive, the most dangerous, and the hardest-to-approve path there is. Writing "not that one, this one" into the tool descriptions is behavior-shaping at the lowest cost on offer: the manual sits in front of the model's eyes, along with the tool catalog, on every single request. This installment watches the house rules enforced at three levels: the manual teaches, the error message corrects, the engine blocks.

The teaching manual goes beyond anti-pattern lists: eight high-frequency tools also append invocation examples to the end of their descriptions.

read_file's description closes with this string (file.rs:32): Examples: {"path": "src/main.rs"} (first 200 lines); {"path": "src/big.rs", "start_line": 201, "max_lines": 200} (next chunk after a truncated read).

The examples are byte-stable static text — no timestamps, no request ids — so Article 9's prefix cache comes to no harm; and a model that copies the parameter shape of an example skips one whole lap of the "wrong parameters → error → re-call" detour that free improvisation pays. edit_file's example even carries a line of home tutoring at the end — "copy the text verbatim from a prior read_file result" — writing "read before you edit" straight into the manual.

This investment in examples has experimental backing: attach 1–5 real invocation examples to a tool description and tool-calling accuracy on some benchmarks climbs from roughly 72% to 90% — a JSON Schema can express types, but not implicit conventions like "is the timestamp in seconds or milliseconds" or "how do the filter conditions nest", and examples are the easiest way to convey those.

More fundamental than the examples is the orientation of the description: "when to use it" matters more than "what it can do" — "search for related content" loses badly to "use when you need real-time information or need to look up unknown facts"; the former describes a feature, the latter helps the model make the call decision. And crisp boundary counter-examples (those ten "Use this instead of ..." lines at the top of this installment) are often more important than capability lists, because the root cause of most failed calls is not that the model doesn't know what a tool can do, but that it doesn't know what it cannot do.

One debugging principle comes along in the same package: when the Agent keeps picking the wrong tool, fix the description before you suspect the model — the return on editing a description beats upgrading to a stronger model by a mile.

Reading Files: One Page at a Time

read_file's pagination constants (crates/tui/src/tools/file.rs:77-81): default 200 lines, hard cap 500 lines, 16 KB of visible bytes. Small files (≤200 lines and ≤16 KB, with no explicit range) take a fast path and are returned as-is, unwrapped — the comment explains this preserves historical behavior so existing sessions don't suddenly see a wrapper format. One compatibility detail exposes the chronic worry of this class of system: any change to the output format gets amplified by the double inertia of prefix cache and model.

Large-file output carries line numbers in the format {line_no:>6}│ {line} — six columns right-aligned plus a full-width bar: cat -n, modernized. Two design choices gleam here. Paging past the end is not an error (file.rs:132-143): a start_line beyond the total line count returns the [NO CONTENT] sentinel instead of raising — the comment says it "lets the model page past the end without raising" — if it flips past the last page, tell it the book is finished; don't punish it.

The truncation notice hands over the next call's parameters (file.rs:181-190): To continue, call read_file with path="..." start_line=N max_lines=M — the model copies them and reads on; no mental line-number arithmetic required.

Editing Files: The Engineering of Precision

edit_file is the textbook case of exact-match replacement (crates/tui/src/tools/file.rs:556-892). The validation chain stacks layer on layer: search == replace is refused outright; replace_all and occurrence are mutually exclusive; occurrence must be a 1-based positive integer. Then comes the most central rule — multiple matches mean failure — and the failure message is itself an instruction (file.rs:791-797):

Search string matched {count} locations in {path}; pass replace_all=true to replace every occurrence, or occurrence=N (1..={count}) to replace a specific one

It tells the model how many places matched, the two ways out, and the legal range of N. A model that receives this error gets its next call right by nature. A good error message is not a status report; it is the construction blueprint for the next step.

After exact matching fails, a three-level fuzzy fallback kicks in, each level aimed at a real failure mode. Level one tolerates leading whitespace: even when the indentation doesn't line up (spaces and tabs that look identical to the eye), the match can still be located, with a byte-mapping table mapping normalized positions back to the original span. Level two normalizes typographic punctuation (file.rs:1023-1051): curly quotes → straight quotes, em-dashes → --, non-breaking spaces → ordinary spaces — the comment says outright that this catches the paste failure of "browsers/chat clients silently substituting punctuation". By the time the model copies a code snippet off a web page, the punctuation has long since been contaminated.

Level three strips line-number prefixes (file.rs:1088,1126): the model copied the {n:>6}│ line numbers from read_file output verbatim into old_string — the fallback chain strips the prefix before matching, recognizing its own format plus cat -n/grep -n variants.

The guardrail is double-layered — multi-line text requires every line to carry the prefix and the line numbers to be strictly increasing; for single-line text the monotonicity check spins idle, and a line-number alignment check takes over as gatekeeper: the stripped text must land exactly on the line the first line number points to (those line numbers are absolute file line numbers, so a genuine transcription is bound to align; prose that begins with digits, 12:30:00, strips down to 30:00, lands on the wrong line, and is rejected outright).

The scan cursors of the two fuzzy fallbacks and the line-number matcher all advance by "skip the entire match", never "+1 byte" — a multi-byte first character can never slice the next cut into the middle of a character, which is exactly the regression the CJK regression tests stand watch over. Each of the three fuzzy levels still demands a unique hit; multiple matches still error out and ask you to disambiguate — the fuzz is elbow room for a trembling hand, not a pass for ambiguity.

The very existence of the fuzzy fallback has an industry cautionary tale for a footnote. A certain version of Cursor silently converted Chinese curly quotes into English straight quotes at the parameter-passing layer: the reading tool returned the curly quotes as-is, while the replacement tool's parameters were quietly straightened at the entrance — and so "the content I plainly saw, the tool says it cannot find": the model retried and failed, retried and failed, with no way to diagnose anything; the write direction was falsified the same way — ask for curly quotes, get straight ones on disk.

What this violates is a bottom line of tool design: there must be no systematic skew between the world the model perceives and the world the tools operate on — normalization is allowed, but it must be written into the description and announced in the return; otherwise the "smart correction" manufactures a systemic failure the model cannot self-diagnose. CodeSmith's normalization takes the righteous path: it changes neither parameters nor files, and tolerates only at match time — a search written with curly quotes can hit an original written with straight ones, while the file itself does not move by a single byte.

When the span is too long for quoting the whole thing to be realistic, there is also an anchor mode: to replace a range of more than 40 lines you need not quote it in full — just pass search_start + search_end, the first and last lines (described at file.rs:565, validated starting at file.rs:626).

The semantics are deliberately tightened: the start anchor must be unique across the whole file; the end takes the first matching closed range after the start; anchors and the traditional search are mutually exclusive — mix the parameters and what comes back is an executable error: "pass search_start and search_end together, or neither". The schema's required loosens accordingly from ["path", "search", "replace"] to ["path", "replace"].

Anchor mode is doing token economics: to replace a 400-line block of generated code, the model needs to quote exactly two lines.

Zoom the lens out and the design space of file editing holds five plans in all, each with its wins and losses. Diff description + Apply Model: generate a diff and hand it to a dedicated small model to merge — Cursor's old route; the separation of duties is pretty, but when the description and the original text disagree, the merge step applies the patch in the wrong place, and the route was abandoned once base models grew strong enough.

Old string to new string: the shared choice of Claude Code, Codex, and today's Cursor — predictable and transparent; if the old string exists and is unique, it succeeds; the price is that deleting a large passage means quoting the original verbatim, where a single character of drift fails the match. Line-number targeting: "delete lines X through Y" — precise and token-cheap, but every edit renumbers everything after it; when one thinking pass emits edits in several places, they must all use the initial line numbers, and models get them mixed up.

Vim-like commands: efficient for restructuring code (moving a function from one place to another), but the learning burden of the command grammar is a disaster for weaker models; one layer deeper, Vim was designed for humans who "look and change in small steps", while the model's way of working is "think long, then change in bulk". Head-tail anchor matching: quote only the first few lines and the last few, omit the middle — the search_start + search_end above is precisely this plan landed, combining the reliability of content matching with the token savings of the line-number approach.

All five plans sit the same exam: between the vagueness of human-language expression and the precision of machine execution, where exactly does the interface make its cut?

Multi-block structured edits go through apply_patch (crates/tool-impls/src/tools/apply_patch.rs:179-331): unified diffs, multiple hunks across multiple files, fuzz widening the search context step by step from 0 to 50, and a cumulative offset maintained between hunks.

Its hunk-failure message likewise ships with its own teaching materials (apply_patch.rs:1079-1082): a preview of the expected context (first 4 lines), the file's actual snippet (±2 lines), plus a Hints line — "confirm the patch corresponds to the current file content, increase the fuzz, or regenerate the patch". The multi-file scenario even gets semi-transactionality: files are written one by one, and the moment any file fails, the ones already written are rolled back in reverse order (rollback_pending_writes, apply_patch.rs:963-1018).

The honesty ledger owes a confession too: file writes are not atomic (write_file/edit_file/apply_patch all call fs::write directly — no temp file plus rename); atomicity is backstopped by the per-round snapshots of Article 7's side-git. But before anything reaches the disk there is now a syntax gate: writes to .rs/.toml/.json pass a parse check before landing, judging the change rather than the state — a file that parsed perfectly until now may not be written broken, while a workspace that never parsed in the first place still gets repaired as usual — and the four disk-writing tools, write_file/edit_file/apply_patch/fim_edit, share this one gate (agent-runtime/src/tools/parse_gate.rs, covered in depth in Article 18). Config and binary writes, meanwhile, have switched to atomic writes (Article 20).

As for "read before you edit" — an early draft of this installment had filed it under "missing" as well; that was a missed detection: agent-runtime/src/tools/freshness.rs has had the full set of gates since the repo's init. FileFreshnessTracker records mtime+len fingerprints, and FreshnessWrappedTool stands outside read/edit/write/fim/apply_patch to block both "no editing what you haven't read" and "no editing what changed after you read it".

Feature::FileFreshness is on by default — it lives in agent-runtime rather than on the file tools' implementation path, and both articles' greps walked right past it (the postscript to section 2.2 of the comparison piece records the comeuppance).

Beyond the gates there is a proactive flank: after a shell command succeeds, the files recorded in the tracker are re-statted in batch, and any change found triggers a reread reminder directly in the tool result (ShellProbe, capped at 5 paths to keep the noise down) — on top of "passive blocking at edit time", it adds a hand of "active reminding at change time".

The probe set has been consolidated into the SHELL_PROBE_TOOLS constant and nailed down by a registry test — every name in the set must actually exist in the tool registry. That test was not written for decoration: without it, task_shell_output was matching thin air (the name simply was not in the registry), while four others — exec_shell_interact/exec_shell_cancel/exec_wait/exec_interact — slipped through the net. With it, no rename and no new variant can silently disarm the probes.

The same drift gets exactly one reminder; it takes a further drift to earn the next (re-reporting on every poll would be spam, and one reread re-arms the reminder). A deleted file counts as drift too — a generator deleting files is precisely the scenario that most deserves a reminder. Background tasks are not probed while Running (the probe would race the command's writes; the poll in the finished state does the probing). And a path field hallucinated into shell input never enters the tracker either — otherwise the attack surface would gain one more route to "stamping yourself as read". Norms do not all have to be written as code, but you must guarantee that every norm has some mechanism holding it up.

Finding Files: Count Everything First, Then Truncate

The search tools share one gentle design in their truncation strategy. grep_files caps at 100 results (crates/tool-impls/src/tools/search.rs:21-29), but the return structure carries "truncated": total_matches > max_results — count all the matches first, then truncate the display. "347 hits in total; here are the first 100" and "there are 100 hits here" are two entirely different decisions for a model.

The excluded-directories list is written as double insurance (each directory spelled both ways, bare name and dir/* alike; the #2200 comment calls itself "belt-and-suspenders"): node_modules, .git, target, *.min.js, dist, build, __pycache__, .venv. Files over 10 MB are skipped outright (search.rs:211-216).

file_search clamps results to 1–200, default 20; list_dir sets no entry cap at all — oversized output is backstopped by the spill-to-disk envelope (already wired up for shell output, see Article 16; the general-purpose tools' 100 KB spill threshold lives at truncate.rs:56 — pipeline present, no production caller yet).

The breeds of search tool themselves come with a four-box map:

1. Regex content matching (grep/ripgrep) — fastest and most accurate when you know the literal text you are after;
2. Filename patterns (glob) — the first step in exploring a project's structure, no file-opening required;
3. Semantic search (vector embeddings) — for code that is "close in meaning, different in wording": search "verify the user's identity" and hit check_credentials;
4. Symbol-level lookup — the IDE-style "go to definition / find references": knowing that authenticate on line 42 is the definition and line 189 is a call, which plain-text search cannot tell apart.

Whether to build an embedding index for semantic search once fought a genuine route war in the industry: terminal Agents like Claude Code deliberately refuse — pure grep+glob retrieval on the spot, no index to keep fresh as the code evolves out from under it, and a whole indexing infrastructure saved; Cursor took the opposite route early on, and later walked back to grep+glob as well.

CodeSmith stands on the no-index side — the model's search tool surface is exactly grep + glob (the repo's tree-sitter symbol-index index crate mainly serves completion and is not on the tool surface). I would argue semantic indexing does not have to mean embeddings or nothing: build an incremental index tree keyed by content hash, reuse unchanged subtrees at zero cost, and perhaps a third way opens between "index freshness" and "maintenance cost" — a question I will leave to a future article.

All file tools share one path boundary (crates/agent-runtime/src/tools/spec.rs:490-597): a three-stage check — first the canonical prefix is looked up; for paths that already exist, canonicalize and look again; for paths that do not exist yet (about to be created), walk up to the deepest ancestor that exists, canonicalize, stitch the remainder back on, and check that.

The third stage defends against symlink escapes: if the link in workspace/link/../../etc/passwd points outside, even the create-it-fresh scenario gets caught. The escape error is one hard sentence: path escapes workspace. The whitelist admits only trusted_external_paths added explicitly through /trust add.

The base cwd for path resolution survives between exec_shell calls — each command is still its own independent process, and the gap where the next call forgets your cd is patched by the fold-back mechanism.

Foreground commands are wrapped in a POSIX wrapper script; when the command finishes, it prints one sentinel line, __CODESMITH_CWD__$PWD, at the end of stderr (wrap_with_cwd_capture at crates/tool-impls/src/tools/shell.rs:227,240); the engine strips the marker, canonicalizes, confirms the path is still inside the workspace boundary, and folds it into Session.cwd at the end of the turn.

And so cd survives across calls; environment variables do not (a comment calls it "a lightweight stand-in for a persistent terminal session"). Before the fold-back, the boundary check must pass: if the path in the sentinel escapes the workspace, it is not adopted; if the command exits abnormally and the sentinel never printed, nothing happened. The session cwd slot itself is bound per turn (session_cwd at tools/spec.rs:168): "a cd cannot outlive the session that performed it".

The textbook advice and the harness's counter-trade: the textbook's default advice is to maintain a persistent terminal session — cd, virtual environments, environment variables, and background services all living on inside one long-lived shell, "closer to the working habits of human developers". CodeSmith pointedly does the opposite: every command is an independent process, only the cwd folds back into the session, and environment variables are kept deliberately non-surviving.

The rationale is security and determinism — a resident shell is a stateful execution environment, and the variables exported into it and the processes left hanging in its background are implicit state the model cannot see and the approval gate cannot reach; independent processes make every execution start from a clean baseline, predictable and auditable. The price is chores like re-activating the virtual environment every time — a few extra prefix lines per command, exchanged for the elimination of an entire class of state-drift failures. On one point, though, the two sides agree: "immediate syntax feedback" — lint on write; Article 10's LSP diagnostic flush is where that principle lands in the engine.

Network Policy: Three States and Deny-Wins

The network is another country. Policy configuration has three states, Allow | Deny | Prompt (crates/agent-runtime/src/network_policy.rs:58-66), and two of its design decisions are written directly into the comments.

A mistyped config is not an implicit allow (network_policy.rs:79-88): unknown values all fall back to Prompt — "a typo never silently disables the policy"; one spelling error must never silently switch off a security policy. Deny gets a one-vote veto (network_policy.rs:203-225): a host that matches both allow and deny is treated as deny.

Domain matching has one counterintuitive detail: .example.com matches any subdomain but not the bare domain — want both, list both: "To match both, list both".

The defaults stand on the safe side — a missing config section is no exception: when the [network] section is absent, config loading returns a default decider with default = "prompt" (network_policy_decider at crates/tui/src/config.rs:1437) — the first time a network tool touches an unapproved host, what pops up is the standard inline approval (with "approve for this session" deduplication), never a silent pass-through.

A comment goes out of its way to reassure the YOLO users: in auto-approve sessions the approval gate lets go automatically and the out-of-the-box experience is unchanged; embeds without a decider attached keep the old behavior (tools/spec.rs:204-207).

Prompt's semantics got a new lease on life too. The old implementation converted Prompt into permission_denied, and the error even coached the user to "re-run after /network allow <host>" — but that was a dead end: follow the instructions, finish writing the config, and the living engine still never refreshes its policy snapshot; the retry fails all the same.

The new mapping (network_approval_requirement at network_policy.rs:107-127) is Prompt → ApprovalRequirement::Required: the "asking" is handed to the dispatch-time inline approval gate (approval_requirement_for_input at fetch_url.rs:149-171); approvals are deduplicated by session-level fingerprints and can also be persisted back into the allow list.

Allow and Deny both map to Auto — Allow has nothing to ask; Deny will hard-fail at execution time, and as the comment puts it, "asking the user to approve a call the policy explicitly forbids would be absurd".

At the execution-time check (fetch_url.rs:397-417), only Deny still raises network call to '{host}' blocked by network policy; Prompt passes straight through — the comment spells out the reason: by execution time, either the user has already approved or there is no approval channel at all (embeds) — raising here would seal off both roads.

The audit log carries on as ever: every outbound call drops one line into ~/.codesmith/audit.log (network_policy.rs:320-390).

fetch_url: Seven Gates for One URL

fetch_url, which fetches a URL directly (fetch_url.rs:93-274), has turned SSRF defense into an assembly line, in execution order:

1. Protocol whitelist: only http/https are recognized; file:///etc/passwd is locked down by a dedicated test case (fetch_url.rs:596-603);
2. Policy evaluation: through the three-state table above;
3. localhost hostname rejection: http://localhost and 127.0.0.1 are blocked outright;
4. Literal IP check: bracketed IPv6 included (the hole patched by GHSA-88gh-2526-gfrr);
5. Restricted-IP table (fetch_url.rs:305-340): loopback, RFC1918 private networks, link-local, multicast, CGNAT 100.64.0.0/10, cloud metadata 169.254.169.254 (the comment names AWS/GCP/Azure), the IETF benchmark range 198.18.0.0/15, Class E; IPv4-mapped IPv6 like ::ffff:127.0.0.1 is unwrapped before the lookup;
6. Per-IP DNS validation + pinning: every resolved address passes the restricted table, and then the first legitimate IP is pinned into the connector with resolve() (fetch_url.rs:201-207) — the comment says outright that this defends against "DNS rebinding (TOCTOU)": the name resolves to a safe IP at validation time, and a fresh resolution at connect time can be flipped to point inward; once pinned, reqwest connects straight to that IP;
7. Per-hop redirect re-checking: redirects are followed manually (cap of 5 hops), and every hop walks the gates again — a safe first hop is no guarantee of a safe third.

One behavioral detail more: 4xx/5xx does not count as a tool error (fetch_url.rs:284-295) — the body and success: false are returned — an API's error envelope (usually a slab of JSON) is itself a diagnostic clue, and a shame to throw away.

Users on TUN transparent proxies get one narrow door: fake-IP mode resolves DNS into the 198.18.0.0/15 placeholder range, and trusted_fakeip_cidrs exempts that range only (network_policy.rs:476-525); the tests verify explicitly that genuinely internal addresses like 192.168.1.1 and 169.254.169.254 are not on the exemption list.

The Network in the Shell: Translating Failure into a Cause

The model runs network commands in the shell too. The NETWORK_COMMANDS table in command_safety.rs (crates/agent-runtime/src/command_safety.rs:588-609) lists 20 first words — curl, wget, nc, ssh, nmap, and the rest — and a hit escalates the approval, with the rationale string "Command may make network requests".

Note that this is escalated approval, not a lockdown: the house rules exist to make actions visible, not to make them impossible.

Finer still is the lookup table for "does this command actually need the network" (crates/tool-impls/src/tools/shell.rs:130-167): git fetch/pull/clone/ls-remote/submodule/push count as network commands, git commit does not; cargo install/fetch/update count, cargo build does not; npm install counts, npm run does not.

A coarse one-size-fits-all would hold git commit hostage too and nag users to death; this table is user experience paid for in patience.

And when the sandbox really has cut the network, the tool output gets translated (shell.rs:169-221): the command genuinely needs the network, the failure mode genuinely looks like a block, and the sandbox policy genuinely has no network — only when all three conditions hold is shell_network_denied_hint attached. What it guards against is the model's reflex on seeing "could not resolve host": tweak the parameters and try eight more times. Once the failure has been translated into "this is the policy's doing", the correct next step (request clearance, or reroute through fetch_url) finally surfaces.

Article 1 quoted that line from HARNESS.md — "Non-zero exit codes, sandbox denials — fed back as correction vectors" — and this is the scene of its implementation.

One detail in passing: web_search's error echo passes through a scrubber first (crates/tui/src/tools/web_search.rs:911-920) — HTML tags stripped, control characters removed, Bearer tokens regex-replaced with Bearer [REDACTED]. Should an API key happen to appear in an upstream error body, it is picked off at this layer: it enters neither the model's context nor the session transcript.

Conclusion: Teach, Correct, Block

Gather the house rules into their three layers. The manual teaches: ten "Use this instead of" lines pull the model back from the shell to structured tools, saving approvals, context, and risk at the source. The error message corrects: the multi-match error carries its parameters, hunk failures carry previews, network blocks carry reasons — even the approval prompt tells you how to unlock — every error is the construction blueprint for the next step. The engine blocks: pagination caps, count-before-truncate, the three-stage path check, seven gates of SSRF, DNS pinning — the model would have to work hard to get it wrong.

The house rules of this installment defend against the model's clumsiness: a trembling hand, a fondness for the shell, one page turned too many. But something else is mixed into the context as well — malice planted with care by someone else. A single "ignore the previous instructions" in a web page, a command in AGENTS.md disguised as a project convention: that is an adversary of an entirely different weight class.