Tool Internals: Fifty Weapons, One Slot
Source version of CodeSmith:
v0.5.0(commit3a74c82f). All paths are relative to the repo root; line numbers refer to this version.
Intended audience: readers who have added a tool to an Agent, or who wonder how a few dozen tools get governed under one roof.
Article 1 recounted that war: open-source models forging tool calls in plain text, and the engine answering with streaming stripping and trust placed solely in the API's structured channel. Once the war died down, a question was left behind — what does the tool itself look like on the other end of the channel?
CodeSmith has fifty-odd built-in tools (50 impl ToolSpec under crates/tool-impls, plus 62 more on the TUI host side), on top of any number of MCP external tools, dynamic-library extension tools, and subagent tools. Read a file, run a shell, open a Python session, fire an HTTP request — weapons gathered from every corner of the world, and all of them end up plugged into the same slot. This installment covers how that slot was built, and why it took this shape.
Three Layers of Traits: The Slot's Family Tree
The slot has three layers of lineage, housed in three separate crates.
At the bottom sits the framework-layer Tool (crates/agent/src/tools/mod.rs:46-78) — the minimal, host-agnostic contract whose doc comment calls itself a "LangChain BaseTool analog": name, description, and input_schema, plus a run that returns a boxed future.
It deliberately knows nothing about approvals or sandboxes. The comment says capabilities at this layer are merely "advisory metadata for hosts" — the framework core does no gatekeeping; gatekeeping is the host's business.
In the middle is the production-layer ToolSpec (crates/agent-runtime/src/tools/spec.rs:809-886) — the layer that actually does the work, and much fatter for it. Beyond the core methods, its most opinionated design is capabilities as approval policy (spec.rs:848-858):
fn approval_requirement(&self) -> ApprovalRequirement {
let caps = self.capabilities();
if caps.contains(&ToolCapability::ExecutesCode) {
ApprovalRequirement::Required
} else if caps.contains(&ToolCapability::WritesFiles) {
ApprovalRequirement::Suggest
} else {
ApprovalRequirement::Auto
}
}
Whatever executes code demands approval; whatever writes files suggests approval; everything else passes automatically. The approval level is not an isolated knob each tool declares on its own — it is derived from the capability set. A new tool only has to declare its capabilities honestly, and the gate positions itself. Want one finer grade based on the input? There is also the approval_requirement_for_input hook: same code-running tool, and rm and ls can receive different treatment.
Two more methods on the interface are quiet but pivotal: validate_input performs semantic validation in one place before approval, and defer_loading registers a tool without sending it to the model, leaving activation to on-demand tool search — the catalog-slimming trick covered in Article 9.
The outermost ring is the seam layer, ToolDispatcher (crates/agent-runtime/src/tool_dispatch.rs:53-105): each turn, the engine holds only this Box<dyn ...> and queries tools through an "eager, input-independent" metadata snapshot (ToolMetadata), never depending on the TUI's concrete registry.
The motive is written in the module comment: ToolContext carries roughly thirty host-coupled service fields (terminal, notifications, task manager, ...) and "cannot live in the runtime crate" — if the engine core is to stay unpolluted by terminal details, it must look at tools through an interface.
Read the three layers together: the framework layer defines "can be executed," the production layer defines "how it is governed," and the seam layer defines "governed without seeing the details." Each layer knows only what it ought to know.
Unified Registration: All Corners of the World, One Vec
Every tool ends up as an Arc<dyn ToolSpec> in the same registry: built-in tools implement it directly, MCP tools get wrapped by an adapter, extension tools cross over via the ExtensionToolSpecAdapter bridge. A subagent's tool surface is built by the same builder as its parent's — the comment calls this "keeping them in lockstep" — which is why a subagent can recursively spawn subagents.
MCP capability verdicts start from a conservative place: only discovery helpers with list_mcp/read_mcp in their names count as read-only; everything else is treated as "needs approval, needs network" —
"Conservatively treat MCP tools as requiring approval and network access unless they're known discovery helpers."
—crates/tui/src/tools/registry.rs:867-925, module comment
The registry's entrance has a fail-closed checkpoint (crates/agent-runtime/src/tools/registry.rs:484-642): tool names must match ^[a-zA-Z0-9_-]{1,64}$, and the schema root must be an object.
A non-compliant tool neither panics nor 400s — it gets replaced by a FailClosedTool whose description reads "Unavailable: this tool failed to initialise and is fail-closed. Do not call it." Any call to it returns unavailable, forever.
The comment names the reason: if a bad plugin's definition blows up mid-turn, it "would either panic mid-turn or trigger a 400 that disables every tool" — one bad tool's crime must not bring collective punishment down on all the tools.
Fifty weapons sounds imposing, but "many" is itself a disease: past a hundred tools, even the most advanced model starts fumbling multiple-choice questions; and flattening them all into the context both eats tokens and lets every change to the tool set puncture the KV Cache (Article 9's fingerprint discipline).
The industry's answers come in three layers, each more "on demand" than the last. The first layer is indexing: Cursor syncs MCP tool descriptions into folders, so the model sees only a name index by default — an A/B test showed total token consumption on MCP-related tasks dropping 46.9%; Anthropic's retrieval-based pre-filtering screens a batch of candidates by semantic similarity before injection, lifting tool-use accuracy from 49% to 74% in settings with over a hundred tools.
The second layer is API nativization: OpenAI's tool_search, CodeSmith's defer_loading, Codex CLI's BM25 retrieval on by default — the static prefix keeps only tool names and one-line blurbs, and the full schema is appended to the end of the context after the model requests it on demand, whereupon it stays fixed in its original spot in the trajectory and keeps hitting the cache (a direct corollary of Article 4's prefix immutability).
The third layer is active discovery: MCP-Zero provisions no schemas at all; the Agent declares "what capability I need" while thinking, and the system routes by semantics in two hops — server first, then tool — matching and injecting from thousands of candidates, reporting token savings of roughly 98%.
CodeSmith sits between the first and second layers: defer_loading registers a tool without sending it to the model, and activation follows the two-pass approach from Article 9 — resident tools up front, tools activated mid-flight appended to the tail of the catalog, leaving the cached byte prefix unbroken.
The Dispatch Pipeline: From Stream to Backfill
From the moment the model emits a structured tool_use to the moment results land back in the context, the full pipeline looks like this:
SSE stream
├─ ToolUse block starts: record id/name, arguments empty (engine/turn/stream.rs:544-585)
├─ InputJsonDelta: assemble input_buf fragment by fragment
│ (comment: the argument JSON is never shown to the user, only buffered and assembled)
│ read-only tools can speculatively start right here (stream.rs:202-236)
├─ block ends: finalize_tool_input parses the arguments
▼
planning (sequential) (engine/turn/batches.rs:160-230)
├─ truncation gate: when the turn was cut off by the output limit, calls whose
│ arguments failed to parse are refused outright and told to resend (StrictOnly, Article 22)
├─ loop_guard: same name + same arguments, 3rd time = blocked outright
├─ approval decision: by capability or by input
▼
batching (engine/dispatch.rs:339-362)
├─ consecutive parallel-safe read-only tools cluster into a concurrent batch
├─ everything else becomes its own serial batch
▼
execution
├─ concurrent batch: FuturesUnordered, started in order, finished in reverse (LIFO pairing)
├─ serial batch: approval gate (Event::ApprovalRequired → block waiting on the user,
│ cancel token guaranteed to win via a biased select!)
├─ RwLock: parallel-safe takes the read lock (overlap allowed), serial tools take the
│ write lock (exclusive); tool-calls-tool reentry skips the lock to prevent
│ deadlock (crates/tools/src/lib.rs:321-388)
▼
post-processing (sequential, by index) (batches.rs:577-666)
├─ loop_guard records success/failure (warn after 3 consecutive failures / halt at 8)
├─ LSP diagnostics collection, read_file result observation
└─ backfill: network-sourced results first get wrapped in an <external_content> delimiter
(batches.rs:642, covered in depth in Article 19), then push the ToolResult block
Two details along this pipeline deserve a pause. The first is the speculative early start: while the arguments are still streaming in, tools already judged read-only set off early — by the time the model finishes its sentence, the file has already been read. The bet is that "the second half of the arguments won't reverse the read-only verdict," and for tools like read_file that bet is a sure thing.
Network tools, however, were recently shown the door out of this casino (stream.rs:154-160): after the network policy's default changed to Prompt, a network tool's per-input approval level could be Required, while speculative dispatch happens before the approval decision — a static approximation cannot see a policy that varies by host, so it fails closed.
The comment does the accounting plainly: hosts the policy allows lose nothing — the call merely shifts from "speculatively early" back to "initiated normally at execution time." The moment approval semantics change, the list of what may start early must be recomputed to match.
The second is the approval fingerprint — one tight, one loose (crates/agent-runtime/src/tools/approval_cache.rs:141,170): rejections use an exact fingerprint — refusing rm -rf /tmp/x shouldn't incidentally suppress some other, different call — while approvals use a looser arity-aware fingerprint: once you have approved cargo build, cargo build --release no longer interrupts you within the session.
Strict against bypass, lenient to spare you interruption — two rulers pointing in opposite directions, each governing its own scenario.
Behind the concurrent-batch discipline stands a textbook principle: every tool should declare whether it supports concurrent execution, defaulting to no (fail-safe); failures propagate only within the same batch of parallel calls and never escalate to the parent operation.
Translated into a concrete scene: read three files at once, and if one of them doesn't exist, only that one reports an error while the other two return as usual — and the whole task certainly must not abort over it.
CodeSmith lands this in two places. Concurrency eligibility is by declaration: tool_plan_is_parallel_safe (engine/dispatch.rs:335) requires read_only && supports_parallel, plus no-approval-needed and non-interactive — supports_parallel is the tool's self-reported concurrency switch, and its default means no parallelism.
The failure boundary is by structure: concurrent batches reclaim results by pre-assigned slot, with the batches.rs comment calling it "Index-preserving drain — completion order is irrelevant" — a single future's failure pollutes only its own slot; neither its batch-mates nor the whole turn are punished alongside it. Not one word of this principle is written in documentation — it all grows in the predicates and the slots.
Fault Tolerance: Treat the Model's Shaky Hands as a Constant
When the model goes off-protocol, stripping handles it (Article 1); when it stays on-protocol but staggers, the repair ladder takes over.
Argument repair (crates/agent-runtime/src/tools/arg_repair.rs:1-15). DeepSeek sends tool_calls.function.arguments as streaming increments, and the module comment records the two common deaths: an SSE chunk boundary slicing through the middle of a JSON string, leaving a trailing comma or an unclosed bracket after reassembly; and certain local backends emitting literal control characters inside string values.
The repair ladder has five rungs: strict parse → strip control characters from strings → remove trailing commas before }/] → close the unclosed brackets → strip the surplus closers. The cap is 1 MiB; beyond that, repair is refused. The fallback always succeeds: when every rung fails, return the empty object {} — dispatch can always proceed, at the cost of merely a missing field reported on the tool side. But fallbacks are situational: when a turn was truncated by the output limit, the ladder is forcibly shut (ArgSalvagePolicy::StrictOnly, engine/dispatch.rs:167) — half a sentence must never be repaired into an executable command — and the truncated call is refused execution and asked, as-is, to be resent (Article 22). Shaky hands are the norm, and the norm deserves a handrail.
Name repair. When the model gets a tool's name wrong, a resolution ladder takes over (tools/registry.rs:394-448): exact lowercase → hyphen to underscore → CamelCase to snake_case → strip the _tool suffix → fuzzy prefix match.
Still no hit? Even the error doesn't come empty-handed (engine/tool_catalog.rs:469-502): candidates within edit distance ≤3 are listed as "Did you mean: ...?". Called a deferred-loading tool that was never registered? The hydrated result returned doesn't just report the error — it appends the correct field-name mapping (old_string → search, new_string → replace) plus "retry_required": true.
The error message is itself the textbook — everything needed to correct the next call is right there in that one message.
Schema sanitization (tools/schema_sanitize.rs:1-33): semantics-preserving transformations before sending to the model — anyOf:[X,{type:"null"}] folded into a nullable X, bare objects given an empty properties, dangling required entries pruned — purpose-built for the dirty schemas various MCP servers cough up getting rejected by the API in strict mode.
Hostility as a Virtue
Parsing tolerance has one line it must never cross: you may repair a shaky hand, but you must not admit a forgery. The module comment at crates/agent-runtime/tests/protocol_recovery.rs:1-23 is worth reading in full:
//! These tests exist to keep the engine hostile to fake tool-call wrappers
//! (XML/Replit/markdown pseudo-calls in assistant text). Their job is to make
//! sure that:
//! ...
//! 2. The legacy text-based `tool_parser` does NOT treat the newer
//! `<function_calls>` wrapper as a real tool call — only the legacy
//! `[TOOL_CALL]` and `<invoke>` shapes ever produced structured calls, and
//! nothing should silently re-enable text-based execution.
//!
//! The point is that protocol drift in the model output should be visible (we
//! still strip it and emit a status notice), not silently turned into tool
//! execution.
The old text parser tool_parser.rs is still in the repo, its module header introducing itself: "the engine no longer invokes this parser" — retired but kept on the books. Its living legacy is the boundary of that iron law: crooked arguments may be repaired, because that is damage inside the protocol; a "call" whose format is wrong altogether must be stripped, because that is a forgery outside the protocol — repairing the latter amounts to opening a channel for the attacker. Article 19 will show this line's full significance for security.
The Economics of Results
The backfill of tool results connects directly to the context ledger of the past few installments. Shell output is capped at 30,000 bytes (crates/agent-runtime/src/tools/shell_output.rs:17-28, comment "30KB like Claude Code"), but the split is not one-size-fits-all: 22K preserved at the head, 8K at the tail — the tail budget is reserved for error messages and test summaries, precisely the high-value content that shows up most often at the end.
The amputated middle is not discarded forever; it goes into an on-disk envelope: the moment truncation triggers, the full output is atomically written to ~/.codesmith/tool_outputs/ (spill_full_shell_output at shell_output.rs:70), while head and tail stay in the context as before.
A retrieval pointer is appended to the truncation marker (Full output saved; call retrieve_tool_result ref=<id> to view it.); the model copies it verbatim and the middle section comes back. All three truncation points — foreground, opensandbox, and delta polling — are wired up; files are kept for 7 days and cleaned up at startup.
The spilling itself has four guardrails. First, polls in the Running state do not spill: otherwise every poll would rewrite the accumulated output wholesale — O(polls × bytes) of disk churn — and only the poll that observes the final state writes the final view.
Second, no spilling once the task has been evicted (stale): the stale details retain only the saved tail, and overwriting a previously spilled full output with that would be a downgrade — better to withhold the retrieval pointer. Third, the synthetic spill id incorporates the pid: the tool_outputs directory is shared across processes, and millisecond-plus-serial would collide between two processes started in the same millisecond.
Fourth, write failures went from silent degradation to a warn! on the record: the retrieval pointer promises "the full output is on disk," and a silent failure would quietly break that promise.
As for the 100 KB spill threshold for general tool results (truncate.rs:56), the pipeline has always been there but long had no production caller — it was the shell's 30KB budget that got wired up first. The tool catalog's serialized results are cached in a OnceLock and sorted by name (prefix caching again); oversized output goes down the summarize route, and oversized results become var_handle tokens (covered in detail in Article 8) — no need to repeat it here.
Conclusion: Everything Hanging Off the Slot
Look back at what this slot is worth: approvals derived from capabilities (a second regret medicine beyond Article 7's snapshots), concurrency declared through read-write locks, the catalog earning its cache hits through sorting, a bad tool fail-closed without collective punishment, shaky hands given a five-rung handrail, forgeries met with hostile stripping, result backfill carrying its own economics — every cross-cutting concern hangs on the hooks of one interface, and tool implementers write nothing but business logic. That is the full meaning of "fifty weapons, one slot": abstraction is not about writing less code — it is about writing each act of governance exactly once.
But the slot only guarantees how tools "get governed." The manual that comes with each individual weapon — why read_file talks you out of cat, why edit_file errors on multiple matches, why fetch_url treats 169.254.169.254 as a mortal threat — is another craft altogether.
Top comments (0)