CodeSmith Tool Internals: Close the Syntax Gate Before Writing to Disk

Tool Internals: Shut the Syntax Gate Before Writing to Disk

Source version of CodeSmith: v0.5.0 (commit 3a74c82f). All paths are relative to the repo root; line numbers refer to this version.
Intended audience: anyone who has watched an agent write fn main() { as fn main() ｛.

When a cheap model writes code, the most common wreck is not a logic error but a syntax error: a bracket gone missing, a line severed at the halfway mark, two files' contents welded into one. CodeSmith's countermeasure is to install a syntax gate before the write to disk — good files are not allowed to be written bad, and a broken workspace still gets repaired as usual. Four disk-writing tools share one gate; the core module runs 455 lines.

Why Before the Write to Disk

Start with the three lines of defense the engine already had. LSP diagnostics: after an edit, ask the language server once and inject the compile errors back into the conversation (config.example.toml:912-917). Article 15's claim verifier: replay the test commands at end of turn and reconcile. run_tests: the model runs the tests itself.

These three lines share one trait — they are all after-the-fact defenses. By the time they raise the alarm, the broken file is already on disk: the side-git snapshot is hauling an extra piece of garbage, the code index may already have swallowed it, and the model's next-round patch is anchoring itself to syntax that snapped in half.

Article 17 summed up the trilogy of tool discipline: the manual teaches, the error message corrects, the engine blocks. The syntax gate is a new instance of "the engine blocks" — and its checkpoint sits further upstream than every predecessor: before the disk.

A Regression-Style Gate: Judge the Change, Not the State

The gate's contract is one sentence, in the second paragraph of the module docs (crates/agent-runtime/src/tools/parse_gate.rs:5-9): a write is rejected only when the file parsed successfully before the edit and the new content does not. A file that was already broken, and a new file that does not exist yet, can never be locked out by this gate.

The core logic is under thirty lines (parse_gate.rs:47-75):

pub fn gate(path: &Path, old: Option<&str>, new: &str) -> Result<GateOutcome, ToolError> {
    use Lang::*;

    let Some(lang) = gated_language(path) else {
        return Ok(pass_through(new));          // Not a gated format: pass through
    };

    // Regression-only: compare parseability, not syntax trees.
    let old_was_valid = old.is_some_and(|old| check(lang, old).is_ok());
    if old_was_valid && let Err(failure) = check(lang, new) {
        return Err(reject(path, lang, new, failure));   // good→bad: reject
    }
    //...... After passing: rustfmt re-normalizes (see the next section)
}

Plain-English translation: the gate compares the change in a single boolean — parseability — not syntax trees, and certainly not the merit of the file. Good turning bad: blocked. Bad turning worse: passed. Something from nothing: passed.

Why let "bad turning worse" through? Flip it around and the answer is obvious: if the gate judged state — rejecting every write that fails to parse — then a workspace that was already broken could never be repaired again. A cheap model must be able to repair a broken workspace; that is its day job. A gate that locks out repairs is worse than no gate at all.

The tests nail the boundary down hard (the parse_gate.rs test module): already_broken_rust_is_never_locked — a broken file written even worse still passes; new_file_creation_is_allowed — a new file whose content is broken from birth also passes; first creation is never blocked.

cargo_lock_is_exempt passes through without even looking at the format; ungated_extensions_pass_through — the world outside the gate's jurisdiction carries on as before.

Only three formats fall under the gate's jurisdiction (parse_gate.rs:107-111): Rust goes through syn::parse_file; TOML and JSON go through their own standard parsers. Cargo.lock is explicitly exempt — machine-generated, bulky, and edited by no human hand.

The House Rules of the Rejection Letter

The error raised on interception follows the rules Article 17 laid down: state exactly what was rejected and where it snapped, paste the offending line back verbatim, then tell the model what to do next (parse_gate.rs:217-226):

ToolError::execution_failed(format!(
    "Parse gate rejected write to {}: this file parsed as valid {} before the edit, \
     but the new content does not — line {}, column {}: {}\n{:>4}│ {}\n\
     The file was NOT modified. Fix the syntax error and send the edit again.",
    //......
));

Plain-English translation: the rejection letter carries its own localization — line number, column number, the offending line as written (truncated at a character boundary when it exceeds 200 characters). The coordinate conversion has two traps to handle: proc-macro2's fallback spans carry real line/column numbers only when span-locations is enabled; and syn numbers lines from 1 but columns from 0, so both must be converted before output (parse_gate.rs:130-146).

serde_json's error tail already carries "at line L column C", and it must be stripped off so the location is not stated twice (parse_gate.rs:172-178).

The test rust_regression_is_rejected_with_line_info asserts it item by item: the error must contain "valid Rust", must contain "line 2", must contain "NOT modified". The error message is itself the model's input for the next round — the more precisely it is written, the fewer detours the next round takes.

rustfmt Re-Normalization: For the Next Anchor

After the gate passes there is one bonus move: if this is a Rust file and it was rustfmt-clean before the edit, the content to be written first goes through rustfmt --edition 2024 (parse_gate.rs:257-283).

The motive hides in a single comment: keep subsequent patch anchors stable. Article 17 described edit_file's anchoring pattern — the patch the model supplies must find its anchors inside the file; every time formatting drifts, the anchor's matching surface changes with it. A file that was clean before the edit comes out normalized clean after it, and the anchor coordinate system does not move.

This stretch of implementation has the highest defensive density in the whole article: the rustfmt binary is probed once per process, and the probe itself is capped at 5 seconds (a hung rustfmt --version must not clog the entire write path, parse_gate.rs:240-242); formatting is capped at 10 seconds; any failure — binary missing, nonzero exit, timeout, non-UTF-8 output — silently skips normalization and writes the original text to disk.

The module docs draw the authority lines cleanly (parse_gate.rs:14-18): the gate alone decides whether to reject; rustfmt only handles layout and holds no power to block. Nor does it hide the fact that normalization changed the content — the tool result carries a disclosure note telling the model "what you wrote has been reformatted".

The tests are just as tidy: rustfmt_normalizes_only_clean_files asserts that a file already messy before the edit keeps exactly the layout the model gave it; missing_rustfmt_disables_normalization asserts that without rustfmt, normalization steps down entirely.

One piece of trivia earned a place in a test comment: syn treats an empty file as valid (a Rust file with no items), so truncating a file down to the empty string is allowed — the gate blocks broken syntax, not the wiping of a codebase (empty_rust_is_valid_and_passes).

Closing: Inside and Outside the Gate

Chain the mechanism into a single thread:

1. The four disk-writing tools (write_file / edit_file / apply_patch / fim_edit) share the same gate before anything lands on disk;
2. The gate looks only at the change in parseability across three formats (Rust/TOML/JSON): good→bad is rejected, everything else passes;
3. The rejection letter carries line number, column number, and the original line; the file stays as it was, and the model is told to fix the syntax and send again;
4. After passing, a Rust file that was clean before the edit is re-normalized by rustfmt, preserving the anchors for subsequent patches;
5. Any failure in normalization silently steps aside — the gate alone holds the power to reject.

Three properties, and their prices:

1. Regression-style: a broken workspace can always be repaired — the price is that the world beyond the three formats (Python, TS, ...) runs naked, out of the gate's reach;
2. The gate sits before the disk: broken content never lands on disk and never enters a snapshot — the price is one extra parse on every write, so the parsing cost of large files enters the write path (thankfully, the syns of the world are fast);
3. Separation of powers between gate and layout: the power to reject is divorced from the power to normalize — the price is that rustfmt's layout may differ from what the model expected, and a disclosure note is needed to keep the trust intact.

The syntax gate catches the model's clumsiness. But tool results and file contents hold something else besides — malice that someone else planted with care. That is no longer a problem a handrail can solve; that is a problem for a city wall.