CodeSmith Frugal Architecture: The Three Zones and Their Deed

Frugal Architecture: The Deed to the Three Zones

Source version of CodeSmith: v0.5.0 (commit 3a74c82f). All paths are relative to the repo root; line numbers refer to this version.
Intended audience: anyone who has read Article 4 and assumed the three-zone model was just a theory.

Article 4 ran the numbers: drift the system prompt by a single byte, and every token after it gets repriced at full fare — the foundation of the prefix cache is "the byte-level prefix does not move." At that stage, the three-zone model was still mostly theory and observation: the fingerprint could perform the autopsy, /cache stats could show the hit rate, but who rewrote the conversation history, and how, rested entirely on the code's own good behavior.

Phase 1's Confession

This landed in three steps, and the middle stretch of that history is worth telling. Step one was observation: /cache stats landed first, letting people see the cache hit rate. Step two was Phase 1: six types were built — PinnedPrefix, FrozenPrefix, PrefixDrift, AppendLog, TurnScratch, ThreeZoneRequest — but not one of them was wired into the engine; all were marked #[allow(dead_code)] and left standing as scaffolding.

In the Phase 1 era, the /cache zones output carried a confession: the three-zone contract was "not yet wired." By Phase 2, that confession had been nailed into its grave by a test — cache_zones_reports_wired_contract in crates/tui/src/commands/debug.rs (debug.rs:928) asserts that the output no longer contains "not yet wired" or "Phase 1 foundation", and a one-line comment inside the test closes the case: "The Phase 1 confessions are gone."

Why two steps? Because welding all six types into the sixteen-thousand-line executor main loop in one go would bury any behavioral drift inside a diff you cannot read. Phase 2's solution: first plant a boundary stone of byte-level regression tests, then touch the wiring (see the "Boundary Stones" section below).

Three Zones, Six Types

The three-zone division, quoted straight from the module doc's original diagram (crates/agent-runtime/src/prompt_zones.rs:1-16):

┌─────────────────────────────────────────┐
│ PinnedPrefix (frozen after construction) │ ← system prompt + tool catalog
│   combined_sha256 computed at freeze()   │   cache hit candidate
├─────────────────────────────────────────┤
│ AppendLog (append-only)                  │ ← conversation history
│   push() only, no insert / remove / edit │   preserves prefix of prior turns
├─────────────────────────────────────────┤
│ TurnScratch (ephemeral)                  │ ← per-turn composition staging,
│   cleared at every turn boundary         │   committed to the log pre-request
└─────────────────────────────────────────┘

The middle zone carries the load. AppendLog is now the storage type of the conversation history itself (prompt_zones.rs:277-294), and its rustdoc contains the most beautiful sentence in the entire design:

"There is deliberately no way to insert, remove, truncate, or index-write: the append-only prefix that DeepSeek's KV cache matches against is a property of the type, not of discipline."

In plain terms: the type deliberately offers no way to insert, remove, truncate, or write by index — the append-only prefix that the KV cache depends on for its hits is a property of the type, not of the team's discipline. Discipline collapses on some late-night overtime crunch; a compile error does not.

Reads go out through a shared Deref<Target = [Message]> — DerefMut is deliberately left unimplemented, otherwise in-place rewrites like swap and sort would compile again. So &log, log[i], log.to_vec() all work as usual, while every mutating method of Vec flatly fails to compile here.

The Escape-Hatch Policy: Every Demolition Must Sign Its Name

append-only has licensed exceptions: compaction, context-overflow recovery, /edit rollback, session restore, cycle reseeding. Exceptions are not the scary part; anonymous exceptions are. Phase 2 gathered every exception into a single entry point — AppendLog::rebuild — whose parameter is an enum that forces the caller to state a reason (prompt_zones.rs:212-244):

pub enum RebuildReason {
    ManualCompaction,
    AutoCompaction,
    Purge,
    MicroCompaction,
    ContextOverflowRecovery,
    FrontTrim,
    VerifyAndReplanReset,
    EditRollback,
    SessionSync,
    CycleReset,
    Runtime(&'static str),
    //......
}

In plain words: every time the conversation history is rewritten wholesale, the caller must sign its name at the type level, and the reason — together with the message counts before and after the rewrite — lands in the audit record. /cache zones can now answer "why was my cache reset" instead of leaving it a mystery.

The companion sentence deserves a frame on the wall (prompt_zones.rs:44-45): these sites punch through the KV prefix cache by design; the type system's job is to make it impossible for this to happen "by accident."

On the executor side, all 9 old clear-and-repush sites migrated onto this channel (in the commit message's own words: "The 9 runtime clear+repush sites migrate to it"), and the host-side rewrite entry points were corralled in right after — which also cured an old ailment: under the old pattern, events either never fired or fired N times in a row; now it is one atomic blood transfusion with exactly one TranscriptRebuilt. The audit records keep the 8 most recent entries in the TUI (crates/tui/src/tui/app.rs:666).

One Fingerprint, One Source of Truth

Phase 2 also put an end to the fingerprint's dual-track arrangement. Before, the request path froze one fingerprint while the stability observer computed another; now PrefixStabilityManager consumes the very same FrozenPrefix the request path freezes at every step (crates/agent-runtime/src/prefix_cache.rs:32-38) — "one fingerprint implementation, one source of truth."

The fingerprint's algorithm got an upgrade too: the tool-catalog digest went from "hash of the name list" to "hash of the full JSON, sorted." The motive is attested by a test comment (around prefix_cache.rs:292): the old name-hash missed an entire class of drift — a tool kept its name but changed its description or schema, so the prefix had actually changed while the fingerprint noticed nothing. The test verify_detects_schema_change (prompt_zones.rs:592) nails down precisely this class.

Boundary Stones: Byte-Level Regression Tests

Finally, the most crucial insurance: the wiring must prove zero behavioral difference.

The test three_zone_assembly_sends_verbatim_log_snapshot (crates/agent-runtime/src/engine/host_executor.rs:4683) serializes the old and new assemblies and compares them byte for byte; the failure message is a one-line verdict: prompt construction is non-deterministic — first diff at byte N.

The comment is just as much a design manifesto: zoned wiring changes the ways a request can be expressed, never the bytes that go on the wire. Transparent retries are therefore cache-safe — the same zone inputs always serialize to the same bytes.

An Appendix to the Deed: The Order of Config Resolution

The same feature branch also landed a document (commit 56503762, merged via the next PR) that gathered the config-precedence rules scattered everywhere into one ordered table (docs/CONFIGURATION.md:46-57): CLI flags > config files (a profile overrides the top level, a project overlay overrides that in turn) > secret fallback (config > keyring > env) > managed config > requirements validation at startup.

This appendix and Article 0's preset tiers are two sides of one coin: presets are the fill-in-the-blanks baseline; this table is the adjudication order for when the same key collides. The project overlay is only read after the workspace has passed the startup trust boundary, the permitted keys are narrowed down to a whitelist, and approval_policy and sandbox_mode may only be tightened, never loosened — project-level config wants to swap the provider or the base_url? The doc cites #417 and refuses outright: otherwise a malicious project file could funnel the user's credentials off to a look-alike endpoint.

Conclusion: From Moral Suasion to Law Enforcement

String the mechanisms into one chain:

1. The conversation history's storage type becomes AppendLog: push is the only everyday mutation; Vec-style rewrites no longer compile;
2. Every licensed wholesale rewrite goes through rebuild(RebuildReason); the reason lands in the audit, queryable via /cache zones;
3. Every request is assembled through ThreeZoneRequest: messages can only be a slice of the log plus a scratch tail;
4. Requests and observation share the same frozen prefix, and the fingerprint is sensitive to tool-schema-level changes;
5. Byte-level regression tests stand as boundary stones: the wiring changes expressiveness, not bytes.

Three properties and their prices:

1. Property instead of discipline: the compiler stands guard over append-only on the team's behalf — the price is that adding a new legitimate rewrite scenario means first adding an enum variant and leaving a name in the audit;
2. Real-name demolition: a cache bust goes from incident to bookkeeping entry — the price is that the lovely zero-bust state exists only in sessions that never touch history;
3. Zero-intervention observation: drift detection reports but does not intercept — the price is that by the time drift is spotted, one step's money is already spent.

With the deed signed, the frugality main line draws to a close: the economics of caching wrote the law (04), and the type system enforces it (this installment). The next question moves to a different plot of ground — the model's main context is both expensive and crowded, so how do the big materials get in, and get computed: the next installment starts by letting the Agent cook in a private kitchen of its own.