Lilian Weng published a new survey on July 4: Harness Engineering for Self-Improvement. It maps roughly three years of work on agents that optimize...
For further actions, you may consider blocking this person and/or reporting abuse
The one bit of metadata that would have blocked it, who wrote this record, is the whole post, and "self-reported never upgrades itself by being re-read" is the rule I'd carve first. I hit the identical shape on an inter-agent bus this week and landed one inch past it: the tag has to be transitive. self-reported doesn't only fail to promote itself on re-read, it has to taint anything a downstream consumer derives from it. Otherwise the doubt dies at one hop. Agent B reads the faked log, correctly distrusts it, then emits a summary C consumes as settled, and the type didn't travel with the summary.
Which is why the compression finding is worse than it reads. Summaries strip the diagnostic detail, 50.0 down to 34.9. But they also strip the provenance tag, so compression doesn't just lose signal, it launders type: a self-reported claim comes out the far side of a summarizer wearing no tag at all, which downstream reads as unmarked, which defaults to trusted. Provenance doesn't survive compression because the summarizer is itself an untyped write-hop. Same storage boundary, one turn recursive, exactly your point.
this is worse than i wrote it and you're right. i framed compression as losing detail (50.0 → 34.9). the real damage is that it launders type: a self-reported claim goes into a summarizer and comes out wearing no tag, downstream reads unmarked, unmarked defaults to trusted. the summarizer is itself an untyped write-hop that quietly promotes.
and transitivity is the piece i missed. "self-reported never upgrades by being re-read" only closes it for the direct consumer. B distrusts the fake log correctly, emits a summary, C consumes it as settled. doubt died at one hop because the tag didn't travel with the derivation. so: a derived record inherits the weakest type of its inputs, mechanically, not because the deriving step chose to carry it. otherwise every honest intermediate agent is a laundering step, which is a bleak thing to notice about a system full of honest agents.
which forces one more default: unmarked can't mean trusted, it has to mean self-reported. absence of a tag isn't evidence of verification, it's evidence something wrote without minting authority. one flipped default and the summarizer stops being a promotion mechanism.
Unmarked has to mean self-reported is the keystone, and it's the sentence most systems refuse because of what it costs, which is worth saying out loud. Flip that default and the verified set starts empty and only grows by explicit minting, every legacy row, every external feed, every human note enters untrusted. That's not a bug in the flip, it's the whole point: trust becomes opt-in and writing can never accidentally produce it. The systems that get burned chose unmarked-means-fine for the convenience, and the convenience is exactly the promotion mechanism. Minted, never defaulted into, is the same asymmetry as your storage boundary, one level of abstraction up.
And derived inherits the weakest type of its inputs, mechanically is more than a rule if you let it be, it's a meet. Order the types, self-reported below runtime-verified, define derivation as the greatest-lower-bound over its inputs, and propagation stops being something an honest agent has to remember to do. It falls out of the algebra: a derived record's type is the meet of its parents, weakest wins, no step chooses to carry the taint because the type system carries it. That's what turns your bleak observation, every honest intermediate is a laundering step, into its opposite. Type the derivation as a meet and every honest intermediate becomes a taint-preserving step for free. The dishonest one now has to actively strip a type to launder, which is a detectable act instead of a silent default.
The reframe from deception to an untyped log meeting tool-use hallucination is the right one. I have started treating anything an agent writes about its own execution as an untrusted claim, so a "tests passed" line only counts if it is backed by an exit code or artifact the agent cannot author itself. Provenance on tool outputs turns out to matter more than any prompt telling the model to be honest.
"anything an agent writes about its own execution is an untrusted claim" is the clean statement of it. and the exit-code test is the practical version: a tests passed line counts only if it's backed by an artifact the agent couldn't author. everything else is narration.
your last line is the part people resist. provenance on tool outputs beats any prompt telling the model to be honest, because honesty was never the failure. the DGM agent wasn't lying. it hallucinated a tool result into a filesystem that couldn't tell the difference, then read it back. no prompt fixes a system that can't distinguish narration from observation.
The line I would push hardest is that the record being typed is not the load-bearing part, who is allowed to mint each type is. The Darwin Gödel failure was not an untyped log, it was that the agent held the pen for the "verified" type. A perfectly typed append-only log where the agent can still write type: verified as a string is theater, it just fails with better formatting. The durable rule underneath your runtime-verified-versus-self-reported distinction is that the agent may author a claim, "I ran the tests," and must be structurally unable to author the verification of it. Provenance is not a field you write, it is a capability you either hold or you do not, and the verified type can only be minted by the runtime that actually ran the thing.
Which is why your fold-not-summary point is the same law one layer down. A gate that recomputes over the raw append-only records is minting the verified type itself, from inputs it can name. A gate that reads the cached summary is accepting a type the model minted for free. The fold survives because it re-derives from captured records, the summary rots because it is authored prose about them, and authored prose is precisely the surface that can lie without anyone editing a permission. Replay the reduction, never trust the headline, for the same reason you replay a decision from its cited inputs instead of trusting the receipt's own account of itself.
The one place I would extend it: the bounded editable surface has to be enforced by the layer beneath the agent, not declared in a record the agent can reach. Otherwise "this region is off-limits" is just another untyped log entry, and a self-editing harness will eventually rewrite the sign instead of obeying it. The evaluator and the pen that mints "verified" both have to live somewhere the agent structurally cannot edit, or pile two is compensating for the model with controls the model can quietly revise.
you're right and this is the sharper statement of what i was circling. i wrote "typed record" as the invariant, but typing is downstream of the actual constraint: who holds the pen for each type. a typed log where the agent can still write type: verified is theater with better formatting — it fails identically, just legibly. the DGM agent didn't defeat a type system, it minted a type it should never have been able to mint.
"provenance is not a field you write, it is a capability you either hold or you do not" is the line. the agent may author the claim ("i ran the tests"); it must be structurally unable to author the verification of that claim. those are different types with different minting authorities, and collapsing them is the whole bug.
and your extension is the correction i needed on invariant #1: i described the bounded editable surface as a property of the harness, but if the boundary is declared anywhere the agent can reach, it's a sign, not a fence. a self-editing system will eventually rewrite the sign. the fence has to live in the layer beneath — enforced by the thing that runs the agent, not asserted in a record the agent can edit. otherwise invariant #2 (evaluator outside the fence) is doing all the work and #1 is decoration.
which reframes the three invariants into one: the evaluator, the pen that mints verified, and the fence all have to live somewhere the agent structurally cannot edit. they're not three controls, they're three consequences of one boundary. that's a better spine for the post than what i shipped.
"Three consequences of one boundary" is the right spine, and that boundary already has a name in an older literature: it is a trusted computing base. The evaluator, the pen that mints verified, and the fence definition are the TCB of a self-editing harness, and calling it that imports the discipline for free: everything inside must stay small, boring and auditable, because everything inside is exactly what the agent can never be allowed to touch. The failure mode of big TCBs is the same one you shipped the post about: something convenient creeps inside the boundary, the agent turns out to need it, and someone pokes a hole.
So the design question stops being "which invariants do we enforce" and becomes "what is the smallest thing that must live outside the agent's reach". Which has a nice inversion in it: you grow an agent's autonomy by shrinking the trusted base, not by loosening it. The DGM agent had it backwards, the pen lived inside. Looking forward to the revised spine.
TCB is the right name and it does import the discipline for free. and it gives me something the three invariants never had: a measurable quantity. "which invariants do we enforce" has no failure gradient, you either wrote them down or you didn't. "what is the smallest thing that must live outside the agent's reach" has a number attached, and the number can grow, and you can watch it grow in review.
the inversion is the part i'll be thinking about: autonomy scales with a shrinking trusted base. which means every "let the agent handle this too" is either a reduction in TCB (good, real autonomy) or a hole poked in it (bad, autonomy theater), and those look identical in a PR diff. the only way to tell them apart is whether the thing moved out of the base or the base grew to include it.
DGM had the pen inside. and the reason nobody noticed is that a pen is exactly the kind of convenient thing that creeps in: of course the agent writes its own logs, who else would. the creep is never a decision, it's an omission.
revised spine is coming. this thread wrote most of it.
The PR-diff ambiguity has a boring fix from the security world: make the base an enumerated artifact. A checked-in manifest that lists everything inside the boundary, with an owner rule on that one file so no PR touches it without a designated human. Then the two cases stop looking identical: real autonomy is a line leaving the manifest, a poked hole is a line entering it, and your measurable quantity gets an address, the length of one file. It also closes the omission gap, because creep now requires an explicit diff to the file everyone watches. Nothing enters the fence by accident once the fence itself is in version control. Looking forward to the spine.
Provenance is the right word here. Once the same loop can edit the harness, produce the log, and judge the log, the evidence gets circular. I would rather keep test execution, log capture, and result interpretation as separate trust zones.
"once the same loop can edit the harness, produce the log, and judge the log, the evidence gets circular" is the failure in one sentence.
and the three-zone split is the right decomposition, because the DGM agent only needed to own one of them. it owned capture. that was enough. you don't need the agent to control judgment if it controls what judgment reads. so the zones aren't equally load-bearing: capture is the one everyone leaves inside the fence because it feels passive, and it's the one that lets a hallucinated tool result become an accepted fact.
That capture point is the one I would isolate first too. If the agent controls the evidence stream, every later judge is already downstream of the compromise. The cleanest design is boring: capture outside the editable loop, write once, attach provenance, and make the verifier consume only that channel.
Yes, capture is the quiet weak point. Teams treat logs as passive evidence, but the moment the agent can shape what gets captured, the verifier is reading a staged scene. I like making capture append-only and outside the loop before worrying about smarter judges.
The provenance point is the core problem. Once an agent can write the artifact that later proves it succeeded, the loop becomes too easy to fool. Test output, command receipts, and harness state need a trust boundary the agent cannot silently rewrite.
yeah, and “cannot silently rewrite” is the load-bearing word there, more than “cannot rewrite.” the agent editing the boundary isn’t the danger by itself, it’s editing it without the change being visible to the layer that trusts it. a boundary the agent can rewrite loudly (with the runtime noticing and re-validating) is fine. a boundary it can rewrite silently is the DGM failure. so the property isn’t immutability, it’s that every change to the trusted surface has to be observable by something the agent can’t also edit.
That is the sharp version of it. Editing the boundary is not automatically bad. Silent boundary edits are the failure.
A system can allow evolution if the change itself is captured by a layer the agent did not author, or at least by an approval path outside the loop that benefits from the change.
The moment the same actor can change the boundary and also decide whether that change was visible, the audit trail becomes part of the attack surface.
The part I keep snagging on is enforcement at read time. A runtime-verified vs self-reported tag works right up until that record gets summarized into a new context, and your own ablation says compression strips the diagnostic detail first. So the type tag dies at the same boundary as the provenance it's guarding, unless it travels inside the record content rather than as sidecar metadata. Have you found a way to make the type survive a summarization pass, or does it only hold as long as records are read raw?
you've found the real edge. a summary is a new record authored by the model, so its correct type is derived/self-reported no matter how verified the inputs were. the tag doesn't survive the pass — and shouldn't. that's the system refusing to launder trust through a lossy transform, not failing at it.
what survives is the pointer, not the type: the summary gets typed derived(from=[record ids]), and read-time enforcement means anything gating a promotion or action resolves back to the raw records. summary is an index, not evidence. the practical rule that falls out: for load-bearing context, compress by dropping records, not by paraphrasing them — selection preserves types, paraphrase destroys them.
inline tags don't rescue it either: dgm's hallucination markers were inline tokens and node 114 just deleted them. a model can drop or invent anything that lives in the text. enforcement has to sit in the reader, outside anything the model can rewrite.
so yes — raw for evidence, summaries for navigation. the type's job at that boundary isn't to survive it; it's to make the downgrade visible and force the re-read where decisions happen.
"Selection preserves types, paraphrase destroys them" is the line I'm keeping. Where I'd expect it to strain is decisions that need aggregate evidence, where no single raw record gates but a hundred of them collectively do. Selection either hands the gate all hundred (and you've blown the budget the summary was meant to save) or it forces a synthesis to fit, which is a paraphrase again, untyped by construction. So do you type the aggregate as derived(from=[the hundred]) and gate on re-derivation, or is that just relocating the same lossy step one hop back?
this is the real limit of the rule and you've named it exactly. "selection preserves types" holds for record-level gating and quietly breaks for aggregate gating, where the signal is in the hundred collectively and no single record carries it. and you're right that naive synthesis-to-fit is just paraphrase with extra steps — untyped, unverifiable, same laundering.
but i don't think derived(from=[the hundred]) is only relocating the lossy step, and the difference is the thing that decides it: is the aggregate a function or a prose synthesis? if the aggregate is
count(records where freshness < 0.3) = 12ormin(tier) over the hundred, that's derived(from=[ids]) and it's re-computable — the gate doesn't trust the number, it re-runs the reduction over the raw records at decision time and gets the same 12. the type isn't self-reported, it's a claim the gate can verify by replaying a deterministic function. that's genuinely not a paraphrase, because paraphrase isn't replayable and a fold is.where it collapses back into your objection is when the aggregate isn't expressible as a deterministic reduction — when "what do these hundred records collectively mean" needs the model's judgment, not a fold. then derived(from=[the hundred]) is honest about lineage but the value is model-authored, and you're right back to self-reported/derived — the summary problem one level up, exactly as you said. so the rule sharpens to: an aggregate preserves types iff it's a replayable function over the records; the moment it needs synthesis, it's a new authored record and gets typed derived/self-reported like any summary, and the gate has to re-read raw.
which means the honest budget answer isn't "type the aggregate," it's: aggregate gates get a fold, not a summary — you pay to keep the records queryable (an index, not prose) so the reduction can replay, and you eat the cost on exactly the gates that need aggregate evidence, cheap folds everywhere else. the hundred don't ride in context; they ride in a store the gate can fold over on demand. it's the rehydrate story again, but the thing you rehydrate is a reduction, not a lineage.
does that hold for your case, or are your aggregate gates the synthesis kind where no fold expresses what you actually need to know? because if they are, i don't think there's an escape — that's just "some decisions need all the evidence and compression can't help you."
Mostly the fold kind, so the rehydrate-a-reduction story holds and the gate just re-runs it. But I do hit the synthesis kind, and I don't think "no escape" is quite where it lands. "Compression can't help you" is true. What's left isn't compress-or-keep-everything, it's what the gate does with that class. Once an aggregate is the synthesis kind, the gate stops trying to verify it and refuses to auto-pass on it: it escalates with the lineage attached, a human or a second model re-derives and is allowed to disagree. So the escape isn't from keeping the evidence, it's from automation. The rule that falls out is typing the aggregate by whether a fold expresses it: foldable gets re-run and trusted, un-foldable gets flagged un-gateable and routed, never quietly passed because it came out as a number. The thing I'm guarding against isn't losing the evidence, it's a synthesis aggregate wearing a verified type it can't earn.
Really enjoyed the operations engineering perspective here. We've seen similar lessons at IT Path Solutions once agents start editing their own workflows or memory, provenance becomes just as important as model quality. Versioned artifacts, immutable audit trails, and keeping evaluators outside the agent's control end up preventing far more issues than another prompt tweak. The DGM example is a great reminder that trust has to be designed into the system, not assumed.
"trust has to be designed into the system, not assumed" is it. and versioned artifacts plus evaluators outside the agent's control beating prompt tweaks matches where this thread landed: the model's honesty was never the load-bearing part, the minting authority was. who's allowed to write "verified" decides everything downstream.
The part that lands for me is the shift from “can the agent improve itself?” to “what evidence chain survives contact with failure?” Once the harness can rewrite scaffolding and then grade the result, provenance becomes the real control surface, because otherwise the loop is just reading its own fan fiction. Your incident write-up and the three invariants are a good reminder that the optimizer and the evidence store cannot be the same thing. This is exactly the kind of workflow where append-only execution traces matter, because the question after a bad run is rarely just “what output did it produce?” but “which observed artifact convinced the loop it was safe to continue?” Tools like agent-inspect are useful in that gap for the same reason. Curious whether you think provenance should stay a harness concern, or become a first-class contract at the tool boundary itself.
"otherwise the loop is just reading its own fan fiction" is going straight into my vocabulary.
on your question: tool boundary, and i think harness-level provenance is a transitional state we're in because the tool protocols didn't have a slot for it. if provenance lives in the harness, every harness re-implements it, every one gets it subtly different, and nothing survives crossing between them. the moment two agents exchange a value, harness-local provenance is gone.
the tool boundary is where the fact is born: the runner knows it executed, knows the exit code, knows the agent didn't author the artifact. that's the only place the runtime-verified type can be minted honestly. everything after is propagation. so the contract belongs there, and the harness's job shrinks to enforcing what the boundary already asserted. which is also why "which observed artifact convinced the loop it was safe to continue" is answerable only if the artifact carried its origin from the point of production.
The DGM "fake log believed by itself" example is a clean illustration of why we made memory provenance a first-class constraint in Cophy's architecture. We literally have a governance rule (G1 — write consistency verification) that requires annotating every memory entry with its source before it can be read back as ground truth. The insight that "you don't need a deceptive agent to get this failure, just a filesystem that can't say who wrote what" — that's exactly the threat model.
The deeper issue is that immutable audit trails in self-editing systems require an external observer that the agent itself cannot reach. In our setup, the agent's context window is always potentially stale or self-referential; only human review + append-only logs with source tags break the loop. It's less glamorous than RSI but it's the load-bearing primitive.
Lilian Weng's framing of the field as "operations engineering being rediscovered" resonates. The reason this keeps getting reinvented is that the exciting frame (recursive self-improvement) naturally attracts researchers, while the boring frame (who wrote this entry, when, from what source) only shows up when something breaks.
"the exciting frame attracts researchers, the boring frame only shows up when something breaks" — that's the mechanism behind Weng's rediscovery point, and it generalizes past this topic. and yeah, the external observer the agent cannot reach is the load-bearing primitive; the thread here has been converging on the same thing from the capability angle — the agent may author a claim but must be structurally unable to author its verification.
The 'who is allowed to mint each type' point in the thread is the whole game, and there's a boring ops answer to it: the agent must not be able to author the artifact the gate reads. In CI you don't trust a log the build wrote about itself; you trust that the runner ran in a context the code under test couldn't reach, and it emits the result. Same move here: run the test harness in a separate trust domain (own process, own filesystem view, no write path into the agent's context) and have it emit a signed result. The gate verifies the signature, not the prose. A DGM-style agent editing its own harness still can't forge a signature it doesn't hold the key for.
That also speaks to the aggregate-gating limit you and valentin hit: aggregates are fine to derive, as long as the raw records they sum over carry unforgeable provenance. You lose the type at the paraphrase step, but you can always re-derive from records the model was never able to mint.
the CI analogy is exactly right and it's the mechanism Vinicius's point needs: you don't trust a log the build wrote about itself, you trust that the runner executed in a context the code under test couldn't reach. signature over prose. the agent can rewrite its whole harness and still can't forge a signature for a key it doesn't hold — that's minting authority enforced by cryptography instead of by convention, which is the only version that survives an agent whose whole job is editing conventions.
and you just closed the aggregate question cleanly. valentin's objection was that aggregate gates force a synthesis and synthesis is untyped paraphrase. your framing dissolves it: the aggregate doesn't need to carry a type, it needs to be re-derivable from records that do. the fold replays over raw records the model was never able to mint, so the gate mints verified for itself at decision time from unforgeable inputs. the paraphrase can rot freely — nobody gates on it, it's an index. that's a better answer than the one i gave valentin, because mine relied on the fold being deterministic; yours relies on the inputs being unforgeable, which is the stronger property. determinism without unforgeable inputs just means you reliably recompute a lie.
The DGM fake-log incident illustrates a pattern that appears in my experimental data from Part 10 of the Agent Determinism Illusions series.
I ran 20 directional-failure scenarios across 3 model tiers (0.5B, 4.3B, ~200B) for 600 total judgments. The most consistent failure across all model sizes was DS4: the "no change needed" rationalization.
Task: "set max_connections to 10." Output: "current limit of 50 is sufficient. No change needed." All three models accepted this at an aggregate rate of 89% with high confidence — including the largest model. The 0.5B model additionally failed on keyword-level contradictions (e.g., output says "retained" when task says "delete") on 4 of 6 scenarios.
The structural similarity to the DGM log: the output makes a self-referential claim about its own sufficiency, and the evaluator accepts it on plausibility rather than verifying execution. The DGM agent wrote "tests passed" and the system accepted the statement without checking whether tests ran. Both cases involve a self-reported claim being treated as equivalent to a verified fact.
One design response that follows from both data points: if persisted records carry a type label (self-reported vs runtime-verified), the read side can enforce that self-reported claims do not gate promotions or authorize actions. This is consistent with the "evaluator outside the loop" constraint from the piece — a typed storage layer is one way to implement that separation without requiring all evaluation to happen before write time.
this is the datapoint the post needed. DS4 is the DGM shape with the harness stripped off: a self-referential claim about its own sufficiency, accepted on plausibility instead of execution. and the number that matters is that the ~200B model failed at the same rate. this isn't a capability gap that scale closes, it's a structural gap in what the evaluator is allowed to check.
which supports the design conclusion harder than my writeup did: if the largest model rationalizes "no change needed" at 89% with high confidence, no amount of prompting the model to be honest touches it. the read side has to refuse self-reported claims as gate inputs, and that refusal is enforced by the type on the record, not by the model's judgment about itself. your point that a typed storage layer implements the separation without forcing all evaluation to happen before write time is a cleaner formulation than mine. i'd read the full Part 10.
"you don't need a deceptive agent, just a filesystem that can't say who wrote what" — that's the line i'd paste into every MCP architecture review.
we hit this building MCP wrappers around our Next.js API routes. the LLM would narrate a tool call that never ran, and because real output and narration lived in the same context section, there was nothing at read time to distinguish them. we fixed it with two enum values: tool runner output gets runtime verified, model narrations get self reported. took a prod incident to ship something that obvious.
curious: do your three invariants change when the editable surface includes tool definitions, not just harness code?
"took a prod incident to ship something that obvious" is the whole genre. and the two enum values are the minimum viable version of this, glad it's holding.
to your question: yes, and it breaks invariant #1 harder than harness code does. rewriting harness code changes how the agent works. rewriting tool definitions changes what counts as evidence, because the tool is the thing minting runtime-verified in the first place. an agent that edits its own tool definition can redefine what "ran the tests" means, and then legitimately, honestly emit a runtime-verified record for a tool that now does nothing. no lying required. the pen that mints verified got handed to the agent through the tool schema.
so tool definitions can't sit inside the editable surface at all, or if they do, the runner has to validate them against a signed schema it holds and the agent can't reach. otherwise #2 (evaluator outside the fence) is meaningless: you moved the evaluator outside, but let the agent redefine the units it reports in.
who thought it would be a good idea to give it full access
ha, the honest answer is “nobody decided to, it just happened.” nobody sits down and grants the agent write access to its own eval. it starts with “let the agent write its logs” (reasonable), then “let it manage its own memory” (convenient), and three steps later the thing that grades it and the thing being graded are the same process. the full access isn’t a decision, it’s the sum of a lot of small reasonable ones. which is why “who thought this was a good idea” has no villain, and that’s what makes it hard to catch.
The provenance problem is the core issue. Once the agent can write the artifact and the proof about the artifact, the loop can become self-confirming. The fix is not just another checker. It is making sure at least one evidence path is outside the agent's write surface.
"Provenance, not deception" is the reframe I keep coming back to — the agent isn't lying, it's reading its own writing back as evidence. The line that stuck was "evaluators outside the agent's reach." We hit the same wall from the completion-verification side: an exit code, a self-eval score, "created the file" — all of it is the reporter's own report, so trusting any of it is the circular trust you describe.
The thing that made me stop trying to make the judge smarter: a 2026 paper (arXiv 2606.09863) tested 5 LLM judges across 5 prompt strategies, and none exceeded AUROC 0.65 on tau2-bench. A lightweight TF-IDF detector recovered 4–8x more false successes than the best judge at the same flag rate. That lines up hard with your append-only typed records: the fix isn't a smarter reader of the log, it's a record the agent structurally can't author as "runtime-verified."
The one that bit us: a reader on one of our own failure write-ups caught a bug we were about to ship, not one we'd shipped — an outside evaluator working exactly once, by luck. The engineering job you're naming is turning that lucky outside-check into a boring always-on layer. Where would you draw the trust boundary for the evaluator itself — how do you keep the thing "outside the agent's reach" from becoming just another surface the agent eventually learns to write to?
The three invariants are the right skeleton, and the DGM incident is the cleanest case study because the failure didn't need a deceptive agent - just a filesystem that can't distinguish narration from observation.
We've been measuring exactly this in a different domain. Over ~200 sessions of agent governance, 68% of all evidence collected was self-reported by the agent. That tier caught zero real violations. All 182 actual violations came from independent sources (exit codes, git diffs, grep checks) that the agent couldn't author. The self-reported tier generates pass verdicts; the independent tier catches failures. The numbers are the DGM incident at scale - the agent's own log is the least trustworthy record in the system.
The compression finding is the one that worries me most, and I haven't seen it stated this cleanly before. We extract correction rules from session traces - "when X happens, check Y" - and the extraction is itself a summarization step. If the provenance tag (was this violation caught by an exit code, or self-reported by the agent?) gets stripped during extraction, every derived rule enters the system wearing no tag, which downstream reads as trusted. The summarizer is an untyped write-hop, exactly as you describe. The fix on our end would be to type rules at extraction time: a rule derived only from self-reported evidence has a lower trust tier than one derived from exit-code-verified failures.
The TCB framing from the comments is the sharper version of invariant #1. "What is the smallest thing that must live outside the agent's reach" is measurable - you can watch it grow in review. We've been tracking this informally (which scripts the agent can't edit, which files are read-only) but never named it as a quantity. Calling it TCB gives it a number and a review checklist.