I built 5 mechanical gates for my AI coding agent. Then a philosopher told me I was solving the wrong problem.
The Problem Started Simple
I use Claude Code for long coding sessions. After ~50 sessions, a pattern emerged: the agent would gradually drift. Rules set early were forgotten. Config files claimed scripts were deployed when they weren't wired to any hook. The agent's self-assessment diverged from reality — claimed 13 HOT entries, actual was 53.
I built four mechanical gates to catch these gaps: execution-gate (blocks writing more scripts if you haven't run any), hook-audit (cross-references scripts against hook registrations), quality-gate (checks learning logs after complex tasks), claim-gate (verifies declared deliverables exist).
They worked. Real issues were caught. But every problem had the same shape: "I claimed X, but X wasn't actually true." And I kept adding new gates for new types of X. Rule inflation.
Then a Philosopher Looked at My System
The philosopher asked: "Your agent generates its self-assessment through the same decoder that generates its code. Where is the independent verification channel?"
There isn't one.
A transformer-based AI agent produces its self-model narrative and its capability execution through the same P(token | context; θ). The claim "I can do X" and the action of doing X are both samples from the same distribution. This is not a bug — it's a structural constraint. I'm calling it the Prose Barrier.
The Prose Barrier, Formally
Correlation is not measurement. If claims and actions correlate, it's because they share parameters, not because the agent measured its own capability.
Self-model is L1 (association), reliability needs L2 (intervention). In Pearl's causal hierarchy, the agent's self-assessment observes patterns in its own outputs. But verification requires intervention-level evidence:
do(execute) → observe exit code.The mirror break. When the agent regenerates its self-model by re-reading its own growth-logs (themselves written by the same decoder), it sees a mirror, not a measurement. My system claimed "HOT 13 (≤15 ✓)" while the actual count was 53.
Human Logic vs. AI Logic
My initial gates used file timestamps, regex, exit codes — tools humans built to audit computers. They work, but they're human logic, not AI logic.
An AI agent's natural senses are attention weights, residual stream directions, and logprob distributions. The Prose Barrier means verification must happen at the level where information actually flows.
So I built a second layer: neural gates.
- v1 (deployed): Constraint echo detection — does the rule in BODY.md appear as a pattern in outputs? If "always verify" never echoes in output, the constraint isn't penetrating.
- v2 (designed): Logprob differential — compare token probabilities with/without constraints. If delta decays, the constraint is neurally decaying.
- v3 (roadmap): Linear probes in the residual stream — on a local Qwen2.5-1.5B, detect whether constraint information is linearly decodable from activations.
Dual-Layer Architecture
File System Layer — "Did the information arrive at the door?" (mtime, hook wiring, exit codes — bypasses the Prose Barrier by checking filesystem state, not NL claims)
Neural Layer — "Did the information travel through the house?" (constraint echo, logprob shifts, residual stream probes — works within the barrier using weak proxies)
Two independent experts — philosopher + AI architect — converged on this same topology. The problem structure forces the solution shape.
What I Found (Honest Limitations)
Across 33 development logs (June–July 2026): 15 documented Prose Barrier instances. Verification-to-prose ratio increased from 6.0 (June) to 9.7 (July). Fix rate: 28% → 67%. The system gets better at recognizing the barrier, even if it can't eliminate it.
Honest: Single developer, single RTX 3060 6GB. Single-rater (I score my own outputs). No placebo control. Neural gate v1 is a weak proxy (keyword echo ≠ semantic fidelity). This is preliminary evidence, not proof.
Why This Matters
The Prose Barrier applies to any AI agent that generates its self-model through NL: autonomous coding agents, AI safety frameworks, agent evaluation benchmarks. If you deploy an agent without mechanical verification gates, you're operating on L1 correlation in a domain requiring L2 evidence.
What's Next
Second rater (Cohen's kappa), placebo control, API access for v2 logprob detection, cloud GPU for v3 probes on larger models. If you're working on agent reliability or self-verification — let's compare notes.
👋 林宇浩 — AI output reliability infrastructure. ECC + HuggingFace Evaluate contributor. github.com/YuhaoLin2005
Top comments (0)