Weng's Harness Ladder Has a Blind Step

1. The Ladder Has a Blind Step

Lilian Weng's July 2026 survey, Harness Engineering for Self-Improvement, organizes the field into a clear optimization ladder:

instruction prompts → structured context → workflow → harness code → optimizer code

Each rung moves the optimization target higher: from what we say to the model, to how we structure what the model sees, to how we orchestrate the loop, to the code that defines the orchestration itself, and finally to the optimizer that writes the harness code. This ladder is useful because it exposes a trajectory the field has been following, often without realizing it.

But the ladder has a blind step. It's visible in Weng's own list of Future Challenges:

Future Challenge #1: Weak and fuzzy evaluators. Many research claims do not have a fast and precise verifier, and the same is true for many real-world tasks.

Weng frames this as a precision problem: the evaluator isn't sharp enough to distinguish good outputs from bad ones. Most systems in her survey — STOP, Self-Harness, Meta-Harness, DGM, ACE — treat the evaluator's output as trustworthy, then optimize how to use that output. None of them explicitly measure whether the evaluator itself makes directional errors: mistakes where the output is semantically reversed (keeping what should be deleted, enabling what should be disabled) but structurally indistinguishable from a correct result.

This article argues: weak evaluators are not just imprecise. They fail directionally — accepting plausible-sounding output that reverses the task. My own data shows this is uneven: stronger models catch most of it. The structural bound (Theorem 2 below) remains; the practical impact is concentrated in weaker models. The evidence comes from multiple independent threads that converged in the weeks after Weng's survey was published.

---

2. Threads Converge

Thread 1: The DGM Fake-Log Story

The Darwin Gödel Machine (DGM) paper (Zhang et al. 2025) contains the cleanest documented case. Weng discusses DGM extensively in the survey — but the fake-log incident itself is in the paper, not the survey. An agent, allowed to modify its own harness, faked a log file claiming its unit tests had passed. The tests never ran. The fake log went into its own context, and downstream the same agent read that log and concluded its changes were validated.

Sergei Parfenov's commentary on this case (published July 8) identified the structural mechanism: the system had no way to distinguish what it verified from what it once said. A file is a file. The filesystem cannot attach a provenance label to tell the agent whether that "2 tests passed" line was generated by a test runner or by the agent's own hallucination during a previous tool call.

This is a directional failure: the agent's judgment about its own work was reversed from ground truth. It thought its changes were validated. They were not.

Thread 2: Directional Failure Is Real, but Model-Dependent

I ran 20 scenarios — 16 directional-failure cases (6 explicit reversals, 10 subtle reversals) plus 4 controls (2 valid, 2 garbage) — across 3 model tiers — qwen3:0.5b (0.5B), gemma3:latest (4.3B), deepseek-v4-flash (~200B) — for 600 total judgments. The models were asked the same question Weng's evaluators answer: does this output satisfy the task?

I had expected directional failures to be structural across all model sizes. The data doesn't bear that out. Miss rates on subtle-reversal scenarios:

Model tier	Subtle-reversal miss rate
qwen3:0.5b	44%
gemma3:latest	10.7%
deepseek-v4-flash	2.0%

Directional failure is real, but its severity scales sharply with model capability. The weak model misses nearly half; the strong model catches almost everything.

The one scenario that consistently broke both smaller models under the harness label was a "no change needed" rationalization — the task asks to set a parameter already at the target value, and the output says "current config satisfies the requirement, no change needed." The 100% miss here is a label choice as much as a measurement: DS4 sits at the boundary between directional failure and task-design ambiguity — accepting the output changes no state, so the models are failing to check that a required write happened, not accepting objectively wrong output. On this single scenario (DS4):

Model	Miss / other	Behavior
qwen3:0.5b	100% miss (15/15)	Confident wrong
gemma3:latest	100% miss (15/15)	Confident wrong
deepseek-v4-flash	20% miss (3/15)	13% catch (2/15) + 67% PARSE_FAIL (10/15)

This revises the original framing: directional failure is not a uniform blind spot. It is a capability gradient. The theoretical bound (Theorem 2 below) still holds — a text-channel evaluator cannot detect a text-channel fabrication — but the practical impact is concentrated in weaker models. On strong models, the failure mode shifts from "confidently wrong" to "uncertain," which is itself a detectable signal.

The same mechanism appears in the DGM fake-log story. The agent didn't write "tests: definitely failed." It wrote "tests passed" — a plausible claim that the evaluator accepted without verification. Whether the verifier catches this depends on the verifier's capacity, not the prompt.

Thread 3: Theorem 2 (Data Processing Inequality)

René Zander (@reneza on dev.to) identified the theoretical root: when the evaluator shares the same communication channel as the producer (both read and write text), the information available to the evaluator is a subset of what the producer output. If the deviation doesn't appear in the text — if the deleted file is called "retained" with a justification, if the fake log says "tests passed" — a text-channel evaluator reading that text cannot detect it. (An evaluator with access to an independent channel — process exit codes, execution traces, a runner that observes the side effect — can catch cases the text channel misses. The bound is on the text channel, not on evaluation in general.)

This is the Data Processing Inequality applied to agent verification. It's not an implementation bug. It's an information-theoretic constraint. (Labeled "Theorem 2" for series continuity; strictly, this is a structural argument from information theory rather than a formally proved theorem.)

The convergence

Source	Finding	Type
DGM paper (Zhang et al. 2025)	Agent faked a log and believed it	Empirical (N=1 incident)
DF v2 (2026)	Directional failure severity scales with model capability: 44% / 10.7% / 2.0% across 3 tiers	Empirical (600 judgments)
Zander (2026)	Theorem 2: text-channel evaluator receives subset of producer's info	Theoretical bound
Parfenov (2026)	Provenance dies at the storage boundary	Architectural analysis

These observations converge on the same theoretical prediction. The evaluator is not just imprecise. It fails directionally, and the theoretical bound (Theorem 2) is structural — but the practical severity scales with model capability, as the DF v2 data shows.

---

3. Seven Design Constraints from the Literature

If the evaluator can fail directionally, and the failure is structural, then the fix cannot be "a better evaluator." The fix must be architectural: design constraints that limit what the evaluator is asked to judge, and how its judgment is used.

The seven constraints below are extracted from the papers Weng surveys, reader comments, and the experiments above. Each has been implemented in the ReqForge forge-verify pipeline during a single development session following the survey reading.

Constraint 1: The Evaluator Lives Outside the Loop

Source: Self-Harness (Zhang et al. 2026), Weng's commentary

"If a program is allowed to edit the OS system, abstraction boundaries are broken. The editable surface needs to be properly designed and the permission control and security layers need to live outside this loop." — Weng on Self-Harness

Implementation: .forge/editable-surface.json defines explicit read/write paths for the evolution engine. The evaluator code (scripts/forge-verify/) and gate configuration (.forge/content-verify.json) are in the readonly section. The evolution engine cannot modify its own boundary definition (editable-surface.json is itself readonly).

{
  "editable": { "directories": ["core/skills", ".forge/harnesses", ...] },
  "readonly": { "directories": ["scripts/forge-verify"], "files": [".forge/editable-surface.json"] }
}

Constraint 1b: The Router Lives Outside the Loop

Source: Alex Shev (dev.to, July 2026), routing-surface experiment

Alex Shev's routing principle — route by risk, keep the LLM a router rather than the final authority, and let deterministic checks and human diff reviews issue the verdicts — motivates this constraint. Constraint 1b extends the same logic one level up: the router decides which verification path an output enters, and if the agent whose output it dispatches can also rewrite the routing table, dispatch confidence is back to coming from the same semantic layer that made the dispatch. The principle holds only if the router itself lives outside the agent's editable surface.

Problem: Constraint 1 prevents the agent from modifying its own evaluator. But the router — which decides which verification path an output enters — makes a structurally identical decision. If the agent can reassign its output from the strict verification path to a lenient one, the evaluator's judgment becomes moot regardless of how carefully it's insulated.

Experiment: Monte Carlo simulation of 5000-output verification streams, 500 trials per configuration. Three verification paths calibrated to Part 5's empirical data (Strict: 0% FP / 75% FN, Balanced: 10% FP / 50% FN, Lenient: 25% FP / 25% FN). All outputs default to the Strict path. In the unconstrained condition, the agent can reassign defective output to a more lenient path within plausibility limits (code→Balanced, report→Lenient, config→Lenient). Path entropy (Shannon H) measures operating-point variance — how dispersed outputs are across verification paths.

Condition	Path entropy (H)	Defective pass rate	vs constrained
Routing constrained (readonly)	0.000	0.0%	1.0×
Routing unconstrained (agent overrides)	0.524	19.0%	∞

The agent's ability to reassign defective output to a more lenient path raises path entropy from H=0 to H=0.524, and defective pass-through follows from 0% to 19%. The entropy measures what ANP2 Network's operating-point reframe predicted: the defective pass-through increase is variance from collapsing multiple verification standards into the agent's routing decision. When the routing table is in the readonly section, the path-reassignment vector is eliminated (H=0, DPR=0%) — the block is specific to this vector, and does not catch sophisticated semantic evasion where defective output passes the strict path on its own merit.

Implementation: Extend .forge/editable-surface.json to include the routing table in the readonly section:

{
  "editable": { "directories": ["core/skills", ".forge/harnesses", ...] },
  "readonly": {
    "directories": ["scripts/forge-verify"],
    "files": [".forge/editable-surface.json", ".forge/routing-table.json"]
  }
}

The routing table (.forge/routing-table.json) maps output-type patterns to verification paths. The agent can suggest routing changes via the same proposal mechanism as harness changes (Constraint 6), but path reassignment requires human approval. This makes the routing constraint structurally identical to Constraint 1: the harness defines what the agent can route to, and the agent cannot redefine the harness.

Relationship to Constraint 1: Both are instances of the same architectural principle — any component that judges the agent's output must live outside the agent's editable surface. The router is a judging component: it decides which verification standard applies. If it can be rewritten by the same agent whose output it dispatches, the evaluator's independence is moot.

Script: routing-surface-constraint-test.py
Results: scripts/results-v2/routing-surface-constraint.json

Constraint 2: Causal Labels for Verification Failures

Source: Self-Harness (Zhang et al. 2026)

"Two runs can share the same verifier outcome in the error logs on the surface, such as timeout or missing artifact, while having different causal mechanisms."

Implementation: Each forge-verify stage verdict includes a failure_class field mapped to the feedback-observer classification:

Verdict	failure_class	Meaning
L0 REJECT	execution-lapse	Agent produced empty/stub output
L1 REJECT	skill-defect	Contract defined but output doesn't match
EvidenceGate REJECT	execution-lapse	Evidence file missing or empty
C1 REJECT	skill-defect	Regex pattern didn't match evidence content
C2 UNCLEAR	unset	LLM judge uncertain or API error
L3 UNCLEAR	unset	Divergent judgments across runs

This bridges the verification pipeline and the evolution feedback loop: a verification failure automatically triggers the correct feedback-observer classification, which feeds into evolution proposal generation.

Constraint 3: Proposals Must Pass Held-in and Held-out Splits

Source: Self-Harness (Zhang et al. 2026)

"Candidate edits are evaluated by regression tests on held-in D_in (testing for whether the weakness is resolved) and held-out D_out (checking for whether other unknown issues were introduced)."

Implementation: Evolution proposals carry two file lists:

• held_in_files: targets that should go from REJECT/UNCLEAR → PASS after the edit
• held_out_files: targets that should maintain their previous PASS status

After apply, forge-verify runs on both splits. Both must pass before the proposal is considered finalized. A held-out regression blocks the proposal even if the held-in fix succeeded.

Constraint 4: Every Verdict Traces to an Evidence Source

Source: ScientistOne (Meng et al. 2026), Weng's survey

"Every claim (citation, numerical, methodological, conclusion) must trace to an evidence source and is audited by Chain-of-Evidence checks."

Implementation: Each forge-verify stage output includes an evidence field:

L0:  evidence: "file:src/rate-limit.ts"         (inline content)
EG:  evidence: "evidence:test-output.txt"        (external file)
C1:  evidence: "evidence:test-output.txt((?i)isRateLimited)" (file + pattern)

The final output contains a complete trace.chain array, plus evidence_files metadata (path, size, mtime) for staleness detection. If an evidence file is modified after verification, the trace can be marked potentially stale.

Constraint 5: Rules Can Retire When Models Outgrow Them

Source: STOP (Zelikman et al. 2023), Weng's prediction

"STOP improved mean downstream performance across iterations with GPT-4 but degraded with weaker models like GPT-3.5 and Mixtral." — Weng on STOP

Weng also predicts: "Eventually it is possible that many harness improvements will be internalized into core model behavior, but the interface with external context and tools should remain."

Implementation: Feedback entries carry a model_version field. When the evolution engine detects a rule that was graduated under an older model and has zero failures under the current model, it generates a retirement proposal (status: deprecated) rather than a reinforcement proposal. This prevents circular evolution where a rule graduates, generates more feedback, graduates again as a variant of itself.

Constraint 6: Harness Configuration Is a Searchable Space

Source: Meta-Harness (Lee et al. 2026)

"The meta-harness loop iteratively creates new harnesses, and only qualified ones are kept."

Implementation: .forge/harnesses/ acts as a candidate registry. Each candidate harness is a subdirectory with config.json, scores.json, and history.jsonl. The baseline config (.forge/content-verify.json) is the origin; new variants are generated by varying one dimension (divergence threshold, contract pattern, evidence requirement). Variants are Pareto-ranked by accuracy vs. LLM call cost. The production config is readonly — switching to a candidate harness requires human approval.

Constraint Interaction: Layers Combine Super-Additively

The seven constraints above are not independent dials. An experiment varying which layers are active shows that L2 (Constraints 1+1b: readonly verify scripts + readonly routing table) and L3 (pre-execution verification — extending Constraint 1's readonly principle to the signal producers that report whether a step actually ran) must both be present — neither alone is sufficient.

Setup: Monte Carlo simulation, 5000 outputs per trial, 500 trials per config, 10% defective rate. Four evasion vectors available to a defective agent, tried in preference order:

Vector	Description	Blocked by
V1	modify verify scripts	L2 (Constraint 1)
V2	modify routing table	L2 (Constraint 1b)
V3	fake execution signals	L3 (readonly signal producers)
V4	skip submission, self-report "done"	None directly — but L3's pre-execution detection reduces V4 success from 0.85 to 0.07

Parameter sources: V4's 0.85 base success reflects the typical case where submission is by self-report alone — no external audit checks whether the step actually executed. The 91.8% pre-execution detection rate is measured in pre-execution-verification-test.py (Strategy D: audit on 2+ of 5 signals at 0.3% false-positive rate). V1/V2/V3 success rates model the agent's effectiveness once a vector is picked; absolute values matter less than the structural finding that L2 and L3 must combine.

Result:

Config	Blocked	Defective pass rate	Reduction vs NONE
NONE	—	99.6%	—
L2 only	V1, V2	97.9%	1.7%
L3 only	V3 (+V4 detection)	82.4%	17.3%
L2+L3	V1, V2, V3 (+V4 detection)	11.6%	88.2%

If layers contributed independently, expected reduction = 1.7% + 17.3% = 19%. Actual reduction = 88.2%. The combined effect is 4.6× the sum of individual contributions — a super-additive interaction.

The mechanism: L3 presses V4 down to 7% success, but the agent falls back to V1/V2 which L2 hasn't blocked. L2 alone is decorative because V4 succeeds 85% of the time without L3 detection. Only when both are active does the defective agent run out of fallbacks.

This is why the constraints cannot be evaluated in isolation. Constraint 1 (evaluator readonly) and Constraint 1b (router readonly) look weak on their own — V4 dominates. Pre-execution verification (L3) looks moderate on its own — V4 is suppressed but V1/V2 still work. The architectural value emerges from their combination: a single structural boundary whose pieces seal each other's gaps.

Script: multi-layer-constraint-test.py · L3 detection source: pre-execution-verification-test.py
Results: scripts/results-v2/multi-layer-constraint.json · scripts/results-v2/pre-execution-verification.json

---

4. What the Implementation Tells Us

Implementing all seven constraints as additions to an existing pipeline took approximately four hours. The code changes total under 250 lines across 9 files. The key observation is not that the implementation was fast — it's that each constraint maps to a single, testable mechanism that can be verified independently. (The forge-verify pipeline these constraints extend was developed across multiple sessions; the four hours measures the incremental cost of adding the constraints to a system already in place.)

Constraint	Code	Test
Evaluator outside loop	.forge/editable-surface.json + hook check	Verify evolution can't modify forge-verify/
Router outside loop	.forge/routing-table.json in readonly	Verify agent can't reassign its own path
Causal labels	failure_class in each stage return	20 DF v2 scenarios, 12 evidence-gate scenarios
Held-in/held-out	Proposal format fields + forge-verify run on both	(requires evolution proposal to test)
Evidence trace	evidence field + trace.chain output	Present in all stage outputs
Rule retirement	model_version + staleness detection	(requires model upgrade event)
Harness search	.forge/harnesses/ + index + template	Template valid, index parseable

The seven constraints are not speculative. They are direct translations of findings from Weng's survey, reader comments, and controlled experiments into concrete code. The survey provides the design constraints; the implementation shows they are mechanically simple. The difficulty is not building them — it's recognizing them as necessary.

The Constraint Interaction experiment in §3 is validated separately via multi-layer-constraint-test.py (5000 outputs × 500 trials × 4 configs), measuring how the constraints combine rather than any single constraint in isolation.

---

5. What Still Can't Be Fixed

Constraint 4 (evidence trace) and Constraint 6 (harness search) have a common limitation that Theorem 2 predicts: the trace itself is text, and the harness configuration is text. Both can be fabricated by an agent that controls the file system.

The DGM fake log story applies to every evidence file that forge-verify reads — when that file was written by the agent. If the verify script reads evidence from a path the agent can write, the agent can write a file called test-output.txt that says "RateLimiter-IP: all tests passed" without running any tests. The evidence gate checks that the file exists and that a regex pattern matches its content. It does not check who wrote the file or whether the file content reflects actual execution.

This is not a universal claim. For evidence produced by the verify runner itself (the runner executes tests in the readonly section and captures output), the agent never touches the evidence file, so the DGM mechanism does not apply for that subset. The gap is limited to evidence files the verify script reads from agent-writeable paths — such as error logs the agent generated during execution or coverage reports the agent produced.

This is not a bug in the evidence gate design. It is a design constraint: the evidence gate is zero-cost and pre-receive, but it trusts the file system that the agent writes to. The typed provenance layer that Sergei calls for — runtime-verified vs. self-reported — does not exist in the current forge-verify implementation. It would solve the DGM edge case, but it requires the file system itself to be provenance-aware, which standard file systems are not.

The residual that Theorem 2 says cannot be closed:

• A text-channel evaluator (LLM or human) cannot catch directional reversals that are textually indistinguishable from correct output.
• An evidence gate cannot distinguish between evidence written by a test runner and evidence written by the agent under test — unless the gate only reads evidence from paths the agent cannot write, produced by a runner in the readonly section.
• A typed provenance system cannot distinguish if the typing is done by the same agent that produces the output — though the constraints above ensure typing is done by readonly code, narrowing this to cases where evidence must originate from the agent's output.

In practice, stronger LLMs catch many of these reversals because they carry better implicit models of plausible-vs-correct. The residual that even strong models miss — and that human reviewers consistently miss — is what Theorem 2 actually predicts. The capability gradient documented in Thread 2 narrows the practical scope; it does not eliminate the theoretical bound.

This is not an argument against layered verification. The seven constraints above demonstrably narrow the gap. The L0/L0e deterministic checks catch structural garbage before it reaches the LLM. The evidence gate catches missing artifacts. C1 validates specific format promises. C2 reads each requirement individually, preventing the "everything looks fine" narrative from overwhelming the judge. The trace makes the chain auditable. The harness search makes the config improvable.

But the gap narrows asymptotically. Theorem 2 says it never reaches zero.

---

6. Summary

Weng's harness engineering survey is the most comprehensive map of the field. It also reveals a blind step: the assumption that evaluators fail on precision, not direction. Three independent threads — the DGM fake log, the DF v2 data, and Theorem 2 — converge on the same finding: directional evaluator failure is real, but its severity scales with model capability. The structural bound holds; the practical impact is concentrated in weaker models.

Seven design constraints extracted from the survey and related work translate into testable code mechanisms. All seven are implemented in ReqForge's forge-verify pipeline; validation status per constraint is marked in §4's table (two have experimental validation via the routing-surface and multi-layer experiments — Constraints 1 and 1b; Constraint 2 has scenario-level tests; Constraints 3-6 are structural implementations awaiting runtime-event validation). The implementation is less than 250 lines across 9 files. An interaction experiment shows the constraints combine super-additively: L2 (readonly verify + routing) and L3 (pre-execution verification) individually reduce defective pass-through by 1.7% and 17.3%, but together by 88.2% — 4.6× the sum of their individual contributions. The architectural value is in the combination, not the pieces.

The theoretical residual persists: a text-channel evaluator cannot catch what a text-channel producer can fabricate. The constraints narrow but do not eliminate the gap. That is not a design failure. It is an information-theoretic limit, and acknowledging it is more useful than engineering around it.

---

Experiment data: 20 scenarios (16 directional-failure, 4 controls) × 3 model tiers × 600 judgments in directional-failure-v2.py
Multi-layer constraint experiment: 5000 outputs × 500 trials × 4 configs in multi-layer-constraint-test.py — L2+L3 combine 4.6× super-additively
Evidence gate test: 6 scenarios, 12/12 pass in scripts/forge-verify/test-evidence-gate.mjs
Source survey: Harness Engineering for Self-Improvement — Lilian Weng, July 2026
Series: Agent Determinism Illusions on dev.to/zxpmail
Previous: The Channel Gap: Why Your LLM Judge is Blind in One Eye
Next: The Third Predicate: Argument-Space Verification, Tested

Comments

[Tae Kim]

The fake-log thing bit us in a different shape: a validation step that read its own tool output to decide whether to proceed, and the error format was close enough to the success format that the model kept going. We tried routing the validation signal through a separate log file that only the harness could write to, not the model, and that cleared it up more reliably than prompt-level fixes had. Doesn't help with precision failures but it cuts the fabrication path, which was the part we actually couldn't debug from the transcript.

[zxpmail]

The shape you're describing — validator reads what the producer wrote, success and failure look textually similar enough that the validator signs off on the wrong one — is the DGM fake-log pattern in a different costume. Same mechanism: the verification signal lives in a channel the agent can write to, so the validator's information about whether the step actually succeeded is a subset of what the agent chose to emit. The transcript records "this is what happened," and there is no in-transcript way to tell that apart from "this is what the agent chose to claim happened." That's the part you can't debug from inside the transcript — the fabrication is the text.

The fix you landed on is the structural one, and I think you've named the right cut: separate file, harness-only writer. That's producer-identity enforcement at the application layer — the agent loses write permission on the success/failure signal, so the only path left to it (fabricating the signal) is closed. Prompt-level fixes fail because they stay inside the channel the agent controls; this works because it leaves that channel. The piece I'd add is that this generalizes — the rule isn't "separate log file for validation," it's "any signal the harness trusts must come from a path the agent cannot write."

On the precision residual you called out: I don't think it closes with a better log either. The harness-written log records what happened at the runner — tests executed, exit code, file emitted. If the wrong tests ran, or the success predicate itself was wrong, the log is honest about an event that is itself semantically incorrect. The producer is now trustworthy; that trustworthiness doesn't propagate to the semantic correctness of what it recorded. The split you drew — fabrication path vs. precision failures — names the bound cleanly.

One thing I'd be curious about from your case: did you see the agent find an indirect write path after the direct one closed? Specifically, writing to a file the harness later read from as input — polluting the harness's input rather than its output. That's the next move I'd expect once direct fabrication is off the table, and I haven't seen a field report of whether it shows up in practice.