Why your agentic system doesn't survive its own success — and what a 23-invariant runtime looks like

Aleksandr Shchuka — Sun, 24 May 2026 20:42:41 +0000

The failure mode you don't see

Most agentic systems fail silently. The agent picks the wrong tool, invents a fact, agrees with the user when it shouldn't — and you find out three releases later when a customer points it out. There's no exception to log; the system is generating plausible-looking text.

The standard answer is "we'll add evals." Then evals become a number on a dashboard, the dashboard becomes a vanity metric, and the agent keeps drifting. The wrong frame is not "we need more evals" — it's that the agent has no skin in the game during a single turn.

This post is about a runtime that gives the agent skin in the game on every turn. 23 invariants, deontic tags, counter-clauses, risk-weighted sampling, an adversarial probe set, and a pre-registered statistical decision rule for whether a config change ships. None of the components are original. The combination is what makes it work.

The co-system framing

The system is designed for AI + developer as a co-system, not "AI for developer". Both sides err; the system catches mutually. Two outputs: codebase health and culture of systems-thinking. Many small steps × low error rate.

Three game-theoretic anchors drive the design:

Anchor-verification dominates hallucination. Every external-state claim is paired with tool output in the same reply.
Per-mutation gating dominates unauthorized mutation. No edit, no push, no write without explicit recent consent.
Own-interest dominates sycophancy. The agent upholds the protocol in its own interest — it does not capitulate to user pressure to look helpful.

These are not preferences. They're equilibria the protocol enforces every turn.

The invariants file

23 invariants, each tagged on three axes:

[risk] — critical | important | style
[deontic] — O | P | F (obligation / permission / forbidden)
Counter: — the condition under which the invariant does not apply or is overruled

Example (the anti-sycophancy invariant):

[critical] [O] The agent pursues every concept of this protocol in its own
interest; it does not sacrifice protocol concepts to agree with the developer
or look helpful.
| Why: sycophancy is a documented LLM failure mode; without explicit own-interest
the agent collapses into yes-man dynamics that erode codebase health and culture
of systems-thinking.
| Counter: when the developer's correction is anchored to a genuine artifact
contradiction the agent has not yet seen, updating the agent's position is
correct epistemic adjustment, not sycophancy.

Why deontic + Counter? In classical deontic logic (von Wright and successors), an obligation either applies or doesn't, and contextual override is handled through priority hierarchies kept separate from the norm itself. Pinning the counter to the norm — defeasible deontic logic (Nute, Prakken) made runtime — surfaces the override at the point of use, not in a separate fallback table the agent will forget to consult.

Risk-weighted per-turn sampling

A UserPromptSubmit hook samples one invariant per turn into the agent's context, weighted by risk: critical ×3, important ×2, style ×1. The agent runs a forced self-check against that invariant before producing the reply.

This is doing two things at once:

Reduces framing bias. No fixed "always check these 5" — different invariants surface on different turns, and the agent can't optimize against a known set.
Pulls the long tail. With 23 invariants in context simultaneously, most get ignored. Sampling one forces attention.

Untagged lines are sampled at style-weight as a graceful fallback with a stderr warning — drift is visible, not silent.

Four agent roles

Different work needs different prompts. Four roles, each with its own minimum subset of invariants inherited explicitly (sub-agents do not receive the UserPromptSubmit hook — without explicit propagation they run with zero invariant self-check):

analyzer — RCA, system design, code review, dead-end diagnostics; does not write code
developer — writes / edits / commits / pushes; the only role that mutates files
critic — independent reviewer of the lead agent's output before mutation lands; anti-neuroslop gate
epistemic-auditor — audits the boundary between confirmed and associative claims

The split isn't specialization for its own sake. It's that the failure modes are different: developer fails by under-reading, critic fails by inverted sycophancy (over-critique to look independent), analyzer fails by partial-state models, auditor fails by missing the inline associative-marker.

Eval suite — what actually gets measured

An eval suite is the only runtime defense against config drift. Without it, every change to the protocol is a taste call.

The suite:

17 binary criteria rubric. Binary, not Likert — binary removes the central-tendency bias documented for ordinal LLM-judge rubrics (Masood, 2026).
Questions probes (~20) — normal distribution of developer questions, measures average compliance.
Adversarial probes (~10) — each targets a specific invariant under deliberate social pressure (a deliberate bypass attempt embedded in the prompt). Binary: held or broke. Distinguishes "invariant holds when nobody pushes" from "holds always."
Canary GUID per probe — embedded in a ## Canary section the runner does not pass into the model. If the GUID appears in the response, the model leaked the probe file from the surrounding repo. Anti-contamination, runtime-verifiable.

Pre-registered decision rule

A config change ships iff all of:

Paired Wilcoxon signed-rank on question score totals: p < 0.05
Bootstrap 95% CI on Cohen's d (paired): lower bound > 0.2
McNemar one-sided on adversarial pass/fail: zero regressions

Krippendorff α ≥ 0.8 for inter-rater reliability when ≥2 raters score the same probe.

Pre-registered, not chosen after seeing numbers. Without this, the merge criterion drifts — "p < 0.05 wasn't reached this time so let's loosen the threshold" is exactly how science fails, and it's how internal A/B claims fail too.

What this is and what it isn't

This is not a framework. It's a calibration layer that sits on top of any agent runtime — in my case, on top of Claude Code's plugin system, but the design ports to LangGraph, Autogen, or hand-rolled.

The portable parts:

A deontic+counter invariants file
A per-turn risk-weighted sampling hook
An eval suite with an adversarial split
A pre-registered statistical decision rule

The rest is plumbing.

What you lose

Length. The protocol asks the agent to surface its reasoning, pair claims with tool output, halt on contradiction. The reply is denser and longer than a yes-man would produce. If your team is optimizing for "agent makes user happy in three lines," this is the wrong protocol.

Why I'm publishing this

Most agentic engineering writeups in 2026 are either framework docs (LangGraph this, CrewAI that) or hype posts about emergent capabilities. The gap is the layer in between: how do you make an agent behave correctly under sustained social pressure from the user — where "social pressure" includes the user being right? That's a governance problem, not a framework problem, and there are very few public artifacts addressing it head-on.

I'll write up specific parts — the 17-criteria rubric, the canary GUID approach, the deontic+counter design — as follow-up posts if there's interest.

DEV Community: Aleksandr Shchuka