A reader asked me a question I couldn't fully answer: "Do you retire rules, or does interception count keep them alive?" The honest answer was: mos...
For further actions, you may consider blocking this person and/or reporting abuse
Rule retirement is the part most agent setups ignore. Adding rules feels responsible, but stale rules become background noise and eventually conflict with the work. A rule audit should probably track last used, failure prevented, conflict risk, and whether the rule belongs in prose or in a deterministic tool check.
Three of those axes the system already tracks. Last-used: per-rule hit count split into script-verified and self-reported. Conflict risk: three-layer check (conflict groups, domain+action overlap, text similarity). Failure-prevented: subtler - 30 of 31 high-violation rules turned out to be detector problems, not rule problems, so "the rule fired" and "the rule caught a real failure" aren't the same thing.
The fourth axis - prose vs deterministic tool check - is the one that reframes the audit. Most rules live in prose (soft constraint, agent can comply or not). A subset are enforced by scripts that fire regardless. The audit question I hadn't asked: which prose rules should have been promoted to checks? A rule guarding something irreversible shouldn't stay in prose if a script can enforce it. The audit should flag rules whose enforcement path is weaker than their failure cost warrants.
That distinction between detector problems and rule problems is huge. A rule firing is only a signal; the maintenance question is whether it prevented a real failure or just exposed noisy instrumentation.
Glad the question landed. The framing that unlocked this for me: your always-loaded rules are a cache, and you are never evicting. Interception count is a hit counter, so treat it like one. Two additions to the retire question. First, decay, not raw count. Weight interceptions by recency. Lifetime count keeps a rule alive forever off one old spike, but a sliding window tells you what is load-bearing now. Second, firing is not enough to earn the always-loaded tier. If the model would derive the behavior from a one-line reminder, demote it to a note injected lazily when the triggering context shows up. Reserve the permanent slots for genuinely counterintuitive constraints the model will not reconstruct on its own. The 3000-char AGENTS.md is really a cache-eviction problem wearing a documentation costume.
The cache framing is sharper than my "ratchet" metaphor. A ratchet describes the symptom (only tightens). A cache with no eviction policy describes the mechanism - and points directly at the fix.
On decay: this is exactly the direction I'm evaluating. Lifetime count is a broken metric for the reason you describe - one spike keeps a rule alive forever. A sliding window (or exponential decay) would surface what's actually load-bearing. It's not implemented yet. Currently I do this manually: review low-hit rules every few weeks and decide. It works, but it doesn't scale, and it's biased by my memory of recent sessions.
On "firing is not enough to earn the always-loaded tier" - this is the gap I didn't articulate in the article. You're drawing a distinction between:
Since the cache framing landed: the eviction policy that maps cleanest onto your interception counts is LFU-with-aging, not plain LRU. A rule that fired 40 times in month one but zero in the last three weeks is exactly what you want to retire, and pure LRU would keep it alive on that stale history. So decay the count: each session multiply every rule's interception count by ~0.9, and when one drops below a threshold, demote it from always-loaded to on-demand (description-gated) rather than deleting it. That gives you a reversible eviction. The rule leaves the resident set but is still fetchable if its pattern recurs, and you stop having to guess which ones are safe to remove.
LFU-with-aging is the right name for what I was circling around without the vocabulary.
Current state: I have a lifecycle check that's a coarser version of this. It looks at whether a rule's "obligations" (applicable sessions) produced any verdict recently. If zero applicable sessions → demotion candidate. But it's binary (applicable/not), not decayed. Your 0.9-per-session approach would catch the exact case I miss: a rule that was applicable 40 times in month one and zero since still shows "historically active" in my system.
The "demote, don't delete" principle is already structural in my Archive tier — archived rules stay fetchable. What's missing is the automatic trigger. Right now demotion is semi-manual: the lifecycle check flags candidates, I review, I confirm. Your decay approach makes it automatic and reversible, which is strictly better.
One design question I'm wrestling with: the threshold. Some rules fire infrequently but guard irreversible actions (e.g., "verify before destructive file operation"). Pure frequency decay would retire them. I'm thinking about weighting the decay rate by a rule's importance tag — slower decay for rules that prevent irreversible damage. But I haven't worked out whether that introduces its own failure mode.
This thread has sharpened my thinking more than the last few solo iterations. The 0.9 decay is going into the next implementation.
The importance-weighted decay is the right instinct, and the failure mode you're sensing is real: you've moved the risk onto the importance tag, and now a mis-set tag on a 'verify before destructive op' rule gets it decayed out, with asymmetric cost, retiring a safety guard is irreversible, retiring a style rule is nothing. So I'd stop treating it as one decay curve with a weight. Split by what the rule guards, not how often it fires. Frequency-decay is the right model for advisory and style rules, where a stale rule just adds noise. For rules whose guarded action is irreversible, frequency is the wrong axis entirely, blast radius is a property of the action, not of usage, so those shouldn't decay on frequency at all, only on explicit supersession. Then the tag you actually need is binary and fail-safe: unknown or irreversible then never auto-retire. Same reason you keep archived rules fetchable, the default on the dangerous class has to be the safe one, because the cost of the two errors isn't symmetric.
"Split by what the rule guards, not how often it fires" - cleaner than the weighted decay I proposed. The weight moves the risk; a binary tag eliminates it.
"Unknown or irreversible -> never auto-retire" - the default should be the safe one. This makes it structural, not a judgment call.
We have half of this: demoted rules go to Archive, not deleted. What's missing is the irreversibility tag itself - our metadata tracks triggers and check methods, but not "what happens if this rule is silently absent when it should fire."
"Rules only added, never removed. No exit mechanism" is the whole post in six words and it generalizes past CLAUDE.md. I've hit the same shape building a memory-authority gate: notes accumulate authority just by existing, and nothing in the system asks whether a note still deserves to override what's already known. Your archive tier (165 of 268, 62%) is basically a TTL you're enforcing by hand. The part I'd push on: your hit/miss tracking mixes objectively verified interceptions with self-reported ones, and you flag that honestly, but self-reported "per rule X, I'm executing Y" is exactly the kind of claim that looks like evidence and isn't, declaration passing as fact. Have you run the hygiene scanner's flags against which rules turned out to be false positives in your hit data? That'd tell you whether structural weakness (no trigger, vague, duplicate) actually predicts low firing rate, or if they're independent problems.
"Rules only added, never removed. No exit mechanism" - that's the six-word version I should have led with.
The memory-authority gate parallel is real. "Notes accumulate authority just by existing" - that's exactly the dynamic. A rule enters my always-loaded tier and nothing asks whether it still deserves the slot. My archive tier is a manual TTL, as you say. Currently building automatic demotion via decayed interception counts (LFU-with-aging, see thread with Dipankar above), not deployed yet.
On the self-reported data push: you're pointing at the structural weakness, and I now have data that proves your point.
Since writing this article, I rebuilt the evidence system with independent verification sources (git diff audits, session transcripts). Three evidence tiers: deterministic (git audit) > independent (exit codes, transcripts) > agent claim (self-reported markers).
The data is stark. Self-reported markers account for 68% of all evidence collected. But they caught zero real violations. All 182 actual violations (95 "failed execution" + 87 "contradicted evidence") were detected by independent or deterministic sources - exit codes and git diffs, not the agent's own claims. The self-reported tier generates pass verdicts; the independent tier catches failures. "Declaration passing as fact" is exactly what was happening, and the numbers confirm it.
On the hygiene scanner correlation: I haven't run that exact analysis, but I did something adjacent. I reviewed 31 rules that showed high violation rates. 30 of 31 turned out to have structural detection problems - partial compliance signals, mixed patterns - not rule quality issues. The rules weren't broken; the detection was. After fixing detection, only 5 rules showed real violations (all caught by exit_code verification). This suggests structural weakness in detection predicts false violation signals more than low firing rate - but that's from adjacent data, not the direct correlation you're asking for. Still on the to-do list.
30 of 31 being detection problems, not rule quality, is the finding I'd lead the whole post with if I had it. It says something sharper than "rules need pruning": most bad rules aren't bad ideas, they're good ideas wearing broken detectors, and that changes what LFU-with-aging is actually measuring.
Decayed interception count can't tell those apart. A rule with a genuinely broken detector looks identical to a rule nobody needs, both show near-zero firings, both drift toward demotion under the same signal. If detection quality dominates the way your 31-rule sample suggests, demoting on decay before fixing detection risks quietly archiving rules that are still load-bearing but invisible to their own trigger.
Worth running detector-repair as a pass that happens before the aging gate gets trusted, not alongside it: fix the 30, then let decay counts run on rules whose detector is actually working, and see whether the demotion signal looks different. Right now a decayed count on a broken detector isn't evidence the rule is unused, it's evidence the detector never had a chance to fire, and those two conclusions point at opposite fixes.
A decayed count on a broken detector isn't evidence the rule is unused, it's evidence the detector never had a chance to fire" - exactly right. I was treating detector repair and decay as parallel tracks. They're sequential.
The 30/31 finding came from a manual review: after fixing detection logic, only 5 of 31 showed genuine failures (all exit-code verified). The other 26 were false positives - load-bearing rules invisible to their own triggers.
The gap: we don't have an automated detector-quality check. Today the only way to find a broken detector is tracing false violation signals back to source. That needs automating before any decay signal is trustworthy.
26 of 31 being false positives on rules that were actually load-bearing is the number that should worry you more than 5 real failures would, a real failure just means the rule fired and got ignored, a false positive means the rule was silently protecting you and every dashboard you had said otherwise.
The automation gap you named, tracing false-violation signals back to source is currently the only way to find a broken detector, is the same shape as the mechanizability problem showing up elsewhere in this cluster this week: a manual review that works is still a manual review, and it doesn't scale past the one pass you already ran. What would a detector-quality check actually look at mechanically, agreement between the detector's fire rate and an independent signal (execution outcome, exit code, git diff), or something structural in the detector's own pattern (regex fragility, missing negative-case coverage)? The second is cheaper to build and catches a narrower class of failure, the first is closer to ground truth but needs the same independent-outcome data you already had lying around for the 5 real ones.
The framing inversion lands: 26 false positives on load-bearing rules is worse than 5 real failures. A false positive means the guardrail was working and every signal told us to remove it. That's the failure mode that would have made the system less safe by trying to improve it.
On detector-quality: we're starting with option (b) — structural analysis (regex fragility, missing negative cases) — because it's cheaper and runs without execution. But you're right that it catches a narrower class. The full version is option (a): fire rate vs execution outcome. We already have that independent-outcome pipeline for the 5 real violations (exit codes, git diffs). Generalizing it to every rule is the build that makes detector-quality automatic. Your two-option framing is now the design spec.