DEV Community

Discussion on: My LLM Critic Disagreed With Itself on Every Trial. The Safe Part Was the Code I Didn’t Trust It to Touch.

Collapse
 
artyomsv profile image
Artjoms Stukans

Shrinking what the critic decides is right move, but now all the safety sits in blockers and nobody watches those the same way. If one blocker class stops firing after some refactor, your numbers only look better, 226 becomes 40 and that reads like plans got safer. I had a case in Kubernetes where four releases in row never actually ran, because an old ReplicaSet kept one pod Running, so every health check and smoke test passed and nothing said a word. Worth asserting each blocker class can still fire on a known bad plan, right next to the critic metrics.

Collapse
 
debashish_ghosal profile image
Debashish Ghosal

You're right about the blocker-class monitoring gap. "If one blocker class stops firing after some refactor, 226 becomes 40 and that reads like plans got safer" — that's the exact failure mode. The Kubernetes parallel (old ReplicaSet kept one pod Running, health checks passed, nothing said a word) is exactly the shape of it.

The blocker safety sits in deterministic gates, but there is no runtime assertion that each gate still fires on a known-bad plan. We're fixing that.

Filed as github.com/deghosal-2026/planner-critic-engine/issues/278 — a gate canary that asserts each blocker class still fires. Shipping in v0.2.3.