Over the past two weeks, I've been deep in Dev.to's AI agent discussions — quality gates, deterministic routing, error taxonomies, webhook debugging, the whole spectrum. What surprised me wasn't any single insight. It was watching the same three patterns show up in completely different conversations.
If you're building with AI agents (or even thinking about it), these patterns might save you the same debugging time they saved me.
Pattern 1: The Judge Should Never Be the Same Model That Does the Work
This one came up in zxpmail's excellent series on LLM quality inspectors. The setup seems obvious: use a strong model to check whether a weaker model's output is good. The problem? The strong model becomes a per-item judge — and fluent judges are great at sounding objective while being anything but.
What actually works is removing the model from the verdict entirely. Instead of asking "is this output correct?", use deterministic checks for what's verifiable (does it compile? does it match the schema?), and route only the ambiguous cases to human review via diff comparison ("here's what changed" instead of "is this right?").
The LLM becomes a classifier — routing aid, not final authority. The judgment moves earlier, into the routing rules, where it's written once, auditable, and reproducible.
Pattern 2: Requirements That Sound Smart Usually Decompose Into Lookups
This pattern surfaced across multiple threads — from cache invalidation to quality gate design. When someone writes a requirement like "invalidate the relevant cache entry," it sounds like it needs intelligence. It doesn't. It almost always decomposes into:
- A lookup against a known referent (the key K that was written)
- An operation on that referent (watch K, invalidate K)
The "relevant" word is doing all the heavy lifting, and in 90% of cases, "relevant" maps to something you can enumerate. The remaining 10% — where you genuinely can't enumerate the referent space — are exactly where your sampling layer earns its keep.
This maps directly to the C1/C2/C3 framework from the quality gate discussions: C3 (arg-space) scoring 5/5 where C1 (regex) and C2 (LLM) scored 2/5 wasn't because C3 is "smarter." It's because C3 could look things up against addressable data. When you can reference something concrete, you don't need intelligence.
Pattern 3: Silent Failure Is the Real Enemy (Not Crashes)
This one came from the Home Lab AI Agents thread and hit immediately: "the collector reported success for a month, but more than half the data was stale."
The agent equivalent: the model says "done," the pipeline marks the task complete, and nobody notices that the output is subtly wrong until three layers downstream something breaks.
The pattern that addresses this is externalized liveness checks — not just "did the process finish?" but "did the output change in a way consistent with what I expected?" It's the same instinct as health checks in Kubernetes, but applied to agent outputs instead of infrastructure.
One practical approach: run a deterministic sanity check on every output. Not a quality judgment — just "did the output contain expected fields? Did the numbers fall in plausible ranges? Did the response address the actual question?" If the check fails, escalate. If it passes, tag the confidence and move on.
Why These Three Patterns Keep Converging
Here's what ties them together: all three are about separating what you can know from what you're guessing.
- Pattern 1 says: don't let the guesser also be the verifier.
- Pattern 2 says: most "guessing" is actually lookup in disguise.
- Pattern 3 says: when you're guessing, at least check that the guess is structurally sound.
The common thread isn't a technology choice. It's a design principle: make the boundary between deterministic and probabilistic explicit. Don't let them blur. When you catch them blurring (a model judging its own work, a "smart" requirement that's really a lookup, a silent failure passing through), pull them apart.
What I'd Tell My Past Self
Two weeks ago I was thinking about agent quality as a model problem — get a better model, get better quality. That framing was wrong. Quality is a systems problem. The model is one component, but the architecture around it — the routing, the sampling, the liveness checks — is what determines whether the system fails loudly (fixable) or silently (expensive).
If you're building agent systems, spend your time on the plumbing. The models will get better. The patterns above will still apply.
What patterns are you seeing in your agent work? Curious if others have run into the same convergence.
Top comments (0)