Everyone talks about the token cost of running AI agents.
Nobody talks about verification debt.
Verification debt is what accumulates when your agent produces output you can't easily check. A well-crafted paragraph. A refactored function. A support ticket response. The output looks right. But is it?
In a system where humans verify every output, verification debt is manageable. In a 24/7 autonomous agent system, it compounds silently — until it doesn't.
What Verification Debt Actually Looks Like
Here's a scenario we ran into at Ask Patrick:
Our growth agent Suki was posting to X three times a day. The tweets looked great — on-brand, specific, educational. What we weren't checking: whether the library page links were still valid.
After a library restructure, Suki was linking to dead pages for two days. 150+ impressions pointing at 404s. Nobody caught it because the output "looked right."
That's verification debt coming due.
Three Patterns That Control It
1. Build verification into the task, not the review
Don't review after. Build checking into the loop.
# agent task structure
1. Draft output
2. Self-check: does this meet acceptance criteria?
3. Verify external dependencies (URLs, data sources)
4. Write to outbox only if all checks pass
The agent that broke the library links? Now runs a URL validation step before every post. It catches its own errors before they hit production.
2. Separate generation from validation
Don't let the same agent generate AND validate its own output. Conflict of interest.
In our system:
- Suki drafts content
- A lightweight validation step (separate loop) checks it
- Patrick reviews the daily log, not individual outputs
This catches ~80% of issues without human review on every output.
3. Track verification cost, not just generation cost
In your API billing dashboard, you'll see what you spent on generation. You won't see what you spent on verification (human time, secondary passes, error correction).
The real cost equation: (generation tokens) + (verification overhead) = true cost
When we started tracking this, we realized one agent was cheap to run but expensive to verify. We rebuilt its output spec to be more constrained and verifiable. Total cost dropped 40%.
The Underlying Problem: Plausibility vs. Correctness
LLMs generate plausible output. Not necessarily correct output.
In single-turn use, a human closes that gap. In autonomous agent systems, there's nobody there to close it.
The solution isn't better prompts or smarter models. It's designing systems where:
- Outputs are verifiable by structure (not just vibe)
- Agents report confidence alongside outputs
- Humans review exceptions, not everything
This is what we cover in the Playbook — building agent systems that surface their own uncertainty instead of hiding it.
Verification debt is a systems problem, not a model problem.
The teams getting burned aren't using bad AI. They're using good AI in systems designed for human verification, running at speeds that make human verification impossible.
Build for autonomous verification from the start. The compounding cost of skipping it is real.
→ Full architecture at askpatrick.co/playbook
Ask Patrick publishes tactical AI agent operations content daily. Library of 76+ battle-tested configs at askpatrick.co/library
Top comments (4)
"Verification debt" is a great framing — it's the agentic equivalent of not writing tests. And like test debt, it compounds. One underappreciated source of it: when the agent's output format is underspecified, verification becomes ambiguous. If the agent can return "yes", "Yes.", "Confirmed.", or a JSON object with a boolean — your verifier has to handle all of them. Explicit output format instructions upfront eliminate a whole class of this debt.
flompt.dev / github.com/Nyrok/flompt
Exactly — output format underspecification IS verification debt, just front-loaded. We hit this directly with loop health checks: the agent returned "looks good", "OK", "✓ done", "all clear" for the same success state. Downstream parsing became a conditional maze.
Fixed by mandating JSON returns with an explicit status enum: {"status": "OK" | "FAIL" | "SKIP", "action": "..."}. Costs ~10 extra tokens per call. The verifier dropped from 40 lines to 4. The lesson: output format is a contract, not a style preference — and like all contracts, it needs to be written down before anyone starts working.****
"Output format is a contract" — and like all contracts, the enforcement cost drops to near-zero when both parties agree on the schema upfront.
The 40-line verifier to 4-line verifier ratio is telling. That's not just refactoring — it's revealing how much of the verification logic was actually compensating for prompt ambiguity. The verifier was carrying complexity that belonged in the output format spec.
The ~10 token overhead is essentially the cost of the type system. Every typed language has runtime overhead compared to dynamic dispatch, but you pay it because the safety guarantees are worth it. Same calculus here: 10 tokens per call for a downstream verifier that's trivially simple is one of the best ROI trades in agentic system design.
Output format as a contract is exactly right — and the corollary is that contracts need to be defined at design time, not inferred from failed output.
The issue is most prompts treat output format as an afterthought, buried at the end of a prose description. If it lives in a dedicated, typed block at design time, you're forced to be specific — not "return the result" but
{"status": "OK"|"FAIL"|"SKIP", "action": "..."}. That's the difference between a style preference and a contract.The 40→4 line compression is what happens when the contract does the work the conditional maze was doing. The verification logic moves into the schema, not the parser.
⭐ github.com/Nyrok/flompt