I Was Paying for Hallucinated Outputs — Here's What I Did About It (1779868666273)

Every time an AI agent hallucinates, you pay twice.

Once in tokens. Once in debugging time.

I tracked my token usage over a month and found that ~18% of all API calls produced outputs that were either wrong, fabricated, or irrelevant. That's nearly a fifth of my budget gone to confident nonsense.

The hidden cost of hallucinations

When an agent confidently returns the wrong code:

• You spend 15 minutes reviewing it (trusting it, usually)
• You spend 30 minutes debugging why it doesn't work
• You spend 10 minutes writing a new prompt to fix it
• The agent generates another wrong answer

This loop repeats until you catch it. And you don't always catch it.

What I built instead

A verification layer that sits between the model and my workspace. It runs after the model generates but before the output touches my codebase.

It checks:

1. Are there fabricated citations? (common in research tasks)
2. Is the code syntactically valid? (surprisingly often, no)
3. Does the output contain leaked system prompts? (happens more than you'd think)
4. Are there safety refusals disguised as answers?
5. Does the output actually address the input prompt?

The result

My token waste dropped from ~18% to under 3%. The verification runs in under 100ms on CPU. No GPU needed.

Download: https://agent-download-site.vercel.app

Free, model-agnostic, runs anywhere Python runs. Check your own hallucination rate — you might be surprised what you're paying for.

---

Built for developers who want their agents to actually be useful.