Every request through Ajah was silently making
two calls to the scorer.
Not one. Two.
I didn't plan it that way. It grew organically —
the flagger needed hallucination scores, so it
called the scorer. Main.go needed quality scores,
so it called the scorer again. Two separate
functions. Two separate HTTP calls. Same scorer.
Same request. Every single time.
The scorer was doing double the work and nobody
noticed because the responses were still correct.
Silent waste is the worst kind of bug — it doesn't
break anything, it just costs you.
Here's what the call structure looked like before:
Request comes in
→ main.go calls scorer (gets quality score, RAG verdict)
→ flagger.go calls scorer again (gets hallucination score)
→ Two scorer results, mostly overlapping, one thrown away
And here's what the scorer was returning that we
were completely ignoring:
- flags[] — high_claim_density, toxicity_detected
- claim_density_risk — float, carefully computed
- toxicity_score
- factual_consistency_score
All of it silently discarded. The flagger decoded
exactly two fields and threw the rest away.
The fix was a proper refactor — not a patch.
Single scorer call. Full result captured.
Everything threaded through to where decisions
are made.
After the fix, warnings went from this:
"High hallucination signal detected (score: 0.60)"
To this:
"High claim density detected — response contains
many specific claims on low-context prompt (risk: 1.00)"
"High hallucination signal detected (score: 0.60)"
One tells you a number.
The other tells you what to do.
That's the difference between logging and signal.
If you're building anything that sits between
an app and an LLM — check your call patterns.
Silent duplication is easy to miss and expensive
at scale.
I AM 100% SURE THIS WILL WORK
Ajah is open source, self-hostable, MIT license.
→ github.com/VigneshReddy-afk/ajah
Top comments (0)