The Problem
In March 2026, Helicone — one of the most popular
LLM observability tools — was acquired by Mintlify
and went into maintenance mode. Thousands of
developers were left looking for an alternative.
But the deeper problem wasn't just Helicone.
Every LLM observability tool available today has
one of these problems:
- Cloud-locked (your prompts leave your server)
- Acquired and abandoned
- Only does one thing (cost OR observability OR evals)
- Requires sending sensitive data to third parties
For enterprises in healthcare, finance, and
government — none of these tools work. They
legally cannot send prompts to external servers.
What I Built
Ajah is a self-hostable LLM gateway that sits
between your application and any LLM provider.
It does 5 things in one tool:
1. Gateway Proxy
Point your app at Ajah instead of OpenAI directly.
One line change. Supports 9 providers automatically
detected from your API key prefix.
2. RAG Verification
When your app uses retrieval-augmented generation,
Ajah verifies whether the LLM response is actually
grounded in your source documents. Contradictions
are flagged before they reach users.
3. Hallucination Flagging
Every response is scored for hallucination risk
in parallel — zero latency added. Uses local ML
models, no external API calls.
4. Multi-Agent Session Tracing
Visual step-by-step trace of every agent run.
Cost, quality, and
Top comments (1)
Nice work! The LLM gateway/observability space has so much churn right now. We're seeing the same need from our users — they want one endpoint for all models without the complexity. Curious if you've compared your latency overhead vs going direct to providers?