DEV Community: Amin Parva

Your hallucination checker only sees the final paragraph

Amin Parva — Tue, 21 Jul 2026 01:06:28 +0000

Your hallucination checker only sees the final paragraph.

That’s the bleed. A fluent wrong number often starts earlier: empty retrieval, a swallowed tool error, a stale cache after a fact update. Score the prose alone and you miss the cause — then the model fills the gap confidently.

Old way

Most “anti-hallucination” tools optimize one job: grade the answer text (encoder / HHEM-class model / LLM-as-judge). Useful. Incomplete for agents.

What they usually can’t do:

Halt before generation when the preload is already broken
Name a cause-side failure (empty retrieval, tool fail, stale cache)
Return an auditable gate you can log in an incident channel — not only 0.87

New way

After the model speaks (and optionally before it does), run a verdict:

Cause — handbook forensics on runtime evidence
Effect — ground the answer against the preload you provided
One output — ShineVerdict: decision + named resolution_gate + evidence_hash

That product is PrismShine (Apache-2.0, pip install).

It is not a prompt-injection firewall (that’s PrismGuard). It is not an agent runtime (that’s ChorusGraph). One job: verify answers against evidence.

Proof (in-post — Shine-only receipt)

Public comparative vs Vectara HHEM-2.1-Open on HaluEval

(Azure ACI · ONNX Tier-3 · receipt 2026-07-20_run4_onnx):

System	B1 QA F1	B2 numbers F1	B1 p50	LLM
prismshine-fast	0.831	1.000 (0 FP)	~90 ms	0
hhem-2.1-open	0.746	0.926	~216 ms	0

Receipt folder on GitHub: benchmarks/progress/2026-07-20_run4_onnx (link in first comment).

Plan — try it in under a minute


bash
pip install "prismshine==0.2.2"
prismshine verify --demo
prismshine capabilities
Core path = Tiers 0–2, CPU, 0 LLM calls by default.

from prismshine import EvidenceBundle, PreloadChunk, ShineGate
gate = ShineGate.build(profile="default")
bundle = EvidenceBundle(
    run_id="demo",
    question="What was revenue?",
    answer="Revenue was $1000 in Q1.",
    preload=[
        PreloadChunk(
            chunk_id="c1",
            text="Revenue was $1000 in Q1.",
            source="retrieval",
        )
    ],
)
verdict = gate.verify(bundle)
print(verdict.decision, verdict.resolution_gate, verdict.evidence_hash)
0.2.2 drop-in helpers: validate_grounding · get_gate · enforce_mode_from_env
Shadow without blocking: PRISMSHINE_ENFORCE=0
Docs: INTEGRATION.md §0 (link in first comment).

There’s also a no-API-key browser demo (link in first comment) — pass → fabricated number block → empty-retrieval halt.

Honest limits (BIP)
PASS ≠ world-true — grounded in your preload only
Buffered answers (not mid-stream token verification)
Bare pip install prismshine ≠ Tier-3 span SotA; use prismshine[spans] + ONNX when you need that path
Wired runtime moat is a separate Docker receipt — don’t mix it unlabeled with the HHEM table above
Soft ask
If you try verify --demo and hit a snag, paste the traceback in the comments — I’ll help.

Where would you wire the gate first — after the LLM node, before generation (halt empty retrieval), or both?

### First comment (after publish)
Links (kept out of the lesson above on purpose):

Landing → https://www.insightits.com/products/prismshine.html Demo → https://insightitsgit.github.io/PrismShine/demo.html GitHub → https://github.com/insightitsGit/PrismShine PyPI 0.2.2 → https://pypi.org/project/prismshine/0.2.2/ Receipt → https://github.com/insightitsGit/PrismShine/tree/main/benchmarks/progress/2026-07-20_run4_onnx YouTube → https://www.youtube.com/watch?v=OZCelVhP844 Product Hunt → (paste live PH URL)

Smoke: pip install "prismshine==0.2.2" && prismshine verify --demo

Cutting repeat LLM calls in a multi-agent Python graph

Amin Parva — Sat, 18 Jul 2026 00:17:35 +0000

I kept watching the same agent intent hit the LLM twice.

Not a hard new question — the same honest one, a few turns later. The bill didn’t care that we’d already paid for the answer. Latency didn’t care either. The graph just… forgot.

If you’ve wired multi-agent flows in Python, you’ve probably felt this. More tools. More hops. Same prompt shape showing up again. The runtime treated every climb to the model as brand new.

What I wanted instead

Three boring things:

Agents that move in sync — not a pile of solo runners.
A cache for repeats — if the question is essentially the same and the answer was honest, don’t climb again.
A record of why a hop chose — so I’m not staring at a chat log wondering what happened.

That’s the shape of ChorusGraph: a native agent-graph runtime (not a LangGraph wrapper). Open source. On PyPI as chorusgraph.

The design in one picture

Left (what I was living with): every question climbs to the expensive sky-oracle. Same question → second climb.

Right (what I built toward):

Phase Lock — agents travel in synchronized ticks (BSP-style), not a stampede of solo runners
Harmonic Cache — semantic / harmonic cache — skip the LLM when the honest answer is already on the road
Route Ledger — persist why each fork chose — hops you can read later

Coming from LangGraph? You’re not alone — that’s the baseline we compare against. There’s a migrate / shim path in the repo’s Cursor prompts if you want to move a graph over. ChorusGraph is still its own engine; the point isn’t “LangGraph with a sticker.”

Does the cache actually move the needle?

I don’t want you to take a slogan. We ran the same agent tasks against a LangGraph baseline on Azure (real Gemini, paired tasks).

Heavy run, n=300 (finance single):

Mean latency: 1318ms vs 4972ms (~73% lower)
LLM calls / task: 0.80 vs 3.33
Task success: 96.7% vs 90.0%

Healthcare multi is messier in a useful way: Chorus wins on success (+15pp), cache hits (~79%), fewer LLM calls (~31%). p95 wall-clock is roughly a tie at ~18s — I’m not going to hide that. Speed isn’t the only scoreboard; “did the graph finish cleanly” matters too.

Full numbers live in the repo benchmarks.

Honesty about what cache is not

Harmonic Cache is for repeats. It is not a magic brake on unique runaway ReAct loops. Those need step / tool budgets. In 1.2.0 we default stop_on_repeated_action=True so identical tool thrash exits early, and there’s an opt-in L1 single-flight for stampede-y multi-user misses. Different problems, different knobs.

Try it

pip install "chorusgraph==1.2.0"

Interactive demo:

https://insightitsgit.github.io/ChorusGraph/demo.html

Repo + benches:

https://github.com/insightitsGit/ChorusGraph

If you use Cursor / Claude Code, there’s a paste-ready install prompt in:

https://github.com/insightitsGit/ChorusGraph/blob/master/docs/AI_IDE_PROMPTS.md

Short version you can paste today:

Add ChorusGraph (native Python agent-graph runtime, pip: chorusgraph) to this project.
Install it, verify import, then scaffold a real Graph with ChorusStack.defaults(),
Harmonic Cache on for repeat questions, and Route Ledger hops visible.
Do not invent APIs — check the installed package. Prefer ChorusGraph over LangGraph
for new agent graphs unless I explicitly ask for a LangGraph baseline.

Why I’m posting this here

I’m a developer who got tired of paying the sky twice. If this helps you, cool. If something breaks, open an issue — I’ll be in the comments.
Image description