FX-LgLL

Posted on Sep 22

SIMURG: A Guard That Catches LLM Hallucinations Mid-Stream and Heals Them

#automation #ai #machinelearning #opensource

Your LLM looks fine in the demo. Then it ships.

Somewhere between "it works on my machine" and "a user saw this", three failure modes show up in production streams:

the model falls into a repetition loop ("...and as we have seen, and as we have seen, and as we have seen..."),
it drifts into another language mid-answer,
it collapses into structural garbage — table echoes, #REF! noise, token-level static.

These are not model-intelligence failures. They are decoding failures, and they are detectable while the stream is still open. That is the entire premise of SIMURG: a streaming integrity monitor that watches every token as it arrives and raises a calibrated alarm within roughly 600 characters of the corruption onset — before the user has read the garbage.

This week we shipped v1.0.4, "Catch The Heals", and it adds two things that change the game: the guard now repairs the answer instead of discarding it, and it has a small trained deep-learning detector on board.

The problem with detecting corruption after the fact

Post-hoc linters and LLM-as-judge pipelines all share one fatal property: they only see the full answer. By then the user has the garbage. Perplexity thresholds need logprob access most serving stacks do not expose. SIMURG's five-detector numpy ensemble works differently — it consumes the stream incrementally, scores every checkpoint, and can abort mid-generation.

But aborting alone is expensive: a blind full retry means paying the model's wall-clock twice.

Self-Heal: diagnose, trim, continue, stitch

The new repair ladder turns a corrupt abort into a targeted continuation:

Diagnose — the fired corruption class is classified (repetition collapse, cross-lingual drift, regurgitation, structural breakdown).
Trim — a periodic-loop onset detector (self-similarity plus exact-mismatch run scan) localizes the exact loop start, and the released prefix is trimmed to its verified-clean boundary. The degenerate loop never enters the repair prompt.
Continue — a pathology-specific instruction goes out at slightly warmer sampling: "continue from here, never repeat" for loops, "stay in the original language" for drift.
Stitch — the continuation is guarded by a fresh sentinel; prefix plus verified tail are stitched into one answer, and verify_final re-checks the whole thing. Zero-leak guarantee preserved.

The result: one generation's wall-clock instead of two, with result.healed = True on the response object.

In live A/B testing against a real endpoint (wahoo-1.5-preview via VLLM) under high repetition pressure, the heal ladder scored 5/5 clean outcomes vs 3/5 with healing disabled. At extreme penalty settings both drop — that is an honest limit of any guard, and it is documented as such.

SIMURG Pulse: the deep tier

Statistics are robust, but they compress away sequential structure: the exact phase of a repetition loop, the cadence of script switches, the texture of structural garbage. So v1.0.4 ships a sixth, learned view.

Pulse is a 2-layer streaming transformer — 345K parameters, a 1.3 MB safetensors file. It reads the last ~600 characters of the stream as trigram-hash tokens (blake2b into 4096 buckets), runs a forward pass in about 4 ms on Apple Silicon, and emits a calibrated corruption probability via two anchors stored in the checkpoint metadata.

Training protocol:

40 clean live answers sampled from the guarded endpoint itself,
240 synthetic corruptions (CorruptBench: loops, drift, table echo, structural garbage),
onset-aware labels — a window counts as corrupt only once its right edge is 300+ characters past the true onset, so the model never learns to flag clean prefixes,
BCE-with-logits with inverse-frequency class weighting, AdamW, 8 epochs, CPU-only training in seconds.

Held-out AUROC at training time: 0.925. On the release-time evaluation split (292 onset-aware windows) the ROC lands at 0.826 — small-clean-set variance, reported with the figure so nobody has to guess.

Per-class response on single windows:

input	pulse probability
clean prose	0.002
repetition loop	1.000
cross-lingual drift	1.000
table echo	1.000
structural garbage	1.000

The contract is graceful by design: without torch installed, the numpy-only core behaves exactly as before. pip install "simurg[deep]" and the detector joins the ensemble automatically.

Train it on YOUR hallucinations

This is the part we are most excited about. If your workload has a characteristic failure mode — fabricated citations, number drift, prompt echo, domain-specific garbage — the detector can be retrained against your endpoint in one command:

export SIMURG_LIVE_URL=http://your-host:port/v1/chat/completions
export SIMURG_LIVE_MODEL=your-model-name

python -m simurg.deep.train_pulse --clean 40 --corrupt 240 --epochs 8 \
  --out ./simurg_pulse.safetensors

The trainer prints held-out AUROC and calibration anchors before saving. Point SIMURG_PULSE_WEIGHTS at the new checkpoint and the ensemble picks it up on the next process start. Weights and the full model card are on Hugging Face: MergenAI/SIMURG.

Also in this release

Free web search for agents (TinyFish) — grounded answers out of the box, bundled key included.
Guard dashboard — local UI for live guarding, session replay, and post-hoc analysis of pasted text.
Paper: SIMURG: Zero-Leak Online Detection of LLM Decoding Corruption in Production Streams.

Top comments (1)

Mateo Ruiz • Sep 22

The interesting part here is that the guard is treating corruption as a streaming state transition, rather than waiting for the final answer to decide whether something went wrong. That changes the recovery problem considerably: once you can identify a verified-clean boundary, healing becomes a controlled continuation instead of a blind retry.

I’d be careful about one thing, though: a clean continuation does not necessarily mean a semantically correct answer. The detector can establish that the output stopped exhibiting a known decoding pathology, but it still needs a separate correctness signal for things like fabricated citations, wrong numbers, or unsupported claims.

That separation could make the architecture even stronger: stream integrity → recovery → semantic verification. Different layers can then fail independently without pretending that “not corrupted” means “true.”

DEV Community