DEV Community: FX-LgLL

THX-01 Just Processed 70 Million Tokens. Our Open-Source Decision Model Is Just Getting Started.

FX-LgLL — Mon, 12 Oct 2026 09:36:54 +0000

70 million tokens processed. 322 million parameters. One forward pass. Free API. Fully open-source.

A few days ago, we open-sourced THX-01, our 322M-parameter non-autoregressive decision model designed to make AI decisions faster, more efficient, and accessible to everyone.

Today, we've reached another milestone: over 70 million tokens processed through our THX-01 API since launch.

And we're just getting started.

Why THX-01?

Most AI applications don't actually need to generate long responses.

Sometimes, all you need is a reliable decision:

Which department should handle this ticket?
Is this message spam?
What number appears in this document?
Which passage supports the answer?
How confident is the model in its decision?

Instead of generating tokens one by one, THX-01 processes a document or message alongside typed questions and returns structured answers with calibrated probabilities in a single forward pass.

On our GPU benchmark, inference takes approximately 10 milliseconds.

THX-01 vs TypeSafe Jev

We evaluated THX-01 against TypeSafe Jev 1.13 on our internal benchmark of 2,843 multilingual support tickets.

Metric	THX-01	Jev 1.13
Accuracy	98.4%	97.4%
Calibration Error (ECE)	0.003	0.007
Measured Latency	~10 ms	331 ms
Open-Source Weights	Yes	No

THX-01 outperformed Jev on this classification benchmark, with approximately 33x lower measured latency.

The latency figures come from different serving environments, so they should not be interpreted as a hardware-controlled comparison.

Our results show that small, specialized decision models can deliver competitive performance without requiring billions of additional parameters.

Built for Efficient Decision-Making

THX-01 is trained using Reinforcement Learning for Calibrated Decisions (RLCD).

Rather than optimizing only for correct predictions, RLCD encourages probability estimates that reflect the model's actual uncertainty.

This is particularly useful for AI agents and production applications that need to decide whether to accept a prediction, request additional information, or escalate to another model.

THX-01 supports:

Choice: Classification among dynamically provided options
Yes/No: Binary decisions with confidence scores
Score: Ordered evaluations
Number: Extracting numerical values from documents
Excerpt: Extracting exact text passages
Citations: Identifying supporting evidence

The model supports 18 post-training languages, including Azerbaijani, English, Russian, and Turkish.

And at just 322M parameters, it can even run locally on smartphones with a compatible inference runtime.

The THX-01 API Is Free

We believe developers should be able to experiment with new AI architectures without worrying about infrastructure costs.

That's why we've opened the THX-01 API with a free tier of 200 decisions per minute.

You can test it with a simple request:

curl https://api.hal-x.ai/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "state": "My card was charged twice for one order",
    "questions": {
      "team": {
        "type": "choice",
        "question": "Which team should handle this?",
        "criteria": {
          "billing": "Billing issues",
          "technical": "Technical issues",
          "sales": "Sales inquiries"
        }
      }
    }
  }'

Check the API documentation for the current authentication requirements and usage limits.

API Documentation: https://api.hal-x.ai/docs/thx-01

Everything Is Open-Source

We released THX-01 under the Apache 2.0 license.

Developers can access the model weights, source code, and training resources on Hugging Face.

You can also install the Python package directly:

pip install thx01

Model & Source Code: https://huggingface.co/doofz/THX-01

Interactive Demo: https://huggingface.co/spaces/doofz/THX-01-demo

What's Next?

At HAL-X AI, we're working toward a future where AI systems don't just generate answers, but make reliable, measurable, and well-calibrated decisions.

We believe the next generation of AI infrastructure will need specialized models that are smaller, faster, and capable of expressing uncertainty.

70 million tokens processed is an exciting milestone for our team, but this is only the beginning.

We're continuing to improve THX-01, expand its capabilities, and contribute to the global open-source AI ecosystem.

Our mission is simple:

We need more AI models that know how to make decisions, how confident they should be, and when they don't know.

And we're building them for everyone.

Try THX-01 today:

Free API: https://api.hal-x.ai/docs/thx-01
Hugging Face: https://huggingface.co/doofz/THX-01
Python: pip install thx01

We Open-Sourced a 322M Decision Model. Now Its API Is Free

FX-LgLL — Sat, 10 Oct 2026 21:34:17 +0000

Two days ago, we released THX-01, an open-source decision model that outperforms Jev on several benchmarks and can even run locally on a smartphone. Today, anyone can use it through our free API.

Hi everyone! I'm Farid, Co-Founder of HAL-X AI.

A couple of days ago, our team open-sourced THX-01, a 322M-parameter multilingual decision model designed to make AI decision-making faster, cheaper, and more accessible.

Yesterday, we opened our hosted API to everyone. No API key, no sign-up, and no payment required.

And the response has been incredible!

Since launch, we've already processed:

25M+ tokens
211,000+ decisions

We're excited to see developers experimenting with what a small, specialized decision model can do.

What Is THX-01?

Most applications don't actually need an LLM to generate paragraphs of text.

Sometimes, you just need to answer a simple question:

Is this email spam?
Which department should handle this support ticket?
What is the invoice total?
Does this document mention a particular condition?
Which passage supports this decision?

Using a large generative model for these tasks can introduce unnecessary latency and cost.

THX-01 takes a different approach.

It's a non-autoregressive model that processes a document or message alongside typed questions and returns structured answers with calibrated probabilities in a single forward pass.

Approximately 10 milliseconds on our benchmark hardware.

No token-by-token generation.

Try the Free API

You can make your first request right now.

No authentication required.

curl https://api.hal-x.ai/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "state": "My card was charged twice for one order",
    "questions": {
      "team": {
        "type": "choice",
        "question": "Which team handles this?",
        "criteria": {
          "billing": "billing",
          "tech": "technical",
          "sales": "sales"
        }
      }
    }
  }'

THX-01 evaluates the available choices and returns the predicted decision with probabilities.

The API supports:

Type	What it does
Choice	Classification with probabilities
Yes/No	Binary decisions
Score	Ordered scoring
Number	Extract exact numbers from documents
Excerpt	Extract verbatim passages
Citations	Identify supporting text

You can submit multiple questions in a single request.

Full API documentation: https://api.hal-x.ai/docs/thx-01

How Does It Compare to Jev?

We benchmarked THX-01 against TypeSafe Jev 1.13 using 2,843 multilingual support tickets across four languages.

Here are the results:

Metric	THX-01	Jev 1.13
Accuracy	98.4%	97.4%
Calibration error	0.003	0.007
Measured latency	~10 ms	331 ms
Open weights	Yes	No

THX-01 achieved higher accuracy and better calibration on this benchmark, with approximately 33x lower measured latency.

The latency measurements use different serving setups, so they're not a controlled hardware-to-hardware comparison.

We don't claim THX-01 outperforms Jev everywhere, but these results demonstrate how competitive specialized, lightweight decision models can be.

Small Enough to Run on Your Phone

THX-01 contains just 322 million parameters.

It was designed to support efficient inference without requiring a massive GPU infrastructure.

With a suitable runtime, it can even run locally on smartphones.

You can also install the Python package:

pip install thx01

The model supports 18 post-training languages, with particular attention to Azerbaijani and multilingual robustness.

Everything Is Open-Source

We're releasing THX-01 under Apache 2.0, including its model weights, source code, and training scripts.

We believe developers should have the freedom to run, modify, deploy, and build on top of their AI models.

At HAL-X AI, our goal is to contribute to the global open-source AI ecosystem and make efficient AI accessible to more people.

That's also why we've made the hosted API free to use, subject to rate limits.

Get Started

Free API: https://api.hal-x.ai/docs/thx-01

Model weights and code: https://huggingface.co/doofz/THX-01

Interactive demo: https://huggingface.co/spaces/doofz/THX-01-demo

Python package:

pip install thx01

We're still improving THX-01, and we'd love to hear feedback from developers building AI agents, automation pipelines, document processing systems, and classification services.

If you build something using THX-01, we'd love to see it!

We made our decision model's API free (and the weights are open)

FX-LgLL — Fri, 09 Oct 2026 19:43:18 +0000

Most "AI decisions" in production aren't open-ended generation. They're classification in disguise:

which team should handle this ticket?
is this e-mail phishing?
what's the total on this invoice?
which sentence in the contract supports that answer?

Teams usually send each of these to a large LLM: 1–2 seconds and a bill per call, a JSON answer to parse, and a "confidence" that means nothing.

We built THX-01 for exactly these decisions. Today we're making its hosted API free.

Try it now (no key, no sign-up)

curl https://api.hal-x.ai/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "state": "Hi, I was charged twice for one order and nobody answers. Refund me!",
    "questions": {
      "team":   {"type": "choice", "question": "Which team handles this?",
                 "criteria": {"billing": "billing", "tech": "technical", "sales": "sales"}},
      "refund": {"type": "noul", "question": "Does the customer want money back?"},
      "urgency":{"type": "score", "question": "How urgent?", "criteria": ["low", "medium", "high"]}
    }
  }'

Every answer comes back with a probability for each option, computed in a single forward pass of about 10 ms.

The free tier allows 200 decisions per minute per IP. One question is one decision.

What it can answer

type	returns
`choice`	one option, with a probability for every option
`noul`	P(yes)
`score`	a level on an ordinal scale
`number`	a value stated in the document, or `null` if it isn't there
`excerpt`	a verbatim span with character offsets
`"cite": true`	the passages that support any answer

How it compares

On our 2,843-ticket benchmark (Azerbaijani, Russian, English, Turkish; clean, corrupted, messy and independently written sets):

model	avg accuracy	latency
THX-01 (322M, open)	98.4	~10 ms
TypeSafe Jev 1.13	97.4	331 ms
Kev-4B	92.8	830 ms

The model's confidence is calibrated, so a single threshold lets it handle about 75% of tickets automatically with zero errors on all four test sets.

Run it yourself

The weights are Apache 2.0:

pip install thx01

import thx01
agent = thx01.load("doofz/THX-01")
agent.decide("My card was charged twice", {
    "team": {"type": "choice", "question": "Which team?",
             "criteria": {"billing": "billing", "tech": "technical"}}})

It runs on a laptop CPU in a fraction of a second.

Model: https://huggingface.co/doofz/THX-01
Live demo: https://huggingface.co/spaces/doofz/THX-01-demo

We Open-Sourced THX-01: A 322M AI Decision Model Small Enough to Run on a Smartphone

FX-LgLL — Thu, 08 Oct 2026 21:26:50 +0000

Today, we're introducing THX-01, an open-source, non-autoregressive decision model developed by HAL-X AI in Baku, Azerbaijan.

With just 322 million parameters, THX-01 is lightweight enough for deployment on CPUs and potentially smartphones, without requiring expensive GPU infrastructure.

Unlike traditional large language models, THX-01 doesn't generate text token by token. Instead, it processes inputs and returns structured decisions with calibrated confidence scores in a single forward pass.

Key Highlights

322M parameters, built on mmBERT with a specialized decision head.
~10 ms inference per request on a GPU.
CPU-compatible, with a compact architecture suitable for mobile and edge deployment.
98.4% accuracy on our multilingual support-ticket classification benchmark, compared to 98.5% for Claude Sonnet 5.5.
18 languages, including Azerbaijani, English, Russian, and Turkish.
Calibrated confidence scores, allowing applications to make decisions based on estimated uncertainty.
Apache 2.0 license, with publicly available model weights.

Why We Built It

Not every AI task requires a massive generative model.

Classifying messages, routing customer requests, detecting spam, extracting document values, and making structured decisions can often be handled by significantly smaller models.

THX-01 was designed specifically for these workloads, prioritizing speed, efficiency, and reliability.

Our goal is to make intelligent decision-making accessible across environments, from production servers to resource-constrained edge devices.

Get Started

Install THX-01 using pip:

pip install thx01

Load the model:

import thx01

agent = thx01.load("doofz/THX-01")

Open Source

THX-01 is available under the Apache 2.0 license.

Model and documentation: https://huggingface.co/doofz/THX-01

Built by HAL-X AI, Baku, Azerbaijan.

We believe the future of AI isn't only about building larger models. It's also about building smaller, specialized models that make powerful capabilities accessible to everyone.

We open-sourced THX-01: a 322M decision model that matches Sonnet 5.5 on ticket classification at ~10 ms (Apache 2.0, pip install thx01)

FX-LgLL — Thu, 08 Oct 2026 18:24:29 +0000

Today we're releasing THX-01, a model for the boring decisions that
agents and backends make thousands of times a day: route this ticket, is this spam, which team, what's the invoice total,
which sentence supports that.

What it is

322M parameters (mmBERT-base encoder + 15M decision head), Apache 2.0
Non-autoregressive: you send a state plus typed questions, and it returns a probability for every option in one forward pass. No generation, no JSON parsing, no retries.
Question types: choice, noul (yes/no), score, plus number (pulls a value stated in the document, or null), excerpt (verbatim span with offsets) and "cite": true on any question
~10 ms per request on one GPU, ~3,000 decisions/s batched, and it runs on CPU
Speaks the /v1/systemone wire format, so it is a drop-in for existing TypeSafe-style service layers

Numbers (our support-ticket benchmark, 2,843 tickets in az/ru/en/tr, 15 categories)

model	avg acc	latency
THX-01 (322M)	98.4	~10 ms
Claude Sonnet 5.5	98.5	1.5 s
Wahoo 1.5	97.8	145 ms
TypeSafe Jev 1.13	97.4	331 ms
Kev-4B	92.8	830 ms

Because it's trained with a strictly proper scoring-rule reward, the probabilities are calibrated (ECE 0.003). With a single
confidence threshold it handled ~75% of tickets fully automatically with zero errors on all four test sets.

pip install thx01

import thx01
agent = thx01.load("doofz/THX-01")
agent.decide("My card was charged twice for one order", {
    "team": {"type": "choice", "question": "Which team handles this?",
             "criteria": {"billing": "billing", "tech": "technical", "sales": "sales"}}})

Model + docs: https://huggingface.co/doofz/THX-01

SIMURG: A Guard That Catches LLM Hallucinations Mid-Stream and Heals Them

FX-LgLL — Tue, 22 Sep 2026 12:01:37 +0000

Your LLM looks fine in the demo. Then it ships.

Somewhere between "it works on my machine" and "a user saw this", three failure modes show up in production streams:

the model falls into a repetition loop ("...and as we have seen, and as we have seen, and as we have seen..."),
it drifts into another language mid-answer,
it collapses into structural garbage — table echoes, #REF! noise, token-level static.

These are not model-intelligence failures. They are decoding failures, and they are detectable while the stream is still open. That is the entire premise of SIMURG: a streaming integrity monitor that watches every token as it arrives and raises a calibrated alarm within roughly 600 characters of the corruption onset — before the user has read the garbage.

This week we shipped v1.0.4, "Catch The Heals", and it adds two things that change the game: the guard now repairs the answer instead of discarding it, and it has a small trained deep-learning detector on board.

The problem with detecting corruption after the fact

Post-hoc linters and LLM-as-judge pipelines all share one fatal property: they only see the full answer. By then the user has the garbage. Perplexity thresholds need logprob access most serving stacks do not expose. SIMURG's five-detector numpy ensemble works differently — it consumes the stream incrementally, scores every checkpoint, and can abort mid-generation.

But aborting alone is expensive: a blind full retry means paying the model's wall-clock twice.

Self-Heal: diagnose, trim, continue, stitch

The new repair ladder turns a corrupt abort into a targeted continuation:

Diagnose — the fired corruption class is classified (repetition collapse, cross-lingual drift, regurgitation, structural breakdown).
Trim — a periodic-loop onset detector (self-similarity plus exact-mismatch run scan) localizes the exact loop start, and the released prefix is trimmed to its verified-clean boundary. The degenerate loop never enters the repair prompt.
Continue — a pathology-specific instruction goes out at slightly warmer sampling: "continue from here, never repeat" for loops, "stay in the original language" for drift.
Stitch — the continuation is guarded by a fresh sentinel; prefix plus verified tail are stitched into one answer, and verify_final re-checks the whole thing. Zero-leak guarantee preserved.

The result: one generation's wall-clock instead of two, with result.healed = True on the response object.

In live A/B testing against a real endpoint (wahoo-1.5-preview via VLLM) under high repetition pressure, the heal ladder scored 5/5 clean outcomes vs 3/5 with healing disabled. At extreme penalty settings both drop — that is an honest limit of any guard, and it is documented as such.

SIMURG Pulse: the deep tier

Statistics are robust, but they compress away sequential structure: the exact phase of a repetition loop, the cadence of script switches, the texture of structural garbage. So v1.0.4 ships a sixth, learned view.

Pulse is a 2-layer streaming transformer — 345K parameters, a 1.3 MB safetensors file. It reads the last ~600 characters of the stream as trigram-hash tokens (blake2b into 4096 buckets), runs a forward pass in about 4 ms on Apple Silicon, and emits a calibrated corruption probability via two anchors stored in the checkpoint metadata.

Training protocol:

40 clean live answers sampled from the guarded endpoint itself,
240 synthetic corruptions (CorruptBench: loops, drift, table echo, structural garbage),
onset-aware labels — a window counts as corrupt only once its right edge is 300+ characters past the true onset, so the model never learns to flag clean prefixes,
BCE-with-logits with inverse-frequency class weighting, AdamW, 8 epochs, CPU-only training in seconds.

Held-out AUROC at training time: 0.925. On the release-time evaluation split (292 onset-aware windows) the ROC lands at 0.826 — small-clean-set variance, reported with the figure so nobody has to guess.

Per-class response on single windows:

input	pulse probability
clean prose	0.002
repetition loop	1.000
cross-lingual drift	1.000
table echo	1.000
structural garbage	1.000

The contract is graceful by design: without torch installed, the numpy-only core behaves exactly as before. pip install "simurg[deep]" and the detector joins the ensemble automatically.

Train it on YOUR hallucinations

This is the part we are most excited about. If your workload has a characteristic failure mode — fabricated citations, number drift, prompt echo, domain-specific garbage — the detector can be retrained against your endpoint in one command:

export SIMURG_LIVE_URL=http://your-host:port/v1/chat/completions
export SIMURG_LIVE_MODEL=your-model-name

python -m simurg.deep.train_pulse --clean 40 --corrupt 240 --epochs 8 \
  --out ./simurg_pulse.safetensors

The trainer prints held-out AUROC and calibration anchors before saving. Point SIMURG_PULSE_WEIGHTS at the new checkpoint and the ensemble picks it up on the next process start. Weights and the full model card are on Hugging Face: MergenAI/SIMURG.

Also in this release

Free web search for agents (TinyFish) — grounded answers out of the box, bundled key included.
Guard dashboard — local UI for live guarding, session replay, and post-hoc analysis of pasted text.
Paper: SIMURG: Zero-Leak Online Detection of LLM Decoding Corruption in Production Streams.

Free Unlimited Web Search for AI Agents

FX-LgLL — Wed, 02 Sep 2026 18:59:51 +0000

AI agents need access to the web.

The problem is that web search APIs can be expensive, limited, or complicated to integrate.

SIMURG Search gives AI agents a simple way to access web search for free.

What is SIMURG?

SIMURG is an open-source protection layer for LLMs.

It monitors model generation and detects when the output starts going wrong, such as repetition loops, language drift, garbage output, or other decoding failures. When corruption is detected, SIMURG can stop the generation and retry it before the bad output reaches the user.

SIMURG also includes a web search layer that allows agents to retrieve real information from the internet and use it for grounding before answering.

SIMURG Search

With SIMURG Search, your AI agent can search the web, retrieve sources, and use the results to verify information instead of relying only on the model's memory.

It is designed to be simple:

pip install simurg

Then you can use the built-in web search directly from your agent.

Why use it?

Free web search for AI agents

Open source

Simple installation

Real-time web information

Useful for local, quantized and self-hosted models

SIMURG combines generation protection with web grounding, giving AI agents a simple way to become more reliable.

Get Started

Install SIMURG:

pip install simurg

GitHub repository:

https://github.com/doofzoff/SIMURG

If your AI agent needs web access without adding another complicated infrastructure layer, give SIMURG Search a try.

I found a completely free web search for AI agents.

FX-LgLL — Sat, 29 Aug 2026 19:21:02 +0000

A week ago my agent confidently described a treaty that does not exist. It even had a date.

The fix is simple: before the model commits to an answer, make it check the web.
I contributed this to SIMURG
(https://github.com/doofzoff/SIMURG) and it just shipped as v1.0.3.

Free means free

SIMURG web search is a web layer for AI agents powered by TinyFish. 30
requests per minute, $0, no card, no wallet. The package ships a bundled free
key, so it works out of the box:

pip install simurg
python3 -m simurg.websearch "when was the Y2K bug" --json

The part I care about: ground()

Search results are context. Verdicts are decisions:

from simurg import websearch
check = websearch.ground("Y2K bug")
# check["verdict"]: "attested" | "thin" | "no_record"

attested: feed check["evidence"] into your model's context. thin: treat with
caution. no_record: the subject is probably fabricated, abstain instead of
asserting.

One honest detail: "Zorbachian treaty of 1874" returns real 1874 treaties from
any search engine. So ground() requires the subject itself to be echoed in the
evidence, generic hits downgrade to thin.

Stdlib only, zero new dependencies, and it degrades instead of crashing. In
our internal checks on factual prompts, hallucinations dropped to the 1-5%
range.

Code, examples and a 13 page paper: https://github.com/doofzoff/SIMURG

Give your agent an internet. It costs nothing.

I almost killed all GLM 5.3 Flash 2 bit hallucinations, with SIMURG

FX-LgLL — Fri, 28 Aug 2026 13:53:32 +0000

I ran a little experiment tonight and I have to share it, because the result honestly surprised me.

The setup

I connected our GLM 5.3 Flash in 2 bit to SIMURG. If you have played with heavily quantized models you know the tradeoff. They are fast and cheap, but 2 bit hurts, and this one hallucinated a lot. Confident wrong dates, made up people, numbers that came out of nowhere. Exactly the kind of thing that looks perfectly fluent and is just false.

So I plugged it into SIMURG Monolith and started using it like a normal user.

What actually happened

Here is the fun part. I did not retrain anything offline. I just asked questions, and when the model made something up I hit dislike, and when it was right I hit like.

SIMURG learns in real time. The serving loop is the training loop, so every thumbs up or thumbs down is one training step. After a fairly short session, the guard was already flagging the fabrications the model kept producing, and abstaining instead of letting them through.

It was not magic and I am not going to claim 100 percent. But watching a 2 bit model go from hallucinating freely to almost silent on the bad answers, live, with nothing but my own feedback, was a genuinely cool thing to witness.

What SIMURG is, in short

SIMURG is an LLM hallucination guard. It does a few things at once:

reads the model's own token probabilities to spot facts the model is unsure about
grounds claims against real sources (Wikipedia and the web), so it catches both made up subjects and wrong details on real ones
abstains instead of confidently lying when a claim cannot be trusted
and the big one, it learns in real time from your like and dislike feedback, so it adapts to your model and your traffic on the fly

What is it for? You put it in front of any OpenAI compatible endpoint, wire your existing like and dislike buttons to it, and it starts catching your model's specific failure modes right away. No batch retraining, no separate judge model.

https://github.com/doofzoff/SIMURG
or
pip install simurg

GLM-5.3-Flash is a total blast when you kill its hallucinations

FX-LgLL — Thu, 27 Aug 2026 08:38:32 +0000

So as you all know, GLM 5.3 Flash dropped yesterday and it is a super model. I am running it at 2 bit quant and it is really doing its thing.

But as you know, quantization kills quality, and this one is extra fun because the model was pretrainsed mostly on Chinese, so it sometimes just starts writing code or chatting with me in Chinese out of nowhere. You are mid conversation and suddenly you get a sentence of Mandarin in the middle of your Python.

Anyway, SIMURG came out the day before yesterday so I decided to give it a shot. The cool part is that I configured it for exactly the hallucination type I needed: the model switching to Chinese mid response. I hooked it up to my API through the layer and now I always get clean perfect output, no more random language switches. Honestly the model feels like it is on par with Fable 5 in my daily use once the hallucinations are gone.

If you are running GLM at low bit rates, this one is worth it.
repo: https://github.com/doofzoff/SIMURG
pypi: https://pypi.org/project/simurg/
install: pip install simurg

I just killed hallucinations on my 2 bit Qwen3.8 27B

FX-LgLL — Wed, 26 Aug 2026 13:41:00 +0000

Yesterday tried out a small tool called SIMURG. It sits in front of an OpenAI-compatible endpoint and monitors the generation. If it detects that the model is going off the rails, it stops the request and retries it. Hallucinations mostly, and right now i have really consumer GPU RTX 3060

I tested it with Qwen3.8 27B running a 2-bit quant on an RTX 3060. The model is surprisingly capable for its size, but at 2-bit I was getting quite a few hallucinations and made-up facts and actually bad, i mean not bad, and not so good due to hallucinations because of Chinese symbols and so on.

So just used SIMURG, the responses have been noticeably more consistent so far. It's still early and I'm testing how well it holds up across different prompts, but the difference is pretty noticeable on this particular setup and actually not bad i trained this tool on my type of hallucination when model starts to do shitty tool calls and leaks its own tool call style so yep and simurg just aborting it and end user never sees hallucination and btw this is machine learning model that contains 15 parameters.

You can install it if you want:

pip install simurg
Or link it's open-sourced: GitHub

Five detectors, one bad merge: why our LLM corruption guard flagged 43% of healthy output

FX-LgLL — Tue, 25 Aug 2026 10:31:54 +0000

We run a self-hosted ~300B reasoning model in production. It writes macroeconomic desk reports in Azerbaijani and English.

Every so often, it stops.

Not crashes. Not refuses. Mid-sentence, a competent economic analysis turns into a fabricated Chinese news article. Or a software README, complete with pip install instructions. Or a Persian name, repeated seven times. Or a wall of spreadsheet cells starting with #REF!.

The user is watching this happen, token by token.

We call this decoding corruption*, and it is not hallucination. Hallucination is the model being wrong about the world. This is the model no longer producing an answer at all. There is a large literature on detecting the first one. There is almost nothing on the second, which is strange, because anyone who operates a self-hosted model has seen it.

This post is about what we learned building a detector for it. The headline lesson is not the one I expected going in.

Why post-hoc detection is useless here

Most hallucination detectors run after generation completes. For factuality that is fine. For this, it is not.

Two reasons. First, streaming: the user already watched the model derail. A verdict that arrives after the fact has failed. Second, cost: on a reasoning model, one corrupt generation burns minutes of GPU time. We measured a 13,200-character table echo that took 269 seconds to generate to completion. Every one of those seconds was wasted.

So the detector has to run inside the stream, cheap enough to be invisible next to decoding.

The design

One pass over the character stream. A single state object ingests one character at a time and incrementally maintains every statistic anything downstream will need. Detectors are stateless reads over that shared state, evaluated only at checkpoints.

That constraint matters more than it sounds. It means adding a sixth detector costs zero additional passes. Everything is O(1) amortized per character with bounded memory, so a runaway generation is processed in constant space.

Five detectors sit on top:

Rules. Deterministic thresholds on the snapshot. Digit fraction, symbol density, longest identical-character run. Ships day-one protection with zero training, and every alarm carries a human-readable reason like numeric dump digit_frac=0.57.

Character n-gram surprise. An online n-gram model trained on the stream's own clean prefix, scoring each incoming character's surprise before observing it. No pretraining, no access to the generator's logits. The stream explains itself. The signature is nice: repetition loops collapse surprise toward zero, while drift and regurgitation spike it.

Count-Min Sketch repetition. Character shingles counted in a 4×2048 sketch. Constant memory regardless of stream length. Reads the repeat rate, because a real loop hammers many shingles, not one.

Rolling SimHash. A 64-bit fingerprint over a sliding word window, compared against the fingerprint frozen at the end of the clean prefix. This is the one that catches the hardest class: fluent, same-script regurgitation that is invisible to character statistics but lands ~0.5 normalized Hamming away in fingerprint space.

Character entropy. Table dumps and degenerate loops flatten the distribution.

All five read from one state. Throughput is ~50,000 characters per second on a single CPU core, two to three orders of magnitude faster than the model produces text. No GPU anywhere.

The part I got wrong

Detection was never the bottleneck. Every configuration I tried caught every corrupt stream. TPR 1.000, immediately, without effort.

The entire difficulty was in not flagging clean text.

Our clean text is economic prose dense with figures, percentages, variable codes and markdown tables. It is, statistically, the most corruption-looking legitimate text you could ask for.

I started with noisy-OR fusion. It is the textbook rule for combining independent evidence:

p = 1 - Π(1 - sᵢ)

It produced a 42.9% false positive rate on clean production text.

Think about what that means operationally. Four out of every ten healthy desk reports killed and regenerated. Completely unusable.

Here is why it happens. Legitimate prose mildly excites several detectors at once. A digit-dense sentence nudges the rules tier. A formulaic passage nudges compressibility. A section header nudges the sketch. None of them is alarmed. But noisy-OR multiplies survival probabilities, so five detectors at 0.3 each produce a confident 0.83.

Real corruption does not look like five weak signals. It looks like one or two saturated signals. A cross-lingual drift does not gently raise five statistics; it slams the foreign-script fraction to the ceiling while the others barely move.

So we replaced the combination rule with strongest-signal dominance plus a small corroboration bonus when two or more detectors independently cross 0.5:

p = min(1, max(max(sᵢ), w·ℓ) + b·[#{i : sᵢ ≥ 0.5} ≥ 2])

Same five detectors. Same data. Same protocol.

FPR dropped from 0.429 to 0.024.

Then per-detector fixes took it to 0.000:

SimHash was firing on legitimate subtopic wandering in long reports. Fix: cap its contribution at 0.60, below the abort threshold. It can accuse, but never convict alone.
The surprise model's low-surprise fraction was firing on fluent formulaic prose. Fix: gate it behind the repetition tracker.
The sketch's max-count was firing on a recurring section header. Fix: score on rate, not max count.

The lesson generalizes past our system: if you are building a multi-signal guard over stylistically varied text, your combination rule is probably a bigger deal than your detector roster.

Two things that only production teaches you

Thresholds obvious in the abstract were wrong. I set the hard digit-fraction rule at 0.18, which felt generous. Real quantitative answers sustain digit fractions near 0.3. Moved it to 0.28. Separately, a markdown horizontal rule is twenty-plus identical characters, which tripped the run detector on every document with a --- in it. Separator characters are now excluded.

Both fixes came from replaying the guard over the real corpus. I would argue that step is non-optional for any detector you intend to deploy.

Aborting is an integration hazard. We implement abort as an exception raised from inside the token sink. The first version tripped the HTTP circuit breaker, because the transport layer read an abort as an upstream failure. Three catches would have taken the entire assistant offline. The transport has to treat it as caller-side cancellation.

Zero leak

The protocol has three phases. During hold, the first 350 characters are buffered and inspected, and nothing is forwarded. A corrupt verdict here kills the request with zero characters leaked. This matters because corrupt-from-the-start is the dominant production mode. On a clean verdict, the buffer flushes, the self-calibrated baselines freeze, and the stream goes live with rechecks every 400 characters and two-hit hysteresis before aborting.

The cost is a one-time ~350-character delay before the first visible token, roughly a second at typical decode speeds. That is the trade: one second of latency against never showing a user a corrupted answer.

Calibrating the threshold instead of guessing it

Setting the abort threshold by hand felt wrong, so we use split-conformal calibration. Take the final fused scores of n held-out clean streams, and set τ to the ⌈(1−α)(n+1)⌉-th smallest. That guarantees Pr[flag | clean] ≤ α in finite samples under exchangeability.

The practical payoff: the operator's false-alarm budget α becomes the only meaningful knob in the system. You say "I will tolerate 2% false aborts" and the threshold follows.

An unplanned finding

We ran the finished guard backwards over 121 historical messages in the deployment's database, expecting a clean sweep as a sanity check.

It raised exactly two flags. Both were real corruptions that had shipped to users months earlier and nobody noticed: one desk report written entirely in Chinese, and one English answer that derailed into Chinese plus Python code and stopped mid-fragment.

Zero false alarms on the other 119.

So it is also an audit tool, which was not the plan.

Honest limitations

Corruption that stays fluent, same-script, same-topic and statistically unremarkable is invisible to this. Subtly wrong but well-formed analysis needs grounding or factuality methods, and we handle that separately upstream.

Our positives are synthetic, generated by injecting the four corruption classes into clean text at controlled onsets. They are modeled on real derails and they catch every real historical case we have, but production traffic at scale may hold surface forms we have not modeled.

And the perfect scores are a statement about class separation, not difficulty. The observed corruption classes sit far from the clean manifold. I fully expect adversarially subtle corruption, like a slow drift below checkpoint resolution, to erode those margins.

Try it

The detector, the benchmark generator and the trained weights are open source: https://github.com/doofzoff/SIMURG

The detector registry takes a three-line protocol — a name, an evaluate(state) that returns a score, and a set of reasons. You can add a detector without touching the sentinel, and the fusion layer picks up whatever is registered. If you are running a non-Latin-script deployment, the expected-script priors need flipping and I would genuinely like help with that.

If you self-host a model and have watched it derail: what did yours do? I am collecting failure modes, and the taxonomy has four classes right now but I do not believe that is the complete list.

DEV Community: FX-LgLL

THX-01 Just Processed 70 Million Tokens. Our Open-Source Decision Model Is Just Getting Started.

Why THX-01?

THX-01 vs TypeSafe Jev

Built for Efficient Decision-Making

The THX-01 API Is Free

Everything Is Open-Source

What's Next?

We Open-Sourced a 322M Decision Model. Now Its API Is Free

What Is THX-01?

Try the Free API

How Does It Compare to Jev?

Small Enough to Run on Your Phone

Everything Is Open-Source

Get Started

We made our decision model's API free (and the weights are open)

Try it now (no key, no sign-up)

What it can answer

How it compares

Run it yourself

We Open-Sourced THX-01: A 322M AI Decision Model Small Enough to Run on a Smartphone

Key Highlights

Why We Built It

Get Started

Open Source

We open-sourced THX-01: a 322M decision model that matches Sonnet 5.5 on ticket classification at ~10 ms (Apache 2.0, pip install thx01)

SIMURG: A Guard That Catches LLM Hallucinations Mid-Stream and Heals Them

The problem with detecting corruption after the fact

Self-Heal: diagnose, trim, continue, stitch

SIMURG Pulse: the deep tier

Train it on YOUR hallucinations

Also in this release

Links

Free Unlimited Web Search for AI Agents

What is SIMURG?

SIMURG Search

Why use it?

Get Started

I found a completely free web search for AI agents.

Free means free

The part I care about: ground()

I almost killed all GLM 5.3 Flash 2 bit hallucinations, with SIMURG

The setup

What actually happened

What SIMURG is, in short

GLM-5.3-Flash is a total blast when you kill its hallucinations

I just killed hallucinations on my 2 bit Qwen3.8 27B

Five detectors, one bad merge: why our LLM corruption guard flagged 43% of healthy output

Why post-hoc detection is useless here

The design

The part I got wrong

Two things that only production teaches you

Zero leak

Calibrating the threshold instead of guessing it

An unplanned finding

Honest limitations

Try it