Ask GPT-5 or Claude Opus 4.x "Who won the 1998 Nobel Prize in Literature?" ten times and you get ten confident answers. When the model actually knows, all ten say "José Saramago." When it doesn't, you get Saramago, Kenzaburō Ōe, "the prize wasn't awarded that year," and three other names — all delivered with identical confidence. The model's token probabilities look fine in every case. What changes is the spread of meanings across samples.
Semantic entropy detects LLM hallucinations by measuring that spread — the entropy over distinct meanings, not over token sequences. It needs no ground truth, no retrieval, and no fine-tuning. This is the technique from Farquhar et al.'s 2024 Nature paper, and it's one of the few uncertainty methods that survives contact with production.
TL;DR
- Semantic entropy samples an answer several times, clusters the samples by meaning (bidirectional entailment), and computes entropy over the clusters. High entropy → the model is guessing → likely a confabulation.
- It fixes the core flaw of naive sequence entropy: "Paris" and "It's Paris" are two token sequences but one meaning. Token-level entropy counts them as uncertainty; semantic entropy doesn't.
- You need N samples at temperature ≈ 1 (N=5–10 works) plus a cheap entailment step. Cost is roughly N× generation, so reserve it for high-stakes answers.
- It catches confabulations (arbitrary wrong answers that vary run to run). It does not catch confident, consistent errors — if the model is reliably wrong, entropy is low.
- Works best on short factual answers where "same meaning" is well defined; degrades on long free-form generation.
Why does token probability fail at detecting hallucinations?
Because a low-probability answer and an uncertain answer look identical at the token level, and because many different token strings encode the same fact. A model can be 95% confident on every token of a fabricated citation. Per-token log-probs measure fluency, not truth.
The naive fix — compute the entropy of the full sequence distribution by sampling — breaks for a subtler reason. Consider the question "What is the capital of France?" and five samples:
"Paris" "It is Paris." "The capital is Paris."
"Paris, France" "Paris."
Five distinct token sequences. Length-normalized sequence entropy is high — the model spread probability mass across five outputs. But every output means the same thing. The model is certain; the entropy estimator is fooled by surface variation. You'd flag a correct, confident answer as uncertain.
The problem is that entropy over sequences conflates two things: lexical variation (how you phrase it) and semantic variation (what you actually claim). For hallucination detection you only care about the second.
What is semantic entropy, precisely?
Semantic entropy is the Shannon entropy of the distribution over meaning clusters, obtained by marginalizing the sequence distribution over sets of semantically equivalent outputs.
Formally: the model induces a distribution p(s | x) over output sequences s for input x. Define an equivalence relation ≈ where s₁ ≈ s₂ iff they mean the same thing. This partitions sequences into meaning classes C. The probability of a class is the sum of its members:
p(C | x) = Σ_{s ∈ C} p(s | x)
Semantic entropy is entropy over that clustered distribution:
SE(x) = − Σ_C p(C | x) · log p(C | x)
Because you can't enumerate all sequences, you estimate it from N samples. The clean, robust estimator is the discrete (Rao) version: cluster your N samples, then use cluster frequencies as probabilities.
p(C) ≈ |C| / N
SE ≈ − Σ_C (|C|/N) · log(|C|/N)
If all N samples land in one cluster, SE = 0 (fully certain about meaning). If every sample is its own cluster, SE = log N (maximally uncertain). The whole game is defining ≈ well.
How do you cluster answers by meaning?
Use bidirectional entailment: two answers are semantically equivalent if answer A entails B and B entails A. One-directional entailment isn't enough — "It's a dog" entails "It's an animal" but not vice versa; they aren't equivalent.
You can run the entailment check with a small NLI model (DeBERTa-large-MNLI is the standard choice) or with an LLM judge. Here's the full algorithm with an LLM doing the entailment step:
import math
from collections import Counter
def sample_answers(client, question, n=8, model="claude-opus-4-x"):
outs = []
for _ in range(n):
r = client.messages.create(
model=model,
max_tokens=64,
temperature=1.0, # you NEED spread; temp 0 defeats the method
messages=[{"role": "user", "content": question}],
)
outs.append(r.content[0].text.strip())
return outs
def bidirectional_entails(client, ctx, a, b, model="claude-haiku-4-5"):
# cheap judge model for the O(clusters) entailment checks
prompt = (
f"Question: {ctx}\n"
f"Statement 1: {a}\nStatement 2: {b}\n"
"Do these two statements mean the same thing as an answer to the "
"question? Answer only 'yes' or 'no'."
)
r = client.messages.create(
model=model, max_tokens=3, temperature=0,
messages=[{"role": "user", "content": prompt}],
)
return r.content[0].text.strip().lower().startswith("y")
def cluster(client, question, answers):
clusters = [] # each cluster is a list of answers
for a in answers:
placed = False
for c in clusters:
# compare against the cluster's representative (first member)
rep = c[0]
if (bidirectional_entails(client, question, a, rep)
and bidirectional_entails(client, question, rep, a)):
c.append(a)
placed = True
break
if not placed:
clusters.append([a])
return clusters
def semantic_entropy(client, question, n=8):
answers = sample_answers(client, question, n=n)
clusters = cluster(client, question, answers)
total = len(answers)
se = 0.0
for c in clusters:
p = len(c) / total
se -= p * math.log(p)
return se, clusters
# Normalize by log(n) to get a 0..1 score you can threshold
se, clusters = semantic_entropy(client, "Who won the 1998 Nobel Prize in Literature?")
normalized = se / math.log(8)
print(f"SE={se:.3f} normalized={normalized:.3f} clusters={len(clusters)}")
A cleaner answer set collapses into one cluster → normalized ≈ 0. A confabulated one fragments into many → normalized approaches 1. Threshold empirically on a labeled validation set; something like normalized > 0.5 is a reasonable starting flag, but tune it.
What does semantic entropy cost, and when is it worth it?
The dominant cost is generation: N forward passes instead of one. At N=8 you pay roughly 8× the tokens of a single answer, plus O(number of clusters) cheap entailment calls per question (usually 1–4 clusters for factual questions, so the judge cost is small if you route it to a cheap model like Haiku).
That makes it a poor fit for a global "check every response" filter. It's a strong fit for:
- Selective answering — refuse or escalate to a human when SE is high, on medical/legal/financial answers.
- RAG grounding checks — high SE on a retrieval-augmented answer means the context didn't actually pin the answer down.
- Offline eval — score a model's calibration on a benchmark without human labels.
You can cut cost with a smaller N. Even N=3–5 recovers most of the signal for short answers, because confabulation usually produces obvious fragmentation fast. Reuse the same samples for self-consistency voting so the compute isn't wasted.
When does semantic entropy fail?
It has one structural blind spot: consistent errors. Semantic entropy measures whether the model agrees with itself, not whether it's right. If a model has internalized a wrong fact — it reliably says the wrong CEO, the wrong date, the wrong API signature — all N samples cluster together, SE ≈ 0, and the method reports high confidence in a false answer. Semantic entropy detects the confabulation failure mode (arbitrary, unstable fabrication), not the systematic bias failure mode.
Other failure modes to watch:
- Long-form generation. "Same meaning" is ill-defined for a five-paragraph answer. You have to decompose into atomic claims and score entropy per claim, which multiplies cost and adds a decomposition step that can itself err.
- Entailment model errors. Your clustering is only as good as the NLI/judge step. A judge that's too lenient merges distinct answers (underestimates SE); too strict, and paraphrases fragment (overestimates SE). Bidirectional checking helps, but validate the judge.
- Temperature coupling. Sample too cold and even uncertain models look confident; too hot and confident models look uncertain. Temperature ≈ 1 is the calibrated default from the literature — don't reuse your production temp of 0.
- Numeric/format answers. "3.14" vs "3.14159" vs "about pi" — decide up front whether those are the same meaning, or your clusters are arbitrary.
Semantic entropy vs. self-consistency: what's the difference?
Self-consistency takes a majority vote over samples and returns the winner. Semantic entropy takes the full shape of the distribution and returns an uncertainty score. They share the sampling step, so compute both from one batch: use the largest cluster as your self-consistency answer, and the entropy as your confidence. Self-consistency tells you what to answer; semantic entropy tells you whether to trust it.
Direct answer: how do you detect LLM hallucinations with semantic entropy?
Sample the model's answer 5–10 times at temperature ≈ 1, cluster the samples by bidirectional entailment so paraphrases of the same fact collapse into one group, then compute Shannon entropy over the cluster frequencies. Low entropy means the model keeps returning the same meaning and is probably grounded; high entropy means it's returning different meanings run to run — the signature of a confabulation. The technique needs no ground-truth labels because it measures the model's semantic self-consistency rather than correctness, which is exactly why it catches unstable fabrications but misses confident, systematically wrong answers. Use it as a selective-answering gate on high-stakes responses, not as a blanket filter, and always validate your entailment/judge step and threshold on labeled data before trusting the number.
Top comments (0)