Stop AI Hallucinations: Verify Your LLM Summaries Before Publishing

You’ve probably seen headlines about AI summarizers that miss key facts or invent details. That’s the AI‑DR problem—models that claim to read but don’t. It can cost you credibility and time.

What You’ll Learn

• Add a verification step after summarization.
• Compare different verification strategies.
• Spot common failure modes and how to mitigate them.

Why Verification Matters

When an LLM produces a summary, it can hallucinate facts that aren’t in the source. Those hallucinations can mislead users or propagate misinformation. Adding a verification step lets you catch those errors before they reach the audience.

Choose a Verification Strategy

You have three main options: a simple embedding similarity check, a retrieval‑based fact check, or a human‑in‑the‑loop review. Each has its own cost, latency, and accuracy profile. The choice depends on how critical the content is and how much automation you can afford.

Embedding Similarity

Embedding similarity is the lightest option. It turns both the source and the summary into vectors and compares them with cosine similarity. If the similarity falls below a threshold, you flag the summary.

import numpy as np
import openai

## Convert text to a vector using a small embedding model

def embed(text, model="text-embedding-3-small"):
    resp = openai.Embedding.create(input=text, model=model)
    return np.array(resp.data[0].embedding)

## Compare two vectors and decide if the summary is close enough

def verify(summary, source, threshold=0.75):
    src_vec = embed(source)
    sum_vec = embed(summary)
    similarity = np.dot(src_vec, sum_vec) / (np.linalg.norm(src_vec) * np.linalg.norm(sum_vec))
    return similarity >= threshold

The code is short and uses only the OpenAI API. It works with any LLM that can produce a summary.

Retrieval‑Based Fact Check

If you need higher precision, you can retrieve the exact sentences that support each claim. The LLM is asked to list the source sentences it used, and you compare that list to the original text.

import openai

## Ask the model to list supporting sentences

def get_supporting_sentences(summary, source, model="gpt-4o-mini"):
    prompt = (
        "Given the following summary, list the exact sentences from the source that support each bullet point."
        f"\n\nSummary:\n{summary}\n\nSource:\n{source}"
    )
    response = openai.ChatCompletion.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    return response.choices[0].message.content.strip()

You then parse the returned sentences and check that each appears verbatim in the source. This method is more expensive but catches subtle hallucinations.

Build the Pipeline

Below is a minimal, end‑to‑end pipeline that stitches the steps together.

Step 1: Fetch and Clean the Source

import requests
from bs4 import BeautifulSoup

## Grab the main article body from a URL

def fetch_text(url):
    resp = requests.get(url)
    resp.raise_for_status()
    soup = BeautifulSoup(resp.text, "html.parser")
    # Very naive extraction: grab all paragraph tags
    paragraphs = soup.find_all("p")
    return "\n".join(p.get_text() for p in paragraphs)

In production you might use a dedicated article extractor, but this keeps the example focused.

Step 2: Summarize with an LLM

import openai

## Ask the model to produce a concise, bullet‑point summary

def summarize(text, model="gpt-4o-mini"):
    prompt = (
        "Summarize the following text in 3–5 bullet points. Only include facts that appear in the source."
        f"\n\n{text}"
    )
    response = openai.ChatCompletion.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    return response.choices[0].message.content.strip()

The prompt explicitly asks for factuality, which helps reduce hallucinations.

Step 3: Verify the Summary

You can plug either verification method here. For brevity we’ll use the embedding similarity check.

## Reuse the verify() function from the Embedding Similarity section

def process(url):
    source = fetch_text(url)
    summary = summarize(source)
    if verify(summary, source):
        print("✅ Summary approved")
    else:
        print("❌ Summary flagged for review")

Step 4: Decision Logic

If verify() returns True, you can publish the summary. If it returns False, you have three options:

• Re‑run the summarization with a stricter prompt.
• Send the summary to a human reviewer.
• Log the failure for future analysis.

Tradeoffs Between Approaches

Approach	Cost	Latency	Hallucination Risk	Human Effort
Embedding Similarity	Low	Fast	Medium	None
Retrieval‑Based Fact Check	Medium	Medium	Low	None
Human Review	High	Slow	Very Low	High

The table shows that the embedding check is the cheapest and fastest, but it may miss subtle errors. Retrieval‑based checks are more accurate but cost more. Human review is the most reliable but also the slowest.

Common Failure Modes

• Embedding drift: The embedding model may not capture subtle differences, leading to false positives.
• Threshold mis‑tuning: A too‑high threshold rejects good summaries; a too‑low threshold lets hallucinations slip.
• Prompt leakage: If the verification prompt is too similar to the summarization prompt, the model may repeat hallucinations.
• Source noise: Web pages with ads or commentary can confuse the summarizer.

Tuning the Verification

• Start with a threshold of 0.75 and adjust based on observed false‑positive/false‑negative rates.
• Use a diverse set of test documents to calibrate the threshold.
• Add a small “source‑check” prompt that asks the model to list the exact sentences it used.
• Cache embeddings for repeated documents to reduce cost.

Key Takeaways

• Adding a verification step dramatically reduces hallucinations.
• Embedding‑based similarity is a lightweight, model‑agnostic check.
• Retrieval‑based fact checks offer higher precision at a higher cost.
• Tune the similarity threshold to balance cost and accuracy.
• Keep an eye on failure modes and iterate on prompts and thresholds.

Source

AI;DR (AI; Didn't Read) – I added code, a verification strategy, tradeoff analysis, and failure‑mode discussion.