How do you know that an LLM-generated response is factually correct? 🤔

Hallucinations are an interesting artifact of LLMs where the model tends to make up facts or generate outputs that are not factually correct.

There are two broad approaches for detecting hallucinations:

• Verify the correctness of the response against world knowledge (via Google/Bing search)
• Verify the groundedness of the response against the information present in the retrieved context

The 2nd scenario is more interesting and useful as the majority of LLM applications have an RAG component, and we ideally want the LLM to only utilize the retrieved knowledge to generate the response.

While researching state-of-the-art techniques on how to verify that the response is grounded wrt context, two of the papers stood out to us:

• FactScore: Developed by researchers at UW, UMass Amherst, Allen AI and Meta, it first breaks down the response into a series of independent facts and independently verifies if each of them.
• Automatic Evaluation of Attribution by LLMs: Developed by researchers at Ohio State University, it prompts the LLM judge to determine whether the response is attributable (can be verified), extrapolatory (unclear) or contradictory (can’t be verified).

While both the papers are awesome reads, you can observe that they tackle complementary problems and hence, can be combined for superior performance:

• The responses in production systems typically consist of multiple assertions, and hence, breaking them into facts, evaluating them individually and taking average is a more practical approach.
• Many responses in production systems fall in the grey area, i.e. the context may not explicitly support (or disprove) them but one can make a reasonable argument to infer them from the context. Hence, having three options - Yes, No, Unclear is a more practical approach.

This is exactly what we do at UpTrain to evaluate factual accuracy. You can learn more about it in our docs