Your LLM gave you an answer. Should your application trust it?

Your LLM gave you an answer. Should your application trust it?

I built BOOTH, a lightweight checkpoint layer for LLM outputs.

The idea is simple: don't automatically pass every model response downstream. Check it first.

For example:

Evidence: Returns are allowed within 45 days.

LLM: Returns are allowed within 90 days.

The answer sounds confident. It's also unsupported by the evidence.

BOOTH's check_with_evidence() lets you check an LLM response against evidence your RAG pipeline has already retrieved.

result = booth.check_with_evidence(
    answer=llm_answer,
    evidence=retrieved_docs,
    compare_fn=your_comparison_function,
)

No need to replace your existing RAG pipeline or commit to a particular LLM provider.

Zero runtime dependencies. Provider-agnostic. Small API.

pip install boothpy

GitHub: https://github.com/Vedantgitbot/booth

How are you currently deciding whether an LLM output is safe to pass downstream?

---

Beta · 2K+ PyPI downloads · 300+ tests · CI passing · MIT · Python 3.9+