DEV Community

Cover image for Footnote: Agentic RAG with Sentence-Level Citations
Ibrar Yunus
Ibrar Yunus

Posted on Originally published at ibraryunus.com

Footnote: Agentic RAG with Sentence-Level Citations

An AI research agent for your documents. It decides what to search for, searches again when the results are thin, and every claim in its answer links to the exact sentence it came from.

The usual 'chat with your documents' build searches once with the user's words and hopes. A two-part question gets one search and half an answer. footnote replaces that fixed pipeline with an agent that works like a researcher, built around one rule: an answer you cannot check is not an answer.

How it was built

Claude is given two tools, search_handbook and read_page, and loops until it can answer. Search results come back as sentence-level blocks, so every citation is validated by the API rather than typed by the model. Retrieval is a hand-written BM25 plus local bge-small vectors, merged with weighted reciprocal rank fusion and tuned on a 59-question eval set. The demo corpus is the public-domain 18F staff handbook, 232 pages.

What it does

Plans its own research. A two-part question gets two searches, worded the way the handbook words things. Each step shows live in a side panel.

Sentence-level citations. Click a citation chip and the source passage scrolls into view with the exact sentence highlighted.

A citation chip opens the source passage with the cited sentence highlighted

Says when it doesn't know. If nothing relevant turns up after retrying, it says the documents don't cover it instead of guessing.

Asked about something the handbook doesn't cover, it says so and cites the closest passages

Swappable search. Run the agent on hybrid, keyword-only or vector-only search, with each passage's rank in both shown.

Stack

Python with FastAPI and a one-file HTML UI, Claude through the Anthropic SDK with no framework, and a CLI that prints each tool call. The screenshots come from a keyless demo mode where the model's turns are scripted and the searches and citations run for real.

Retrieval is the easy half. The hard half is making every sentence of the answer checkable.


Source code: github.com/IbrarYunus/footnote. More of my work: ibraryunus.com.

Top comments (0)