Ask a long legal document a question in plain language and keyword search falls apart. The answer is in there, but it's phrased three different ways in three different articles, and Ctrl+F only finds the wording you already know.
So I built Ask the Constitution: a full-stack RAG (retrieval-augmented generation) app that answers questions about a constitution in plain English — and shows you the exact source text behind every answer.
Try it live: https://ask-constitution-web.vercel.app
Code: https://github.com/rayyan4t4/ask-the-constitution-fullstack
Why RAG and not fine-tuning
The document is fixed and factual. Fine-tuning would bake the text into model weights — expensive, hard to update, and the model would still answer from memory with no way to verify it. RAG keeps the document as data: retrieve the relevant passages at query time, hand them to the LLM as context, and the answer can point back at its sources. For anything a user needs to trust, that citation trail is the whole game.
The architecture
- Ingestion: the document text is cleaned and split into 768 chunks
- Embeddings: ONNX build of MiniLM — small, fast, runs locally, so embedding costs nothing per query
- Vector store: FAISS index over the chunk embeddings
- Backend: FastAPI on Render — retrieval + generation endpoint
- Generation: an LLM via OpenRouter, prompted to answer only from the retrieved chunks
- Frontend: React on Vercel, with expandable source cards under each answer so you can read the passage yourself
A query flows like this: embed the question → FAISS returns the closest chunks → those chunks become the LLM's context → the answer comes back with its sources attached.
Three decisions that mattered
1. Sources are a UI feature, not a debug log.
Every answer renders its source chunks as expandable cards. If the model gets something wrong, the user can see why — bad retrieval or bad generation — instead of just distrusting the whole app. Verifiability is what separates a RAG demo from a tool.
2. Keep the embedding model boring and local.
MiniLM via ONNX is not glamorous, but it embeds the whole document once at build time and each query in milliseconds, with zero API cost and no rate limits. Spend your complexity budget on chunking and retrieval quality, not on the embedder.
3. Design for the free tier failing.
Free LLM routes throw intermittent 429s and cold starts cause 502s. The frontend retries once automatically, which turns most hard failures into a two-second pause. If you build on free tiers, assume flakiness and absorb it in the client.
What I learned
- Retrieval quality is the product. A better model can't fix chunks that don't contain the answer. Chunk boundaries and overlap mattered more than any generation setting.
- "Answer only from the context" is a prompt you have to enforce, then verify. Testing with questions the document can't answer is as important as testing the ones it can.
- Citations change user behavior. People read answers differently when the evidence is one click away — they check, and they trust the checked ones more.
What's next
The same pattern — ingest your documents, retrieve with evidence, generate with citations — is what I'm now building for businesses: support and website chatbots trained on a company's own docs instead of generic model knowledge. My other projects are on rayyankhan.online.
I'm Rayyan Khan — AI/ML Engineer and Team Lead at ITSimplera Solutions, final-year BS Computer Science student at UET Mardan. I build production AI systems: RAG chatbots, LLM applications, and computer vision. Say hello in the comments — especially if you've shipped RAG and fought with chunking too.
Top comments (0)