The latest articles on DEV Community by Jake (@janderswag). https://dev.to/janderswag https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4027935%2Fed59eb44-6a90-4820-aa6e-ee7ca665e4a2.jpg https://dev.to/janderswag en Jake Tue, 14 Jul 2026 01:47:12 +0000 https://dev.to/janderswag/local-first-rag-for-privileged-legal-documents-why-citations-need-verification-4936 https://dev.to/janderswag/local-first-rag-for-privileged-legal-documents-why-citations-need-verification-4936 <p>Lawyers want the same thing everyone else wants from AI: ask a question, get a useful answer, move faster.</p> <p>The difference is that legal documents often come with privilege, confidentiality, protective orders, client expectations, and professional responsibility duties. Uploading a contract set, deposition transcript, pleading file, or discovery bundle into a generic cloud chatbot is not a neutral implementation detail. A vendor sits in the data path.</p> <p>That is the design problem behind <a href="https://github.com/janderswag/docuchat.app" rel="noopener noreferrer">DocuChat</a>: an open-source, local-first document chat app for attorneys.</p> <p>The core idea is simple:</p> <blockquote> <p>Put the AI next to the documents, not the documents next to the AI vendor.</p> </blockquote> <h2> The architecture </h2> <p>DocuChat is a self-hosted legal document RAG stack:</p> <ul> <li> <strong>Local models</strong> through Ollama</li> <li> <strong>FastAPI</strong> backend bound to loopback</li> <li> <strong>LanceDB</strong> embedded vector store</li> <li> <strong>PyMuPDF / Docling / Tesseract</strong> for PDFs, tables, and OCR</li> <li> <strong>Matter-scoped retrieval</strong> so one client/matter cannot bleed into another</li> <li> <strong>Verified page/span citations</strong> before answers are displayed</li> </ul> <p>The query path does not require OpenAI, Anthropic, Google, or any cloud AI API key. After local models are downloaded, the workflow can run offline.</p> <h2> Why ordinary citations are not enough </h2> <p>A citation-looking string is easy for an LLM to generate. That does not mean the cited text actually supports the claim.</p> <p>DocuChat treats citations as a mechanical verification problem. The model can propose an answer and source quote, but the app checks whether that quote actually appears in the retrieved source text at the claimed page/span. If a claim cannot be verified against the source, it is dropped instead of shown.</p> <p>That creates a more useful failure mode for legal work:</p> <ul> <li>Better: “I could not find that in the documents.”</li> <li>Worse: a confident answer with a fake or sloppy citation.</li> </ul> <h2> What this is not </h2> <p>DocuChat is not an AI lawyer. It does not give legal advice, draw legal conclusions, or act outside the user’s machine.</p> <p>It is a cited-retrieval assistant for document-heavy work: contracts, transcripts, scanned PDFs, exhibits, pleadings, and matter files.</p> <h2> Why open source matters here </h2> <p>For sensitive workflows, “trust us” is not enough. Legal technologists should be able to inspect the data path, citation logic, storage layer, and model boundaries.</p> <p>The repo is here:</p> <p><a href="https://github.com/janderswag/docuchat.app" rel="noopener noreferrer">https://github.com/janderswag/docuchat.app</a></p> <p>If you care about local-first AI for private legal documents, a GitHub star/watch helps validate that this is worth building further.</p> legaltech ai opensource privacy