Building retrieval-augmented generation for a law firm teaches you something general chat products tend to hide: the most important output is sometimes "I can't answer that."
The pattern itself is familiar. Ingest the firm's corpus (briefs, memos, contracts, matter files, playbooks), usually from SharePoint, iManage or a document store. Group it into matter workspaces. At query time, retrieve passages by meaning, then generate an answer only from what was retrieved, with a numbered citation per claim that links back to the source paragraph.
Three design choices matter more than model choice:
- Semantic retrieval. Lawyers ask questions the way a partner would. "Who carries the risk if the platform leaks customer data" should surface indemnity and confidentiality clauses even if neither contains the word "leak." On one SEC filing search project, plain-English queries ran 3.4x faster than the 256-character Boolean strings analysts had been writing.
- Hard refusal. If retrieval does not support an answer, the system says so and stops. Filling gaps with something plausible is exactly the behaviour you are designing out.
- Human gate in the pipeline. Drafts come back as tracked changes with Accept or Reject per change. Nothing is filed or sent until an attorney signs off, and that event is logged.
Corpus condition also beats corpus size. Clean case law ingests quickly. Twenty years of scanned matter files need cleanup first, and that work shows up in the estimate.
Takeaway for engineers: write your evaluation set before you tune anything. Use research the firm's team already did by hand, measure precision and recall against it, and add missed-case audits on live work. Those numbers, not demo impressions, should decide whether the system goes to production.
For scope, integrations and how a build runs, see our custom legal AI for law firms service page.
Full step-by-step architecture: Custom Legal AI for Law Firms: How It Works
Top comments (0)