There's a reflex, when you want an LLM to "know" your domain, to fine-tune it — bake the guidelines into the weights and hope it remembers. For a copilot that has to be correct and defensible (say, one helping an underwriter), that's usually the wrong instinct. Retrieval beats memory, and not because it's smarter — because it's auditable, fresh, and doesn't require retraining.
Fine-tuning vs retrieval, for knowledge
Fine-tuning is great for teaching a model a skill, format, or style. It's a poor mechanism for teaching it facts you need to be current and citable:
| Fine-tune knowledge into weights | Retrieve at inference (RAG) | |
|---|---|---|
| Update a guideline | retrain | edit a document |
| "Why did it say that?" | ¯\_(ツ)_/¯ | cite the retrieved source |
| New product next week | retrain | index the doc |
| Wrong/outdated fact | baked in, hard to find | fix the source, done |
For an underwriting copilot, the killer feature isn't fluency — it's that the human can see the source and overrule it. A copilot that can't cite is a confident stranger. One that retrieves the specific guideline, clause, and prior case is a junior analyst who brings the file.
The architecture
question ─► retrieve(top-k from: guidelines, policies, prior cases)
─► rerank + assemble context (with source IDs)
─► LLM answers ONLY from retrieved context
─► response + citations ─► human reviews, can click through
The rule that keeps it honest: the model answers from retrieved context, not from its own memory. If the answer isn't in the retrieved sources, the correct output is "I don't have that," not a confident guess. In insurance that boundary is the difference between a useful tool and a liability.
Why this matters beyond correctness
- Freshness. Guidelines change. A retrieval system is current the moment you update a document; a fine-tuned model is frozen at its last training run.
- Auditability. Every answer traces to a source. When a regulator or a senior underwriter asks "on what basis?", you have one.
- Cost & iteration. Editing a knowledge base is minutes; retraining is a project. You iterate on the data, not the model.
- Least surprise. You control the knowledge by controlling the corpus, not by hoping the weights generalized the way you wanted.
When should you fine-tune?
When you need consistent structure, tone, or a task-specific behavior the base model does poorly — not to inject a knowledge base you'll need to keep current. The two techniques compose: fine-tune the behavior, retrieve the facts.
Full write-up with the insurance-copilot framing (retrieval beats memory, and why it builds trust):
AI Copilots for Underwriters: Why Retrieval Beats Memory →
From IntelliBooks' series on the data foundation under insurance AI.
Where do you draw the fine-tune vs retrieve line in your stack?
Top comments (0)