Most knowledge bases quietly turn guesses into facts. This one is designed not to.
I’ve been building a structured domain knowledge system — in my case for procurement and sourcing — and the hardest engineering problem wasn’t storage or retrieval. It was provenance: keeping what is proven separate from what is merely plausible.
The source-of-truth rule
Treat all existing material as one connected knowledge base. Then:
- KEEP what is correct
- EXPAND what is incomplete
- ADD what is genuinely missing
- REPLACE what is wrong or weak
- REMOVE unnecessary duplication
Never destroy good existing work just to produce a new version.
Evidence discipline
Every item carries a source class, and they never blur:
- Documented experience — things actually done, with something to point to
- General domain knowledge — true, but not personal proof
- Inference — reasonable, but unproven
- New research — added, and explicitly marked as added
Flag the gap, don’t fill it
When the material doesn’t support a claim, the system records a gap rather than fabricating a bridge. A gap you can see is a task. A gap you can’t see is a landmine.
Why builders should care
The moment you point an LLM at your own notes, provenance stops being academic. A retrieval system that can’t tell you why it believes something will confidently hand you inference dressed as fact. Tag the source at write time, not query time — and make “unsupported” a first-class value.
Top comments (1)
Provenance at write time is the important design choice here. If source class gets added only when someone asks a question later, the system has already mixed evidence with inference. Making unsupported a stored value gives the next user a real task instead of a confident guess.