It accumulates evidence.
I used to believe something that now feels obvious.
If we documented enough, understanding would take care of itself.
Write the ADR.
Keep the runbook up to date.
Record the incident.
Update the architecture diagram.
Future engineers would have everything they needed.
I don't think that's true anymore.
The day I read everything
I was trying to understand a service I hadn't worked on before.
The documentation looked excellent.
There was an ADR explaining the original design.
A migration guide from the last major release.
An incident review.
An API reference.
A runbook.
An architecture diagram.
Everything was current.
Everything was internally consistent.
I spent almost two hours reading.
When I closed the last page, I realized something uncomfortable.
I knew what had happened.
I still didn't understand the system.
Every document was telling the truth
The ADR explained why the original architecture was chosen.
The migration guide explained why it changed.
The incident review explained why it failed.
The runbook explained how to recover it.
None of those documents contradicted each other.
They simply weren't trying to answer the same question.
Each one captured a moment.
I was looking for a story.
That's when I noticed the difference
Documentation preserves observations.
Understanding comes from connecting observations.
The documents never said:
"This incident changed the team's priorities."
"That migration only happened because of the outage three months earlier."
"This design decision stopped making sense after customer X arrived."
Nobody wrote those sentences because nobody needed to.
At the time, everyone already knew them.
Months later, those invisible connections turned out to be the only thing I actually needed.
Evidence isn't explanation
A detective doesn't solve a case because there are enough fingerprints.
The fingerprints matter.
But only after someone connects them.
Documentation works the same way.
Each page is evidence that something happened.
Knowledge appears only when someone reconstructs the relationships between those pieces of evidence.
The documents weren't incomplete.
My understanding was.
What changed for me
I stopped asking:
Do we have documentation?
Now I ask:
Could someone who wasn't here reconstruct the story?
Those are different questions.
The first measures documentation.
The second measures understanding.
I've become much less convinced they're the same thing.
Documentation preserves evidence.
Knowledge is reconstructed.
The more systems I study, the more I think confusing those two ideas is the reason so many organizations feel well documented—and still spend hours asking each other "Why does this work like this?"
Next: why I stopped treating duplicate documents as clutter—and started treating them as evidence that an organization disagrees with itself.
Top comments (4)
This matches the failure mode I see with runbooks. They preserve the decision, but not the pressure around it. The useful artifact is often a small trail of rejected options and constraints, because that is what lets the next person tell whether the old answer is stale or just unfashionable.
I like the way you put it: they preserve the decision, but not the pressure around it.
The constraints are often the most time-sensitive part of the story. The implementation may survive for years, but the assumptions behind it can change in months.
That's why I increasingly think the most valuable question isn't "What did we decide?" but "Under what conditions did this make sense?" Without those conditions, it's hard to tell whether a decision is still valid or whether we're just following history.
The documentation-as-evidence vs knowledge-as-reconstruction gap maps to exactly why RAG over documentation tends to fail for onboarding tasks: it retrieves the ADR, the incident review, and the runbook as separate chunks, but the relationship between them ("the incident changed priorities, which is why the next ADR reversed the earlier decision") exists between documents, not inside any of them. Retrieval precision can be high while answer fidelity is low — every chunk is accurate, but the reasoning chain the user actually needs was only ever implicit in the collective history. One partial fix is explicit relation extraction at index time: rather than chunking documents independently, build a graph where the incident review links to the ADR it caused the reversal of, and retrieval follows those typed edges rather than treating each document as an isolated embedding. The diagnostic question you've named — not "is everything documented" but "can someone reconstruct why" — is also the right evaluation criterion for a RAG corpus: can a retriever return not just the facts but the causal chain that connects them?
I think you've captured the distinction better than I did.
The individual documents aren't wrong. The missing piece is the relationship between them. An ADR explains a decision. An incident review explains a failure. A migration guide explains a change. Understanding comes from knowing that one caused the next.
That's also why I like your evaluation question. Instead of asking "Did retrieval find the relevant documents?", ask "Could someone reconstruct the reasoning from what was retrieved?" Those are very different tests.
I increasingly think documentation succeeds at preserving evidence. The harder problem—and the one most systems still leave to the reader—is preserving the story that connects that evidence.