When we first started integrating Claude into our development workflow, the idea seemed brilliant: let the LLM handle all the documentation. Write a function? Generate docs. Refactor a module? Generate docs. Rename a variable? Generate docs.
We pushed hard. We prompted aggressively. And Claude delivered — more than we ever expected.
The Documentation Avalanche
Within a few months, our repository had accumulated an astonishing volume of documentation files. README files for every directory. Inline comments for every function. Architecture decision records. API references. Onboarding guides. Glossaries. Runbooks. Postmortems for incidents that had long since been forgotten.
At first, it felt productive. It looked like a mature, well-maintained codebase.
But something was wrong.
No human could actually read all of it. And — this is the part that stings — neither could the LLM. When we asked Claude to help with a refactor, the context window filled up just loading the documentation layer. The docs had become noise, crowding out the actual signal: the source code.
The Source Moved On. The Docs Didn't.
The real problem wasn't the volume. It was the drift.
Our codebase kept evolving — new features, architectural pivots, dependency upgrades, renamed services. But the documentation? It stayed frozen at whatever moment Claude had generated it. There was no automated mechanism to detect that a doc was now stale. There was no diff that told us "this README no longer matches the module it describes."
So we ended up with a documentation graveyard — hundreds of files that were technically present but practically misleading. Worse than no documentation at all, because they created false confidence.
The Refactoring Trap
The consequences showed up most painfully during refactoring cycles.
When a developer opened a module they hadn't touched in six months, they'd find three or four documentation files associated with it: one from the original implementation, one from a partial refactor, one that seemed like a summary of the other two. Each file had slightly different descriptions of how the system worked. Each contradicted the others in subtle ways.
Which one was current? No one knew.
So developers did what developers do under uncertainty: they ignored all of them and went straight to the code. The documentation was not just useless at this point — it was actively slowing people down by creating doubt and requiring manual cross-referencing.
And then the same bugs came back. The same architectural mistakes. Because the lessons learned from the last refactor were buried in an outdated doc that nobody trusted anymore.
What We Should Have Done
We treated documentation like output, not like a living artifact.
The mistake wasn't using Claude to write docs. Claude is genuinely excellent at drafting clear, structured documentation from code context. The mistake was treating every generated doc as "done" — then moving on without any strategy for keeping it synchronized with the source.
Documentation needs ownership. It needs a lifecycle. If you can't commit to updating a doc when the code changes, you'd be better off with a short, manually-written doc that's actually accurate, than ten AI-generated docs that are confidently wrong.
A few principles we've adopted since then:
- One source of truth per module. One doc file, linked directly in the code, with a clear owner.
- Docs are part of the PR. If you change behavior, you update the doc. No exceptions.
- Regularly audit and delete. An outdated doc should be treated like dead code — remove it.
- Use Claude to draft, not to decide scope. The LLM can write the words, but humans should decide what needs to be documented and at what granularity.
The Real Lesson
LLMs are remarkably good at generating documentation. That's precisely the danger.
The ease of generation creates an illusion of completeness. You can produce a thousand words of polished, coherent documentation in seconds. It feels like you've done the work. But documentation that isn't maintained is a liability, not an asset.
We confused volume with value. And for a while, our codebase paid the price — in confusion, in duplicated effort, and in bugs that kept resurfacing because no one could find the lesson that had already been learned and written down, buried somewhere in a file that no one trusted anymore.
Top comments (0)