It's 14:00, while you're focusing on your task, here's new slack message. "Hey, can you give 30 minutes to explain how to re-deploy your service?"
You will stop your work, schedule a meeting, make presentation, and have Q&A. It is the third time this month, and you are repeating same thing. This interruption is invisible tax on engineering productivity.
Of course, this is duty of seniors, and service maintainer. But let's think of making this process more efficient.
Recent software development teams are beginning to use a different approach. Instead of storing rules in private chats, they record these conventions directly in the repository in formats optimized for Large Language Models (LLMs).
By treating the codebase as a machine-readable knowledge factory, teams can delegate "question-answering + context sharing" to AI agents.
Moving Beyond the Chatbot Assistant
With the advancement of coding agents, general AI coding workflows are evolving to be more 'assistant centric'.
Now developer writes code and occasionally asks a chatbot to generate a regular expression or debug a stack trace. This leads to increase of individual throughput, but it leaves team level communication untouched.
A more powerful pattern is agent-driven development. Here, AI agents are primary authors of code, tests, interface schemas, and documentation. The human engineer shifts from an implementer to an architect and reviewer. The agent reads the repository, aligns with its conventions, runs tests, and submits pull requests.
This model changes what a repository must contain.
When a new developer joins a project, they absorb implicit guidelines through code reviews and casual conversations. An AI agent lacks access to these channels. Without explicit codification, agents write code violating team standards.
The goal is to design repositories that are optimized for AI agents while remaining readable for humans. This approach makes implicit project knowledge explicit.
The Knowledge Gap and the Need for Hierarchy
When AI agent receives a task in an unfamiliar repository, it faces immediate questions:
- What is this code for?
- Where do new modules go?
- Naming/Comment patterns?
- How errors need to be handled?
The distance between "what agent needs to know" and "what the repository files explicitly state" is knowledge gap. In early stages, agents might write a correct logic but place the file in the wrong place or ignore the project's error-handling wrapper. Correcting these errors requires manual code review, forcing the human to explain the rule after their work.
Writing longer system prompts is poor solution. LLMs have real attention limits. Dumping pages of instructions or irrelevant files into context degrades performance fast, and it will make trivial bugs. Structuring repo documentation into a clean hierarchy will help avoiding this.
A Four-Tier Documentation Hierarchy
Organizing documentation into distinct tiers limits prompt length and keeps instructions focused.
| Tier | File or Artifact | Target Audience and Purpose |
|---|---|---|
| Level 0 | AGENTS.md |
Universal dispatcher: build commands, lint rules, dependency paths, and document pointers. |
| Level 1 | agent-reference.md |
Architecture overview, JSON schemas with examples, and CLI conventions. |
| Level 2 | specs/*.md |
Interface contracts using formal language. |
| Level 3 | Source Code & Tests | Execution truth of the project. |
Level 0: The Dispatcher (AGENTS.md)
Coding agent/tools inject AGENTS.md into agent sessions automatically. It lists basic commands for building and testing, specifies formatting tools, and points to deeper documentation files. Keeping Level 0 lean preserves agent context space.
Level 1: The Reference Guide (agent-reference.md)
The guide contains high-level system design, data schemas with concrete JSON examples, and common CLI syntax. It provides context without cluttering daily coding prompts.
Level 2: The Specifications (specs/*.md)
These documents define inputs, outputs, error handling, and performance boundaries as formal contracts. When an agent implements a feature, the corresponding spec is its target.
Modern coding tools extend Level 2 through pattern-scoped instructions. Small markdown files inside here automatically loaded only when agents modify specific file paths.
For example, tests.instructions.md placed in test directories injects rules like "import helpers from tests/utils.py" or "avoid hardcoding port numbers" only when agents enter that directory. This keeps guidelines local and prevents prompt bloat.
Level 3: Ground Truth
Source code and test suites form the final verification layer.
A strict rule governs this hierarchy: every fact exists in exactly one place. If a database schema exists in both README and agent-reference.md, files eventually drift. Duplicate documentation leads to cached errors. Teams must use hyperlinks to reference single sources of truth rather than copy content across files.
Preventing Documentation Drift: Keeping the Knowledge Factory Fresh
Bridging knowledge gaps with structured documentation works only if documentation stays in current state. Documentation drift is a lifecycle problem, because codebases evolve while written project guidelines remain static. Common failure occurs when dependencies change.
For example, if a team updates a task orchestrator from version 2 to version 3, documented patterns become legacy. If developers do not manually update repository rules, agents will write obsolete code from stale guides.
So here's 2 layer process.
Code-to-Doc Validation
- Harness Enforced Correction: Linters and compiler checks run inside the agent execution environment. When an agent follows stale instructions, the harness rejects changes and feeds error logs back into the prompt for immediate correction.
- Semantic Mismatch Detection (Doc-to-Test): Pipelines use LLMs to generate unit tests directly from repository instructions. If tests fail against the current codebase — for example, calling an API signature that no longer exists — the pipeline flags documentation as stale.
- Dependency-Driven PR Triggers: CI pipelines map package configuration files to related documentation. Bumping a dependency version without updating related agent guidelines triggers build alerts or blocks merge attempts.
- Executable Source-of-Truth: Teams generate specifications directly from code
docstringsor runtime schemas like OpenAPI. Documentation rebuilds with every compile, eliminating manual synchronization.
Retrieval Pipeline Integrity
Vector database indices represent another drift vector:
- LangChain Indexing API (
RecordManager): MD5 hashes track Markdown file changes. Modified or new segments re-embed automatically; deleted pages clear automatically. - Semantic Markdown Chunking: Header boundaries (
##and###) split files instead of arbitrary character counts. When specific sections update in Git, only those chunks upsert into vector stores. - Disposable Vector Indexes: Vector databases act as derived artifacts. Git remains source of truth. If indices drift or embedding models change, databases clear and rebuild from scratch.
The Human Workflow
Automated checks detect inconsistencies without solving them. The ultimate defense against documentation drift is team discipline. Teams build cultures where documentation maintenance matches code review standards.
Two practices matter:
- Definition of Done: Updating associated agent guide files is a mandatory checklist item before code merges.
- Commit-on-Failure: Correcting an agent's mistake during code review requires committing the updated rule to instruction files immediately.
Without these habits, automated checks create noise. Keeping knowledge factories fresh remains a human responsibility.
Top comments (0)