RAG knowledge rots because it's maintained separately from the code it describes. The Knowledge Publisher Pattern fixes it at the source: every module ships its own versioned knowledge artifact.
Everyone is optimising the wrong part of RAG. The conversations I see in enterprise architecture circles are almost entirely about retrieval: chunk size, embedding models, vector database selection, reranking strategies. These are real problems. They are not the biggest problem. The biggest problem is that the knowledge corpus you are retrieving from is already stale by the time the agent queries it — and it gets staler with every release.
This is not a technology problem. It is a coupling problem. And we have solved structurally identical coupling problems before.
The Knowledge That Nobody Maintains
Here is what a typical enterprise RAG pipeline looks like in practice. A team decides to give an AI agent access to knowledge about their platform — how the billing module works, what fields the contact entity exposes, what rules govern order processing. They write documentation. They chunk it, embed it, load it into a vector database. The agent performs impressively.
Three months later, the billing module ships a breaking change. The contact entity gets two new fields. The order processing rules are updated to handle a new jurisdiction.
Nobody updates the vector database. The team that owns the RAG pipeline doesn't know about the change; the team that shipped the change didn't know there was a RAG pipeline to update. The agent continues answering questions confidently — and confidently wrong.
This is not a hypothetical failure mode. It is the default outcome in any organisation where knowledge maintenance is a separate activity from software development. And in enterprise ecosystems — ISVs, platform teams, large inner-source shops — knowledge maintenance is almost always separate.
The root cause is architectural: the knowledge is decoupled from the source that produces it. It has a different owner, a different release cycle, and no version coupling to the thing it describes. When the thing changes, the knowledge doesn't.
We Have Solved This Before
The rot-from-decoupling problem is not new. We have encountered it in adjacent domains and solved it — each time by moving the responsibility for publishing knowledge to the component that owns the thing being described.
| Prior art | How it defeated staleness |
|---|---|
| OpenAPI | API documentation used to live in Word documents, diverging immediately from the implementation. The fix was code-first annotation: the team annotates the implementation, documentation is generated at build time, and the spec ships with the binary. They cannot diverge because they share a source. |
| Maven classifiers |
mylib-1.0.0-sources.jar, mylib-1.0.0-javadoc.jar. Versioned identically to the binary, referenced by coordinates. When v2.0.0 ships, its documentation ships with it. There is no v2.0.0 javadoc maintained separately by a documentation team. |
| Spring Boot auto-config |
META-INF/spring/…AutoConfiguration.imports: each module declares its own integration points inside its own JAR. There is no central registry to keep current. The module is the authority on its own capabilities. |
| Java SPI |
META-INF/services/: every module that implements a service interface self-registers inside its own JAR. The consumer discovers implementors from the artifacts on the classpath — no prior knowledge required. |
The pattern is consistent across all four cases: move the knowledge publication responsibility to the component that owns the knowledge. Ship knowledge alongside the binary. Version them together. Let consumers pull by coordinates.
RAG has not made this move yet.
The Knowledge Publisher Pattern
Core principle: each module that owns knowledge publishes a structured knowledge artifact alongside its binary. The knowledge artifact is versioned identically to the binary. RAG seeders are consumers that declare artifact coordinates, not content.
Three roles carry the pattern. A Publisher is any module that owns knowledge and ships it as an artifact. A Seeder is the consumer that pulls those artifacts by coordinate and indexes them into a vector store. And — for larger estates — an optional Assembler composes the Publishers it depends on into a single aggregate artifact. Most of this article concerns the Publisher; the Seeder and the Assembler follow from it.
The name "Knowledge Publisher" designates a structural role — the same way "event publisher", "metric publisher", or "configuration contributor" designates a role in other distributed systems patterns. A module playing the Knowledge Publisher role is responsible for producing and maintaining the knowledge that describes itself. It is not responsible for knowing who consumes that knowledge or how.
This role inversion is the key move. In the conventional pipeline, a central knowledge team owns the corpus and must track every module's changes. In the Knowledge Publisher Pattern, each module owns its own knowledge slice. The central seeder owns nothing except a list of coordinates to pull from.
Structure
The Knowledge Artifact
A Knowledge Publisher produces a structured artifact — a zip file published using the classifier convention of the ecosystem's artifact registry. In a Maven-based ecosystem, this artifact carries the -knowledge classifier:
com.example:billing-module:2.1.3:knowledge
com.example:contact-service:4.0.1:knowledge
com.example:order-engine:1.7.0:knowledge
The artifact contains a set of knowledge chunks. Each chunk is a self-contained unit with enough metadata for a seeder to index it intelligently:
{
"module": "com.example:billing-module",
"version": "2.1.3",
"chunks": [
{
"id": "billing-module/invoice-lifecycle",
"title": "Invoice Lifecycle",
"body": "An invoice transitions through: DRAFT, PENDING, APPROVED,
REJECTED, VOID. The APPROVED transition requires...",
"tags": ["invoice", "lifecycle", "state-machine"],
"kind": "concept"
},
{
"id": "billing-module/api/create-invoice",
"title": "Create Invoice API",
"body": "POST /api/v2/invoices. Required fields: customerId (string),
lineItems (array)...",
"tags": ["invoice", "api", "create"],
"kind": "api-reference"
}
]
}
The kind field lets seeders apply different indexing strategies to different types of knowledge — concept explanations, API references, domain rules, examples, error codes. The id is stable across versions, enabling a seeder to detect exactly what changed when pulling a new release.
The Build-Time Publisher
The knowledge artifact is generated at build time, not authored by hand as a separate document. This is the constraint that enforces co-versioning: because the knowledge lives inside the build, it must be updated when the code is updated, or the build fails.
The generation mechanism depends on the ecosystem. The most practical approaches are:
- Annotation-driven generation — developers annotate key classes, methods, and fields with knowledge metadata. A build plugin collects annotations and emits the chunk manifest. This mirrors the OpenAPI approach and produces knowledge tightly coupled to the actual implementation.
-
Convention-driven discovery — the build plugin scans structured Markdown or YAML files in a designated source directory (e.g.,
src/main/knowledge/). These files are first-class source artifacts, committed alongside code, reviewed in the same pull request, and subject to the same linting. Coordinates and version are injected at build time. - Hybrid — annotation-driven for API surface (fields, endpoints, state transitions), convention-driven for conceptual knowledge (domain rules, integration notes, operational constraints). Most practical for mature codebases.
The critical property of all three approaches: the knowledge is authored close to the thing it describes, in the same repository, by the team that owns the code.
The Seeder as Consumer
The RAG seeder's job changes entirely under this pattern. It is no longer a knowledge curator; it is a knowledge consumer. Its configuration is a list of artifact coordinates:
knowledge-sources:
- artifact: com.example:billing-module:2.1.3:knowledge
- artifact: com.example:contact-service:4.0.1:knowledge
- artifact: com.example:order-engine:1.7.0:knowledge
At deployment time — or on a scheduled refresh — the seeder pulls each artifact from the registry, unpacks the chunk manifest, and indexes the chunks into the vector database. When billing-module ships 2.1.4, the coordinate is updated and a re-index is triggered. The seeder has no opinion about what billing-module knows about itself; it only knows how to pull and index.
This separation of concerns has a significant operational consequence: the seeder can be operated by a platform team with no domain expertise. They do not need to know what an invoice lifecycle is. They need to know how to pull an artifact and run an indexing job.
Version Coupling
The most important property of the pattern is version coupling: the knowledge artifact for version 2.1.3 of a module describes exactly version 2.1.3 of that module — no more, no less.
This is not a promise that teams make to each other. It is a structural guarantee enforced by the build. Because the knowledge artifact is produced by the same build that produces the binary, and carries the same version coordinate, an environment running billing-module:2.1.3 can pull billing-module:2.1.3:knowledge and know it is accurate for that exact version.
Rollbacks are handled correctly by the same mechanism. If the team rolls back from 2.1.4 to 2.1.3, the seeder pulls 2.1.3:knowledge and the agent's knowledge regresses with the system. There is no stale-forward problem.
Composing Knowledge: The Aggregate Artifact
Listing every module coordinate in the seeder configuration works, but it quietly pushes a dependency-tracking burden onto the seeder — the very coupling the pattern set out to remove, resurfacing one level up. When a deployable already knows its own dependency graph, there is a better option.
Introduce the third role: the Knowledge Assembler. This is the module that sits closest to the agent — often an "agent data" module. It owns the agent-specific knowledge that no leaf module owns (calling conventions, domain framing, the glue an agent needs), and it already references the business modules the agent reasons over. At build time it does one extra thing: it resolves the knowledge artifacts of the Publishers it references, composes them with its own chunks, and publishes a single aggregate knowledge artifact.
The seeder configuration collapses to a single coordinate:
knowledge-sources:
- artifact: com.example:agent-data:2.3.0:knowledge
This is not a new idea; it is the knowledge-tier equivalent of two mechanisms every enterprise build already uses. An assembly (or shaded / uber-jar) rolls a module's transitive code dependencies into one deployable unit. A Bill of Materials lets a single coordinate stand in for a curated set of versioned components. The aggregate knowledge artifact is a bill of materials for an agent's knowledge: it pins exactly which Publishers, at exactly which versions, compose the knowledge the agent reasons over — and it is one coordinate to deploy.
Composition, not merging. The cleanest assemblers do not flatten every Publisher's chunks into one namespace; they compose by placement — each Publisher's chunks and its own manifest travel together, in their own space within the aggregate, and the Assembler adds its overlay on top. The consumer resolves each chunk relative to where it came from. This keeps a Publisher's knowledge self-contained, and lets a Publisher be added to or removed from the aggregate without rewriting anyone else's chunks.
Because the aggregation is transitive and happens at build time, it inherits every property established so far. The aggregate is itself versioned — pinning agent-data:2.3.0 transitively pins every Publisher version it rolled up, so version coupling holds all the way down. And because the composition runs inside the Assembler's build, a referenced Publisher that ships a breaking knowledge change forces the Assembler to rebuild, surfacing the change where it can be reviewed rather than letting it drift in silently.
One rule the Assembler must fix deliberately: when two Publishers — or a Publisher and the Assembler's own overlay — describe the same chunk id, the collision needs a deterministic resolution. The two workable strategies are last-wins by document id at seed time, or precedence pinned by declaration order in the Assembler. Either is defensible; choosing neither is not. An aggregate that resolves the same collision differently on different builds is its own kind of staleness.
Design Considerations and Trade-offs
What this pattern solves — and what it does not
The Knowledge Publisher Pattern solves structural knowledge staleness — the drift between a module's documented behaviour and its actual behaviour as the module evolves. It does not solve behavioural knowledge problems: emergent patterns in logs, performance characteristics under load, failure modes discovered in production. That class of knowledge requires runtime observation and belongs in a different pipeline.
Do not conflate the two. A module can perfectly publish its invoice state machine in a knowledge artifact and still have an undocumented failure mode under concurrent high-volume processing. The agent will know the rules; it will not know the edge cases that only appear in production telemetry. Both matter; neither substitutes for the other.
Discipline at authoring time
The pattern transfers responsibility for knowledge quality to the teams that own the modules. This is exactly where that responsibility belongs — but it requires discipline. Teams must treat knowledge authoring as a first-class engineering activity, not a documentation afterthought. Pull requests that change behaviour without updating the corresponding knowledge chunks should fail review.
The most reliable enforcement is structural. If knowledge is annotation-driven, the annotation is adjacent to the code. If it is convention-driven, a CI check verifies that the knowledge artifact is non-empty and passes a schema lint before the binary is published. The social norm is not enough; the build gate is.
Chunk granularity
Chunk granularity determines retrieval quality. Too coarse — one chunk per module — and retrieval is too broad to be useful. Too fine — one chunk per method — and the context window fills with fragments. A useful heuristic: one chunk per concept that a domain expert would explain as a standalone topic. "Invoice lifecycle" is a concept. "What fields does an invoice have" is a concept. "What is field 14 in the invoice schema" is a detail that belongs inside a concept chunk, not a chunk of its own.
Artifact registry dependency
This pattern requires an artifact registry with classifier support — Maven Central, Artifactory, Nexus, or equivalent. Organisations that already distribute software through such a registry get this for free; those that don't must adopt one. In enterprise ecosystems this is almost never a blocker, but it is worth stating explicitly: the pattern trades a knowledge maintenance process for an artifact distribution infrastructure dependency. That is nearly always the right trade for organisations at the scale where RAG becomes valuable.
The cold start problem
New modules have no knowledge artifact until they ship one. Existing modules have years of knowledge that is not yet in any artifact. Retrofitting an entire codebase is non-trivial. The pattern is additive: new publishers appear over time, and you do not need a full corpus migration before going live.
When Not to Apply
Small teams with a single module. The overhead of a knowledge artifact publication pipeline is not justified when one team owns the entire knowledge surface. A well-maintained README or documentation site is simpler and sufficient.
Rapidly prototyping systems. In a system changing daily, the latency between a code change and a re-indexed knowledge artifact introduces friction that may slow experimentation. Apply the pattern when the system has reached a release cadence where artifacts are a natural unit of deployment.
Primarily behavioural knowledge. If the most important knowledge about a system is how it behaves under load, what its failure modes are, and how incidents have been resolved — that knowledge lives in logs, traces, and incident reports, not in code-adjacent artifacts. Pair the Knowledge Publisher Pattern with a separate observability pipeline for behavioural knowledge; do not try to make one serve both purposes.
Organisations without artifact registry infrastructure. The Knowledge Publisher Pattern is a layer on top of artifact distribution, not a substitute for it. Solve that problem first.
The Deeper Principle
The Knowledge Publisher Pattern is an instance of a broader architectural principle: make the producer responsible for the interface it exposes to consumers.
We apply this principle consistently in modern software architecture. The team that owns an API publishes its specification. The team that owns a library publishes its documentation. The team that owns a Kafka topic publishes its schema. The team that owns a module publishes its metrics. In every case, the insight is the same: the producer has the most accurate, most current knowledge of its own interface. Delegating that publication to a separate team introduces a lag, a translation, and an ongoing maintenance burden that compounds over time.
RAG knowledge is an interface. It is the interface between a module's internal logic and the agent that reasons about it. Applying the producer-publishes principle to that interface is not a novel architectural leap — it is the consistent application of a principle that enterprise architecture has validated repeatedly in adjacent domains.
What is new is the artifact. Give it a coordinate, version it with the binary, and it becomes first-class infrastructure — pullable, cacheable, auditable, and replaceable with the same tooling already used to manage binaries.
The structural ancestors of this pattern — Maven classifiers, Spring Boot contributors, Java SPI — did not just solve a technical problem. They changed what teams were responsible for. They made decentralised ownership of integration points the path of least resistance. The Knowledge Publisher Pattern proposes the same shift for RAG.
Applying the Pattern to an Existing Estate
The practical migration path for an organisation with an existing RAG corpus:
- Identify the high-value publishers first. Not all modules generate agent queries equally. Audit your RAG query logs to find which topics the agent is queried about most frequently. Those are your first Knowledge Publisher candidates.
- Choose your authoring approach per module type. Annotation-driven generation suits modules with a clear public API surface. Convention-driven suits modules with rich domain logic that resists annotation. Hybrid suits both. Do not impose a single approach across the estate.
- Run the seeder in parallel during migration. While the existing corpus is still being maintained manually, run the artifact-seeded knowledge alongside it. Compare retrieval quality. Retire the manual entries module by module as their artifact-based replacements come online.
- Version the seeder configuration. The list of artifact coordinates is itself a configuration artifact. Version it, review it, and deploy it with the same rigour as any other infrastructure configuration. A seeder configuration out of sync with the running environment is the same problem you started with, one level up.
Closing
The RAG staleness problem will not be solved by better retrieval. It will be solved by better publishing.
The ecosystem already has the infrastructure: artifact registries, classifier conventions, build plugins, deployment pipelines. The missing piece is the cultural and architectural move that says knowledge publication is a first-class responsibility of the team that owns the code — not a documentation activity, not a central platform concern, but an engineering deliverable that ships with every release.
The agents that reason over your systems are only as good as the knowledge you give them. If that knowledge is maintained by a team that does not own the code, it will diverge. If it is authored alongside the code and published with every release, it will not.
That is not a retrieval problem. That is an architecture problem. And it has an architecture solution.
This is a pattern proposal, and patterns mature through use and argument. If you are building RAG pipelines in enterprise ecosystems today — or deliberately not — I would genuinely like to hear how you are drawing these lines.
Top comments (0)