DEV Community

Fernando Paladini
Fernando Paladini

Posted on

Your Specs Are Lying: How to Detect Documentation Drift in Every Pull Request

#opensource #productivity #git #devtools

Your repository says one thing.

Your production code does another.

The pull request was approved. The tests passed. The feature works. But the requirement that originally described the behavior was never updated. Neither was the architecture decision record. Six months later, another developer reads those documents and implements the wrong thing with complete confidence.

The documentation exists. It looks authoritative. And it is lying.

This is one of the most dangerous failure modes in spec-driven development: the code changes, but the artifacts that explain the code remain frozen in the past.

That is the problem I built SpecGov to address.

The problem is not writing specifications

Engineering teams already write a surprising amount of documentation:

  • product requirements;
  • architecture decision records;
  • technical design documents;
  • implementation plans;
  • .specs directories;
  • Spec Kit, Kiro, OpenSpec, or custom Markdown artifacts.

Creating these artifacts is not the hardest part. Keeping them aligned with the implementation is.

A specification is usually accurate when someone writes it. Then the code evolves through bug fixes, product changes, refactoring, migrations, and emergency decisions. The specification does not always evolve with it.

Eventually, the repository contains two versions of the system:

  1. The system described by its specifications.
  2. The system implemented by its code.

Git tracks every changed line, but it does not understand that a change under src/auth/ may require an update under docs/auth/ or adr/security/.

Code review can catch that relationship, but only when a reviewer remembers that the relationship exists.

Memory is not a governance system.

Why spec-driven development breaks down

Spec-driven development often focuses on the beginning of a change:

  1. Write a requirement.
  2. Create a design.
  3. Break the design into tasks.
  4. Implement the solution.

That flow works well for creating software. The failure appears later.

A future pull request changes the implementation without returning to the original specification. The team may not even realize that the affected document exists.

This creates several problems:

  • requirements stop representing current behavior;
  • ADRs lose authority;
  • onboarding documentation teaches obsolete assumptions;
  • AI coding agents receive stale context;
  • reviewers must reconstruct relationships manually;
  • teams stop trusting documentation because they cannot tell what is current.

Once developers stop trusting the specs, they stop maintaining them. Stale specifications reduce trust, and reduced trust produces even staler specifications.

The missing piece is not another format for writing specs. It is a governance layer for the formats teams already use.

What is specification governance?

Specification governance is the practice of keeping implementation changes connected to the requirements, decisions, plans, and documentation that explain them.

In practical terms, it answers questions such as:

  • Which artifacts govern this part of the codebase?
  • Did this pull request update a relevant specification?
  • Which implementation paths have no declared specification mapping?
  • Which governed artifacts are stale, empty, or superseded?
  • Who owns an active specification?
  • Can another tool consume the traceability data?

A governance tool does not need to own your specification format. It needs to understand where your artifacts live, how they relate to code, and when those relationships may have been bypassed.

That is the design principle behind SpecGov.

Introducing SpecGov

SpecGov is an open-source, framework-agnostic CLI and GitHub Action for specification governance.

It maps implementation paths to the artifacts that govern them, then checks whether code changes include the expected specification impact.

It works with the files you already have. You describe those files and relationships in a .specgov.yml manifest:

version: 1
mode: advisory

artifacts:
  - path: "docs/**/*.md"
    kind: documentation
    owner: docs
  - path: "adr/**/*.md"
    kind: decision
    owner: architecture
  - path: ".specs/**/*.md"
    kind: specification
    owner: engineering

mappings:
  - code: "src/auth/**"
    specs:
      - "docs/auth/**"
      - "adr/auth/**"
      - ".specs/features/auth/**"
    description: "Authentication behavior must stay aligned with its specs."

rules:
  require_spec_impact_for_code_changes: true
  require_lifecycle_status: false
  require_owner_for_active_specs: false
  stale_after_days: 180

ignore:
  - "node_modules/**"
  - "dist/**"
  - ".git/**"
Enter fullscreen mode Exit fullscreen mode

Now the relationship between code and specifications is no longer tribal knowledge. It is versioned, reviewable, and executable.

What happens when code changes without its spec?

Imagine a pull request changes this file:

src/auth/session.ts
Enter fullscreen mode Exit fullscreen mode

The manifest declares that src/auth/** is governed by docs/auth/**, adr/auth/**, and .specs/features/auth/**.

If none of those artifacts changes, SpecGov reports SPEC_IMPACT_MISSING.

The finding does not automatically claim that the code is wrong. It asks a more useful review question:

This governed implementation changed. Why did none of its source artifacts change with it?

Sometimes the answer is legitimate. A refactoring may preserve all documented behavior. But that decision is now visible in review instead of remaining accidental.

Start with warnings, not bureaucracy

Governance tools often fail because they try to enforce everything immediately. A team installs the tool, receives dozens of failures, and removes it before the rules have a chance to become useful.

SpecGov uses an advisory-first model.

In advisory mode, findings produce warnings without blocking the pull request:

mode: advisory
Enter fullscreen mode Exit fullscreen mode

This allows the team to observe real changes, refine mappings, and identify false positives.

When a mapping becomes trustworthy, the repository can move to strict enforcement:

mode: strict
Enter fullscreen mode Exit fullscreen mode

The same missing spec impact can then fail CI.

A practical rollout looks like this:

  1. Map one or two high-value areas.
  2. Run SpecGov in advisory mode.
  3. Review findings across real pull requests.
  4. Improve the manifest.
  5. Enforce only the relationships the team trusts.

Governance grows with evidence instead of arriving as an all-or-nothing mandate.

SpecGov is not another spec framework

SpecGov does not ask you to migrate every requirement into a proprietary schema. It does not replace Spec Kit, Kiro, OpenSpec, ADRs, Markdown design documents, or your internal process.

It governs those artifacts from one shared layer.

A repository may use:

docs/product/**
adr/**
.specs/**
.kiro/specs/**
specs/**
Enter fullscreen mode Exit fullscreen mode

SpecGov can treat all of them as governed artifacts.

You do not need one universal spec format. You need one way to declare which artifacts matter and how they relate to the implementation.

How to use SpecGov

Install the CLI globally:

npm install -g specgov
Enter fullscreen mode Exit fullscreen mode

Or run it without a global installation:

npx specgov --help
Enter fullscreen mode Exit fullscreen mode

1. Create the manifest 

specgov init
Enter fullscreen mode Exit fullscreen mode

Edit the generated .specgov.yml to match your repository.

2. Discover governed artifacts 

specgov scan
Enter fullscreen mode Exit fullscreen mode

For machine-readable output, use:

specgov scan --format json
Enter fullscreen mode Exit fullscreen mode

3. Check a code change 

specgov check-pr --changed-file src/auth/session.ts
Enter fullscreen mode Exit fullscreen mode

You can pass multiple changed files or let the GitHub Action calculate the pull request diff.

4. Generate a trace index 

specgov trace --out .specgov.trace.json
Enter fullscreen mode Exit fullscreen mode

The trace index connects artifacts, mappings, and matched files in a machine-readable format.

5. Inspect drift 

specgov drift
Enter fullscreen mode Exit fullscreen mode

The drift report identifies structural maintenance problems such as stale, empty, orphaned, or superseded artifacts.

SpecGov does not use an LLM to decide whether two paragraphs are semantically equivalent. Its analysis is deterministic and based on repository facts and declared metadata. That boundary is intentional.

Add SpecGov to pull requests

The GitHub Action runs the same checks during code review:

name: SpecGov

on:
  pull_request:

jobs:
  specgov:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v5
        with:
          fetch-depth: 0

      - uses: paladini/specgov@v0.1.0
        with:
          mode: advisory
          base-ref: ${{ github.event.pull_request.base.sha }}
          head-ref: ${{ github.event.pull_request.head.sha }}
Enter fullscreen mode Exit fullscreen mode

Start with advisory. Move high-confidence mappings to strict when the warnings have proved useful.

SpecGov uses three exit-code categories:

  • 0 for a pass or advisory warnings;
  • 1 for governance failures in strict mode;
  • 2 for runtime or configuration errors.

Lifecycle metadata

Governed Markdown files can include lifecycle information:

---
status: active
owner: platform
last_verified: 2026-07-22
---

# Authentication session contract
Enter fullscreen mode Exit fullscreen mode

Supported statuses include draft, active, superseded, deprecated, and archived.

The goal is not to delete every obsolete decision. The goal is to make its status explicit.

Why SpecGov is deterministic

SpecGov runs locally or in your CI environment. It does not require an API key, hosted service, LLM, telemetry, or repository upload.

The same manifest and changed files produce the same governance result. This matters because a CI gate must be reproducible.

SpecGov focuses on checks a repository can enforce reliably:

  • a governed code path changed;
  • its mapped artifacts did not;
  • an implementation path has no mapping;
  • a declared artifact is missing or stale;
  • required metadata is absent.

These are facts a pipeline can evaluate.

What problem does SpecGov solve?

The concise answer is:

SpecGov detects when code changes bypass the specifications, decisions, and documentation that are supposed to govern them.

It turns hidden relationships into a Git-native contract.

For developers, it identifies which specification may need to change with a piece of code. For reviewers, it highlights whether a pull request contains enough specification context. For platform and architecture teams, it makes governance coverage visible and machine-readable.

What SpecGov does not do

SpecGov does not prove that a specification is correct. It does not prove that the implementation satisfies every sentence in a requirement. It does not replace tests, code review, architecture review, or domain knowledge.

It detects structural governance gaps.

A changed artifact may still contain a poor update. A mapped document may still be inaccurate. SpecGov provides the signal that a relationship deserves review; humans and specialized verification tools still evaluate its quality.

Frequently asked questions

What is specification drift?

Specification drift happens when implementation behavior changes while its requirements, design documents, ADRs, or other source artifacts remain unchanged or become obsolete.

Does SpecGov require a specific spec framework?

No. SpecGov is framework-agnostic. It works with Markdown documents, ADRs, custom .specs folders, Kiro artifacts, Spec Kit plans, and other repository-local conventions.

Does SpecGov use artificial intelligence?

No. Version 0.1 uses deterministic filesystem, configuration, and Git analysis. It does not call an LLM or send repository data to an external service.

Can SpecGov block a pull request?

Yes. Advisory mode reports warnings without failing CI. Strict mode returns a failure when governance rules are violated.

Can I use SpecGov for only part of a repository?

Yes. Begin with one high-value code path and its related artifacts, then expand the manifest gradually.

Is SpecGov free and open source?

Yes. SpecGov is available under the MIT License.

Specifications only matter while they remain trustworthy

Writing a specification is not enough.

A requirement that nobody updates becomes historical fiction. An ADR that no longer describes the architecture becomes a trap. A design document that quietly diverges from the code creates confidence where there must be doubt.

The real challenge is not producing more documentation. It is keeping the documentation connected to the software as both evolve.

SpecGov makes that relationship visible in the place where engineering decisions already happen: Git and pull requests.

Try it:

npx specgov init
npx specgov scan
npx specgov check-pr --changed-file src/auth/session.ts
Enter fullscreen mode Exit fullscreen mode

Then ask the uncomfortable question:

If this code changed, which specification was supposed to change with it?

Which part of your repository is governed only by human memory?

Top comments (1)

Collapse
 
alexshev profile image
Alex Shev •

Documentation drift is especially dangerous with AI-assisted work because the model will happily trust stale specs if they look official. Checking docs in the PR loop turns the spec back into a living artifact.