DEV Community

Morgan Sun
Morgan Sun

Posted on

Lease Your Doc Claims: Revoke Model-Drafted Prose the Moment a Source File Moves

Docs written by a model go stale in a way human docs do not. The prose keeps the same confident tone long after the flag it describes was renamed.

Illustrative scenario, not a reported incident: a team documents --timeout in docs/cli.md. Two sprints later the flag becomes --request-timeout. Nothing in review flags the paragraph, because the paragraph still reads well. Readers hit a usage error and trust the next page a little less.

The usual fix is "review docs more often." That does not scale. A better fix is to treat each factual sentence as a lease: it is valid only while the source it was derived from is unchanged. When the source moves, the lease expires and the sentence loses its right to be published until a human renews it.

Split the doc by who can be wrong

Before drafting anything, classify each claim block. Three classes are enough.

Class Drafted by Bound to CI behavior when bound source changes
machine model, from extracted source values a file path + selector lease expires; block is quarantined
narrative model, from a human outline nothing (no facts allowed) untouched, but scanned for banned tokens
human a named reviewer only reviewer identity CI refuses a model-authored diff

The rule that makes this work: a model may only write machine blocks while their lease is fresh, and it may only use values that the build extracted from the bound source. If a value it needs is not in that extracted set, the correct output is a placeholder, not a guess.

OWNER-REQUIRED: default value of --request-timeout (not present in bound source)
Enter fullscreen mode Exit fullscreen mode

That placeholder is the whole safety property. A model that would rather write 30s than admit ignorance is the failure mode; a model that emits OWNER-REQUIRED is doing its job.

The lease manifest

One YAML file declares the leases. Anchors refer to HTML comment markers already in the Markdown, so the checker never has to parse prose.

# docs.claims.yaml — illustrative, not executed
version: 1
claims:
  - id: cli.flag.request-timeout
    doc: docs/cli.md
    anchor: claim-request-timeout
    owner: machine
    bindings:
      - path: src/cli/args.ts
        selector: 'regex:--request-timeout[^\n]*'
    digest: 'sha256:REPLACE_ME'

  - id: retry.semantics
    doc: docs/retries.md
    anchor: claim-retry-semantics
    owner: human
    reviewer: '@docs-owner'
Enter fullscreen mode Exit fullscreen mode

And in the Markdown:

<!-- claim:cli.flag.request-timeout:start -->
`--request-timeout` defaults to `30s`.
<!-- claim:cli.flag.request-timeout:end -->
Enter fullscreen mode Exit fullscreen mode

A checker you can audit line by line

This script is illustrative and has not been run as written. It is deliberately small so you can read the whole failure surface.

# tools/docs_lease.py — illustrative, unexecuted
import hashlib, pathlib, re, sys
import yaml

ROOT = pathlib.Path('.')

def digest_for(claim):
    h = hashlib.sha256()
    for b in claim['bindings']:
        raw = (ROOT / b['path']).read_bytes()
        h.update(b['path'].encode())
        h.update(raw)
    return 'sha256:' + h.hexdigest()

def main():
    manifest = yaml.safe_load((ROOT / 'docs.claims.yaml').read_text())
    stale = []
    for claim in manifest['claims']:
        if claim['owner'] != 'machine':
            continue
        actual = digest_for(claim)
        if actual != claim['digest']:
            stale.append((claim['id'], claim['doc'], claim['anchor']))

    if not stale:
        print('docs-lease: all machine claims fresh')
        return 0

    for cid, doc, anchor in stale:
        print(f'LEASE EXPIRED {cid} in {doc}#{anchor}')
        print(f'  -> quarantine to docs/_quarantine/{cid}.md and assign an owner')
    return 1

if __name__ == '__main__':
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode

Run it in CI as a required check. When it exits 1, the pull request that changed src/cli/args.ts cannot merge until someone either re-stamps the digest after reading the new value, or moves the claim to owner: human and rewrites it by hand.

The drafting pass, and where free capacity matters

Lease checking is cheap. Drafting is the part that needs a model, and drafting is also the part you want to run on every documentation PR rather than once a quarter.

That is the narrow place where MonkeyCode's free model access and free server option are relevant to this workflow — and only there. The drafting job takes the extracted machine claims plus a human outline, writes candidate text into the narrative blocks, and is barred from touching human blocks. Running that job on a free tier (the operator currently advertises a free token allowance and a no-cost server option; confirm the current figure and terms on the project page before you depend on them, since allowances are time-sensitive) is what makes "draft on every PR" affordable for a small team.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Nothing in the lease design depends on that product. The manifest, the checker, and the ownership split work with any model endpoint, or with no model at all.

Known weaknesses of this design

  • Whole-file hashing is coarse. A comment-only edit in src/cli/args.ts expires the lease and creates review noise. The selector field exists to narrow this, but the illustrative checker ignores it; implementing selector-scoped digests is real work.
  • Renames look like changes. Moving a file invalidates every claim bound to it, even when the content is identical.
  • Generated sources are a trap. If args.ts is itself code-generated, the digest tracks the generator's churn, not the contract you care about. Bind to the schema instead.
  • Review fatigue is the real risk. If leases expire weekly, people will re-stamp without reading. Track re-stamp-without-diff as a review metric; if it climbs, your bindings are too broad.

Who should skip this

Teams with fewer than a few dozen documented facts, or docs that are mostly tutorials and rationale, should not build a lease system. The overhead exceeds the benefit. Skip it too if no single reviewer can own documentation decisions — the human class is the load-bearing part, and it collapses without a named person.

How to verify it on your own repo

  1. Pick ten machine claims and one human claim from an existing doc page.
  2. Write the manifest and stamp digests with a small docs-lease stamp command of your own.
  3. Change one bound file and confirm CI fails for exactly that claim and no others.
  4. Run the drafting pass with a deliberately under-specified binding and confirm the model emits OWNER-REQUIRED rather than a number.
  5. Count how many claims expire in a normal week. If the number is high, narrow your bindings before widening the model's drafting scope.

Step 4 is the honest test. Everything else checks plumbing; that step checks whether the model has learned to stop when the evidence stops.

If you want to run the drafting pass before provisioning anything, the free model access and free server option on the MonkeyCode project are a reasonable place to start — and if the lease checker misbehaves on your layout, that is worth filing upstream.

Top comments (0)