DEV Community

Avery Lin
Avery Lin

Posted on

A Free-Tier Docs Drafting Pipeline: What the Model May Commit and What You Own

Documentation generation breaks at the ownership boundary more often than it breaks at the prompt, and the fix is structural rather than textual. A model can draft an API reference from an AST, but it cannot decide that a changed parameter breaks a compatibility promise. Free model access and a free server change the economics of regeneration, which makes the boundary rules the only part of the pipeline that matters.

The workflow below uses MonkeyCode's free model access and free server option to keep the marginal cost of a draft near zero. That cost profile lets the job run on every merge instead of once per release. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The artifact is a small Python script that extracts changed public symbols and checks them against your reference docs, and it produces a draft branch containing only the missing fragments.

Why Docs Rot Is a Cost Problem, Not a Quality Problem

Metered per-token generation gets skipped on boring commits, and boring commits are exactly where documentation rot begins. A one-line refactor rarely justifies a paid API call, so the paragraph about the refactored function stays stale until someone files an issue. A scheduled job with no per-run meter removes the skip decision entirely, because regeneration costs nothing at the margin.

The second reason is infrastructure friction, because a docs job that needs a dedicated VM lives on someone's laptop. A free server option removes that excuse, and the job becomes a cron entry that produces a draft branch while you sleep. None of this fixes prompt quality, and that is the point, because the prompt is the smallest part of a docs pipeline. The model's draft quality matters less than the rules that decide what happens to the draft.

Step 1: Write the Ownership Matrix Before the Prompt

The matrix is a contract between the model and the humans, and it must exist before the first draft is generated. Anything derivable from the diff belongs to the model, and anything that implies a promise belongs to a human.

Artifact Model may draft Human must own
API reference Signatures, parameter names, return types Parameter semantics, error behavior, edge cases
Changelog entry Summaries from commit messages Breaking-change classification, version impact
Migration note Diff hunks and renamed identifiers Compatibility promises, rollback plan
Architecture decision Never Rationale, alternatives, consequences
Security note Never Threat model, disclosure status, affected versions
Deprecation notice Draft text only Timeline, replacement guarantee, removal policy

The pattern is simple: the model owns what the diff proves, and the human owns what the diff implies. When a draft crosses a row in the owned column, the pipeline must stop and ask for a human, not silently commit.

Step 2: Schedule the Draft Job on the Free Server

The script below is the reproducible core of the pipeline, and it is deliberately small. It reads the diff between your base branch and HEAD, and parses changed Python files with the standard library. Each public symbol is then checked against your reference docs, and the missing ones are listed in a JSON report.

#!/usr/bin/env python3
"""docs_draft.py -- draft doc fragments for changed public symbols."""
import ast, json, os, subprocess, sys
from pathlib import Path

DOC_ROOT = Path("docs/reference")
BASE = os.environ.get("BASE_REF", "main")

def changed_files():
    out = subprocess.run(
        ["git", "diff", "--name-only", BASE, "HEAD"],
        capture_output=True, text=True, check=True,
    )
    return [p for p in out.stdout.splitlines() if p.endswith(".py")]

def public_symbols(path):
    tree = ast.parse(Path(path).read_text())
    found = []
    for node in tree.body:
        # top-level symbols only; extend for methods if needed
        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and not node.name.startswith("_"):
            found.append(f"{path}::{node.name}")
        elif isinstance(node, ast.ClassDef) and not node.name.startswith("_"):
            found.append(f"{path}::{node.name}")
    return found

def covered(symbol):
    name = symbol.split("::")[-1]
    return any(name in p.read_text() for p in DOC_ROOT.rglob("*.md"))

def main():
    symbols = [s for f in changed_files() for s in public_symbols(f)]
    missing = [s for s in symbols if not covered(s)]
    report = {"changed_symbols": len(symbols), "undocumented": len(missing), "items": missing}
    print(json.dumps(report, indent=2))
    if missing and os.environ.get("DRAFT_DIR"):
        draft_dir = Path(os.environ["DRAFT_DIR"])
        draft_dir.mkdir(exist_ok=True)
        (draft_dir / "draft.md").write_text(
            "<!-- model draft: verify before commit -->\n"
            + "\n".join(f"## {s}\n\nTODO: one-paragraph description.\n" for s in missing)
        )
    return 1 if missing else 0

if __name__ == "__main__":
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode

The script intentionally does not call a model, because the model call belongs in the orchestration layer. Point the job at MonkeyCode's free model access through any OpenAI-compatible client, and let the script decide which symbols need a draft. Schedule the job on the free server option as a cron entry or a scheduled workflow, and point the output at a draft branch. A typical schedule is nightly for a small repository, and the job should take under a minute for most diffs.

Step 3: Route Drafts to a Branch, Not to Main

The ownership matrix only works if the pipeline has a place to put drafts that is visibly not production. Follow these steps for every run:

  1. Create or reset a branch named docs/draft from the current main.
  2. Run the coverage script and write the missing fragments into docs/draft/draft.md.
  3. Send the fragments to the model for a first-pass expansion, with the matrix as the system prompt.
  4. Open a pull request labeled docs-draft, and never enable auto-merge on it.
  5. Assign the PR to a human whose name appears in the owned column of the matrix.

The branch name and the label are the enforcement mechanism, because they make the draft's status visible to everyone. A draft that looks like a real PR will get reviewed like a real PR, which is exactly what you want.

Step 4: Measure Staleness With a Coverage Gate

The script returns exit code 1 when any changed symbol is undocumented, and that return code is the raw material for a trend. Run it on a schedule, store the JSON report, and track the undocumented count over time instead of treating each run as a one-off. A coverage gate that blocks merges will be disabled at the first Friday deadline, so keep this job as a report rather than a hard gate. Watch the delta between the previous run and the current run, because a rising delta means the matrix is being bypassed.

Limitations and Who Should Not Use This

Free-tier access generally implies rate limits and no service-level agreement, so this pipeline should never block a release. If the model is unavailable, the job should fail softly and leave the draft branch untouched, because stale docs are better than fabricated docs. Teams that maintain regulated documentation or external API contracts should not use this workflow at all. A wrong sentence about a compliance requirement is an incident, and a model draft is not a substitute for a human author in that domain. The same applies to security notes, where the cost of a confident error is measured in incidents, not in doc debt.

The Boundary Is the Pipeline

The free tier removes the cost excuse, and the server removes the infrastructure excuse, but neither removes the ownership decision. Write the matrix first, schedule the job second, and treat the model as a drafting intern who never merges their own work. If you already run a docs job, add the coverage script before you invest in a better prompt. The measurement will show you where the boundary is breaking, and that is the only signal worth optimizing.

Top comments (0)