A Token-Budgeted Pipeline for Documenting and Testing a Legacy Repo

A legacy repo without tests or docs is a candidate for automation, not a rewrite. A token-aware pipeline can generate Markdown documentation and pytest skeletons in one pass, and a free managed server makes the math work for small projects. The script below walks a repository, estimates token cost per file, and calls an OpenAI-compatible endpoint with a hard budget — so the bill never surprises you.

MonkeyCode is an open-source project that provides a free server option and a 10M-token allowance for model access. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The pipeline described here works with any OpenAI-compatible endpoint, but the free allowance makes it practical to run against a whole repo.

Why Docs and Test Skeletons Are the Right Workload for a Free Tier

Documentation and test scaffolding are repetitive, low-risk, and bounded in size. Each file produces a small output, so the token cost per unit of value is low. That fits a free tier perfectly: the bottleneck is the allowance, not the quality of the result.

The alternative — writing docs and tests by hand — consumes hours that most teams do not have. The alternative to automation is not a better outcome; it is no outcome. A pipeline that generates a first draft is strictly better than an empty docs folder.

Step 1: Estimate Tokens Before You Spend Them

The first rule of a token budget is to measure before you spend. A rough estimate is len(text) / 4, which works well enough for English and code. Save this as a function:

def estimate_tokens(text: str) -> int:
    return max(1, len(text) // 4)

This is not precise, but it is consistent. A consistent estimate is more useful than an exact one, because the goal is to compare files against a budget, not to bill a customer.

Step 2: Walk the Repo and Batch Files

The pipeline needs a list of Python files, sorted by size, with small files first. That ordering maximizes the number of files processed before the budget runs out.

import os
from pathlib import Path

SKIP_DIRS = {".git", "node_modules", "venv", "__pycache__", ".venv"}

def iter_python_files(root: Path):
    for dirpath, dirnames, filenames in os.walk(root):
        dirnames[:] = [d for d in dirnames if d not in SKIP_DIRS]
        for fname in filenames:
            if fname.endswith(".py"):
                yield Path(dirpath) / fname

def files_by_size(root: Path):
    files = list(iter_python_files(root))
    return sorted(files, key=lambda p: p.stat().st_size)

The sort matters. A 500-line module consumes more tokens than a 20-line helper, so processing small files first gives you more complete coverage when the budget is tight.

Step 3: Generate Docs and Tests With Retry Logic

The core loop calls the model once per file, with a prompt that asks for both documentation and a pytest skeleton. The response is written to two locations: docs/<relative_path>.md and tests/<relative_path>_test.py. Retry logic handles transient failures without burning the whole budget.

import os
import time
import requests
from pathlib import Path

API_URL = os.environ.get("MC_API_URL", "https://api.monkeycode.example/v1/chat/completions")
API_KEY = os.environ.get("MC_API_KEY", "")
MODEL = os.environ.get("MC_MODEL", "default")
MAX_TOKENS_PER_FILE = int(os.environ.get("MAX_TOKENS_PER_FILE", "1500"))
TOTAL_BUDGET = int(os.environ.get("TOTAL_BUDGET", "100000"))

def call_model(prompt: str, max_tokens: int) -> str:
    payload = {
        "model": MODEL,
        "messages": [{"role": "user", "content": prompt}],
        "max_tokens": max_tokens,
        "temperature": 0.2,
    }
    headers = {"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"}
    for attempt in range(3):
        try:
            resp = requests.post(API_URL, json=payload, headers=headers, timeout=60)
            resp.raise_for_status()
            return resp.json()["choices"][0]["message"]["content"]
        except Exception as exc:
            print(f"  attempt {attempt + 1} failed: {exc}")
            time.sleep(2 ** attempt)
    raise RuntimeError(f"failed for prompt: {prompt[:80]}...")

def generate_for_file(path: Path, root: Path):
    code = path.read_text(encoding="utf-8", errors="replace")
    prompt = (
        "You are helping document a legacy Python repo. "
        "Return exactly two sections separated by '---':\n"
        "1. Markdown documentation for the module: purpose, public functions, usage example.\n"
        "2. A pytest skeleton with basic test functions for the main public functions.\n\n"
        f"```
{% endraw %}
python\n{code}\n
{% raw %}
```"
    )
    response = call_model(prompt, MAX_TOKENS_PER_FILE)
    doc_part, _, test_part = response.partition("---")
    rel = path.relative_to(root)
    doc_path = root / "docs" / rel.with_suffix(".md")
    test_path = root / "tests" / f"{rel.with_suffix('').as_posix().replace('/', '_')}_test.py"
    doc_path.parent.mkdir(parents=True, exist_ok=True)
    test_path.parent.mkdir(parents=True, exist_ok=True)
    doc_path.write_text(doc_part.strip() + "\n", encoding="utf-8")
    test_path.write_text(test_part.strip() + "\n", encoding="utf-8")
    print(f"  wrote {doc_path} and {test_path}")

def main(root: Path):
    for path in files_by_size(root):
        code = path.read_text(encoding="utf-8", errors="replace")
        est = estimate_tokens(code)
        if est + MAX_TOKENS_PER_FILE > TOTAL_BUDGET:
            print(f"  skip {path}: est {est} tokens exceeds remaining budget")
            continue
        print(f"  process {path}: est {est} tokens")
        generate_for_file(path, root)

if __name__ == "__main__":
    import sys
    root = Path(sys.argv[1] if len(sys.argv) > 1 else ".")
    main(root)

The prompt is deliberately strict about the output format. Without a separator, splitting documentation from tests is guesswork. The partition call handles a missing separator gracefully by leaving the test part empty, which the validation step catches later.

Step 4: Validate the Output Locally

Generated tests are a starting point, not a guarantee. Run pytest to see which ones pass and which ones are nonsense:

python -m pytest tests/ -q

Expect failures. The value is in the skeleton: function names, import paths, and edge-case hints. Fixing a failing test is faster than writing one from scratch.

Who Should Not Use This Pipeline

Teams with strict data-residency rules should not route source code through any managed server, free or paid. Large codebases with hundreds of files will exhaust a 10M-token allowance quickly; the pipeline is designed for small to medium repos. Projects that require precise coverage metrics need human-written tests, not AI-generated skeletons.

Limitations

The 10M-token allowance and the free server option are availability claims that can change; verify current terms before committing a workflow to them. The token estimate is rough, so the actual consumption may differ. Generated docs and tests must be reviewed; they are not a substitute for a human maintainer. The script has no caching, so re-running it on the same repo spends tokens again. Add a file-based cache if you plan to iterate.

Run the pipeline on a single directory first. The output files and the token report will tell you whether the full repo is worth the spend — and whether the free tier is the right fit for your workflow.

MonkeyCode provides free models that can run this workflow.