Nightly Setup Drift: A Free Server and Free Models That Keep Your README Honest

Your README is already wrong. Not today, maybe. But a dependency bump changes everything. The setup command stops working. Nobody notices until a new hire burns a morning. This article builds a nightly drift check. It keeps setup docs honest. The bot extracts commands from Markdown. It drafts a verification plan with free models. It runs that plan on a free server. Then it reports what broke. You stay the final reviewer.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode's free models generate the plan. MonkeyCode's free server option runs the job. No GPU, no cloud account, no extra cost to try it. The approach is small and reproducible. It fits one repo or ten.

Setup Docs Are the Highest-Risk Artifact

Setup instructions rot faster than code. Code changes are covered by tests. Documentation changes have no safety net. One version bump breaks a command. The README still says the old thing. Nobody knows until someone follows it.

Teams rarely test setup steps. They test features, not installation. Fresh environments are expensive to spin up. So the docs stay stale. The result is a hidden tax. Every onboarding pays it. Every contractor pays it. The fix is a cheap, automated truth check.

What the Nightly Check Produces

The check runs in four stages. Each stage has a clear output. The output feeds the next stage. You control the final decision. Here is the whole pipeline:

1. Extract every shell command from the README.
2. Plan a safe verification sequence with a free model.
3. Execute the plan on a free server.
4. Report failures to your issue tracker.

The pipeline runs on a schedule. Nightly is a good default. Weekly works for slowly changing docs. The schedule is your call. The artifact is a reproducible script. You can run it locally too.

Step 1: Extract Commands From Markdown

The extractor reads fenced code blocks. It looks for sh or bash tags. It ignores commands that need input. Those get flagged for manual review. Here is the essential script:

# tools/extract_commands.py
import re
import sys
from pathlib import Path

def extract_code_blocks(path):
    text = Path(path).read_text()
    pattern = re.compile(r"```

(?:sh|bash)\n(.*?)

```", re.DOTALL)
    return [block.strip() for block in pattern.findall(text)]

if __name__ == "__main__":
    for doc in sys.argv[1:]:
        for i, block in enumerate(extract_code_blocks(doc), 1):
            print(f"### {doc} block {i}")
            print(block)

This script is deliberately small. It gives you a plain text transcript. The transcript becomes the model input. No hidden parsing logic. No fragile AST.

Step 2: Draft a Verification Plan With Free Models

The transcript alone is not enough. Some commands need ordering. Some commands are dangerous to rerun. A free model can propose a safe sequence. You then approve it. This draft is a proposal, not a fact. Treat it that way.

MonkeyCode's free models turn the transcript into a plan. The prompt asks for three things: order, isolation, and cleanup. Here is the prompt template:

You receive a setup transcript from a README.
Propose a verification plan.
List commands in a safe order.
Wrap every command in a fresh container.
Add a cleanup step for generated files.
Return only the plan as shell code.

This is a prompt, not proof. The model output is a draft. You review the plan once. Then you hardcode it if you prefer determinism. The following pseudocode shows the integration:

# tools/plan_draft.py (pseudocode, not executed)
import os
import requests

def draft_plan(transcript: str) -> str:
    payload = {
        "model": "free-model-endpoint",
        "prompt": PLAN_PROMPT + transcript,
    }
    response = requests.post(
        os.environ["MONKEYCODE_ENDPOINT"],
        json=payload,
    )
    return response.json()["plan"]

The endpoint and key stay in environment variables. They never enter source control. This step is where the free model access matters. Planning is cheap and imperfect. Perfect execution comes next.

Step 3: Execute the Plan on the Free Server

The plan runs on a free server. The server gives you a disposable environment. It does not touch your laptop. It does not require local Docker. It exists only for the job. The simplest runner is a cron job. Here is a runner script:

#!/usr/bin/env bash
# tools/nightly_setup_check.sh
set -euo pipefail

DOC="$1"
TRANSCRIPT=$(python tools/extract_commands.py "$DOC")
PLAN=$(python tools/draft_plan.py "$TRANSCRIPT")

# Execute each line, stop on failure, capture exit code.
while IFS= read -r cmd; do
  if ! bash -c "$cmd"; then
    echo "FAILED: $cmd"
    exit 1
  fi
done <<< "$PLAN"

echo "PASS: setup steps executed cleanly"

This script is the core gate. It runs each planned command. It stops at the first failure. It prints the exact failing command. The free server executes the script nightly. You wake up to a report, not a surprise.

You can also wire the same script into a scheduled GitHub Action. The action gives you logs and history. Here is a minimal schedule:

name: nightly-setup-check
on:
  schedule:
    - cron: "0 3 * * *"
jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run setup check
        run: bash tools/nightly_setup_check.sh README.md

The report is plain text. Failure makes the job red. Success makes it green. That contrast is the whole signal. No dashboards, no alerts, no noise.

Which Docs Earn a Nightly Check

Not every README deserves this treatment. The decision depends on risk and churn. Use this table as a starting point:

Documentation type	Change frequency	Cost of failure	Enable nightly check?
Setup guide for contributors	High	High	Yes
API quickstart (no auth)	Medium	Medium	Yes
Deployment runbook	Low	Critical	Maybe, with human gate
Internal wiki notes	High	Low	No
Tutorial with manual UI steps	Low	Low	No

The rule is simple. The check pays for itself when failure is expensive. It annoys everyone when the doc changes weekly for trivial reasons. Start with one high-risk guide. Add more after the pattern is stable.

Limitations: What This Bot Cannot Catch

This approach has hard limits. It cannot exercise commands that need a human. Interactive prompts, GUI tools, and browser flows fail by definition. It cannot verify visual output. A command can succeed and still be wrong. It cannot handle secrets safely. Do not feed real credentials into a free server. Keep those steps out of the transcript.

The free model's plan is a draft. It can propose a wrong order. It can misunderstand a transcript. That is why a human reviews the plan once. The free server has no strong SLA. Treat it as a best-effort runner. Do not use it for production releases.

Who should skip this workflow? Teams with unstable shell environments. Projects with heavy GUI installers. Repos where setup docs are already tested by real CI. The bot adds little there.

The Bottom Line

Setup documentation is a truth problem. Your README claims the setup works. The nightly check proves it. The extraction script is fifteen lines. The runner script is twelve. The free model drafts the plan. The free server executes it. The only expensive resource is your review, spent on real failures.

Start with one README. Run the check once by hand. Let the first report teach you where the docs lie.