Weekend Build Log: Stop Agent Plans That Invent Files

You sit down Saturday with too much ambition. You want an agent that reviews pull requests. Then the plan cites a helper you never wrote.

That miss is the whole weekend risk. You do not need a larger model today. You need a smaller demo and a path check.

The scene you already know

You paste a repo tree into a chat window. You ask for a short implementation plan. The plan names three files that do not exist.

You feel tempted to let the agent create them. Invented files become invented functions by lunch. Your Sunday then pays for Saturday's optimism.

What this weekend actually ships

You ship one working checker, not a product. The checker compares claimed paths with the real tree. A missing path stops the plan immediately.

You also ship a one-page scope card. The card lists the demo, the skips, and the stop rule. That pair is the complete weekend deliverable.

1. Pick one repo you already own today.
2. Freeze a two-hour demo target right now.
3. Ban new hosted services for this weekend.
4. Require a path ledger before any code edits.

Write the scope card first

Create the WEEKEND_SCOPE.md file before you send any prompt. Keep the scope card ugly and short. Pretty writing steals the only free morning.

# Weekend scope — assumption trap

Demo: print missing paths from an agent plan.
Success: exit 1 when any claimed file is absent.
Skip: PR comments, web UI, extra database, auth.
Stop: if the checker needs a network call.

Read the finished card out loud once. Cut any step that needs a vendor account. Cut any step that starts a second process.

Build an assumption ledger

The ledger is a JSON object, not a paragraph. You extract every path the plan claims. You refuse to trust free-form prose here.

Ask the model for JSON output only. Reject extra commentary in the same reply. Save the object as claimed_paths.json immediately.

{
  "task": "add a dry-run lint command",
  "claimed_paths": [
    "src/lint.ts",
    "scripts/dry_run.sh",
    "docs/LINT.md"
  ],
  "new_paths_allowed": [
    "scripts/dry_run.sh"
  ]
}

new_paths_allowed is the only door for new files. Every other claimed path must already exist. That single field is your scope cut.

A checker you can run

Treat this script as a proposal, not production control. Save the checker script as tools/check_claimed_paths.py locally. Run it before you accept any generated patch.

#!/usr/bin/env python3
"""Fail if an agent plan cites missing paths."""

from __future__ import annotations

import json
import sys
from pathlib import Path


def load_ledger(path: Path) -> dict:
    data = json.loads(path.read_text(encoding="utf-8"))
    if "claimed_paths" not in data:
        raise ValueError("claimed_paths missing")
    if "new_paths_allowed" not in data:
        raise ValueError("new_paths_allowed missing")
    return data


def check(repo: Path, ledger: dict) -> list[str]:
    allowed = set(ledger["new_paths_allowed"])
    missing: list[str] = []
    for raw in ledger["claimed_paths"]:
        rel = Path(raw)
        if rel.is_absolute() or ".." in rel.parts:
            missing.append(f"illegal path: {raw}")
            continue
        target = repo / rel
        if target.exists() or raw in allowed:
            continue
        missing.append(raw)
    return missing


def main() -> int:
    if len(sys.argv) != 3:
        print(
            "usage: check_claimed_paths.py REPO LEDGER.json",
            file=sys.stderr,
        )
        return 2
    repo = Path(sys.argv[1]).resolve()
    ledger = load_ledger(Path(sys.argv[2]))
    missing = check(repo, ledger)
    if not missing:
        print("ok: all claimed paths are present or allowed")
        return 0
    print("blocked: agent cited missing paths")
    for item in missing:
        print(f"  - {item}")
    return 1


if __name__ == "__main__":
    raise SystemExit(main())

The script stays boring on purpose today. Boring checks survive a tired Saturday brain. Fancy retry loops do not survive that.

Prove it with a fixture

Do not skip tests because the file is small. A two-path fixture catches the weekend bug. Use a throwaway directory, not your main clone.

mkdir -p /tmp/assumption-demo/src
printf 'export function lint() { return 0 }\n' > /tmp/assumption-demo/src/lint.ts
cat > /tmp/claimed_paths.json <<'EOF'
{
  "task": "add a dry-run lint command",
  "claimed_paths": ["src/lint.ts", "scripts/dry_run.sh", "docs/LINT.md"],
  "new_paths_allowed": ["scripts/dry_run.sh"]
}
EOF
python3 tools/check_claimed_paths.py /tmp/assumption-demo /tmp/claimed_paths.json
echo "exit=$?"

You should see docs/LINT.md blocked in the output. src/lint.ts should pass because it exists. scripts/dry_run.sh should pass because you allowed it.

If the exit code is not 1, stop immediately. Fix the checker before you touch the agent. A silent miss is worse than no checker.

Add one more illegal-path case after that. Point the ledger at ../.env and run again. The script must print illegal path and refuse the plan.

cat > /tmp/claimed_escape.json <<'EOF'
{
  "task": "should hard-fail",
  "claimed_paths": ["../.env"],
  "new_paths_allowed": []
}
EOF
python3 tools/check_claimed_paths.py /tmp/assumption-demo /tmp/claimed_escape.json
echo "exit=$?"

Where a free draft box still helps

You still need a draft of the JSON ledger. You also need a box that runs Python. Keep both steps cheap and local when you can.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode provides free model access and a free server option. You can draft the ledger there and run this checker on that server. Drop the product and the same loop still runs on your laptop.

Keep secrets off any hosted prompt you use. Copy a throwaway tree without env files. Never put keys inside claimed_paths.json at all.

Numbered Saturday sequence

Follow this exact order on Saturday morning. Do not rearrange steps under time pressure.

1. Clone a throwaway copy of your application repo.
2. Write WEEKEND_SCOPE.md in ten focused minutes only.
3. Prompt for JSON paths, not for code.
4. Save claimed_paths.json without any hand-edited fields.
5. Then execute python3 tools/check_claimed_paths.py against your claimed_paths.json file.
6. If it fails, revise the plan, not the tree.
7. Only then allow writes in the allowed list.
8. Record every skipped idea in the same markdown file.

Step six is the whole weekend lesson. You change the plan when paths are missing. You do not invent files to make the plan true.

What you skip on purpose

You skip a web dashboard this weekend entirely. You skip storing ledgers in a Postgres instance. You skip Slack alerts and auto-created files.

You also skip making the agent look smarter. Smarter prompts still invent missing paths anyway. The filesystem does not invent those paths.

Skipped:
- GitHub Action matrix
- Embedding index of the repo
- Comment bot on pull requests
- Retry loop that creates missing files

If a skipped item sneaks back, stop the weekend. Ship the checker in its current shape. Do not open a second feature branch.

Failure analysis you can reuse

When the checker fails, classify the miss first. Use the table instead of arguing in chat. Speed matters more than a clever rebuttal.

Miss type	Example	Weekend action
Invented helper	src/utils/secret.ts	Delete from plan
Wrong extension	app.js vs app.ts	Correct the ledger
Case mismatch	Readme.md	Normalize on disk
Allowed new file	scripts/dry_run.sh	Keep if scope says so
Path escape	../.env	Hard fail, do not run

Print the table next to your terminal window. Decision speed matters on a short Saturday. You have hours, not a full sprint.

Honest limits

This does not prove the plan is correct. It only proves cited files exist or were allowed. A present file can still be the wrong file.

The script does not parse generated patches yet. It does not verify imports or types. It will not stop a bad refactor inside a real file.

Symlinks can fool a simple exists check. Nested parent segments can still look nasty. Treat the checker as a seatbelt, not a vault.

Who should not use this

Do not use this as a production gate yet. Do not run it on working trees that hold secrets. Do not use it if you cannot read the JSON.

Skip it if your task is purely documentation. Skip it if a code owner bot already blocks unknown paths. Skip it if you need a networked multi-agent demo.

If you cannot cut scope, do not start. An uncut agent demo turns Saturday into Monday. Protect the calendar before you protect the prompt.

Close the log

Your working demo is a red exit code. That signal is enough for this weekend. You learned where the plan diverges from disk.

Keep the scope card next to the script. Reuse both files on the next free Saturday. Change only the allowed paths list next time.