Choose Agent Compute by What You Can Falsify

The right default remains a free shared agent lane until you cannot falsify the assumption behind a generated diff. Paid APIs and self-hosted boxes are evidence tools, not prestige upgrades you purchase because a usage graph looks lonely. I treat compute choice as a paper-trail question: can another engineer reconstruct why the agent touched that file? If that reconstruction fails, the cheaper tokens were never the bargain, because you now pay in review time.

The failure mode is confident help you cannot replay

Coding agents rarely explode in a cinematic way; they assume a runner, a lockfile, or a “safe” rename, then ship a tidy patch. Have you opened a pull request and been unable to name the turn that invented a dependency? That gap is not a model-quality problem first. It is an evidence problem, and it decides whether free shared compute is still honest.

I am not arguing against free model access or a free server option for spikes, docs, and throwaway refactors. Those lanes are the correct default while the work is public-shaped and the blast radius stays inside a disposable worktree. The expensive mistake is switching vendors for vibes, or staying shared after private context and replay become the actual product. The rest of this piece is a fit test, not a brand ranking.

This is a proposed heuristic, not a production audit I ran against a customer fleet. If your compliance team already dictates log ownership, skip to the matrix and treat my first-person voice as a checklist, not testimony.

Kill criteria beat weighted scorecards

I do not want another weighted score that pretends architecture is a spreadsheet. You stay on the free lane while every kill criterion still holds, and you move only when one breaks. When it breaks, you choose a paid API versus self-hosting by who must own the logs, not by who published a prettier playground. Why would a larger context window repair an unnamed assumption?

Step 1. Write the one assumption the agent may keep

Write a single ugly sentence the agent may treat as true without extra evidence from the repo. Examples I actually use: “tests run with pytest from the repository root,” or “.env.example contains no production secret.” If you cannot write that sentence, you are not ready for any lane, free or paid. A bigger model will only fail faster, with better grammar.

# assumption_contract.txt  (commit this next to the agent prompt)
MAY_ASSUME: pytest is invoked from repo root via `python -m pytest`
MUST_NOT_ASSUME: package manager, cloud region, or secret locations
FALSIFY_WITH: python -m pytest -q tests/test_assumption_contract.py

Step 2. Attach an evidence pack to every file touch

An evidence pack is local text: prompt id, path, claimed intent, and the command that would falsify the change. I keep packs in agent_evidence.jsonl because git already diffs line-oriented logs. A free shared server is still fine while that file can live on your laptop without private source inside it. It stops being fine when the pack would contain customer data, credentials, or a proprietary tree.

{"prompt_id":"p12","path":"src/app.py","intent":"guard empty payload","falsify":"python -m pytest tests/test_app.py::test_empty_payload"}

Step 3. Name who must replay the session later

Ask who has to reconstruct the agent turn in six months, not this afternoon. If the answer is only you, a free shared workspace is enough for scaffolding. If the answer is an on-call rotation, security, or legal, you need a lane whose logs you can retain and delete on purpose. Paid APIs often help when you want vendor-side retention hooks. Self-hosting helps when the disk itself is the control.

Step 4. Split blast radius from “the model seemed dumb”

People upgrade models because a free agent looks lost, then watch a stronger model smash configuration with more confidence. Is the defect really quality, or missing isolation around network and secrets? If the agent can write only inside a worktree and cannot reach production, stay free. If it needs deploy keys, a long-lived shared workspace, or outbound network, move lanes before you tune prompts.

Fit matrix: free, paid API, or your own box

Use this table as a gate, not as marketing. The free column assumes a shared coding environment with free model access, which is a starting lane rather than a forever contract. Paid API means you send prompts to a billed endpoint you do not operate. Self-hosted means the runtime and disks are yours, including the boring backup work.

Signal you can observe	Stay on free models + free server	Move to a paid API	Move to self-hosted
Assumption contract fits in one sentence	Yes, and tests can falsify it locally	Contract spans several services	Contract includes air-gapped or regulated data
Evidence packs stay off private source	Packs are paths and commands only	You need vendor retention or export	Packs include source, secrets, or customer text
Replay audience	Solo, same day	Team that wants an API audit trail	Compliance or on-call that must own disk
Blast radius	Disposable worktree, no deploy keys	Managed isolation is enough	You must control network egress
Failure you actually hit	Weak first draft, easy to reject	You need SLA-backed availability	You cannot send the tree off-box

Notice what is missing: token folklore, model-name shopping, and “this is always cheaper.” Those are not fit criteria. If you cannot point at a row that broke, you are not graduating the lane; you are fidgeting.

Artifact: a proposed classifier you can unit-test

The following is labeled on purpose: it is a decision procedure, not a benchmark, and I have not attached fake latency numbers to it. Put it in paper_trail.py and refuse to change vendors until a test names the broken criterion. If a future you wants weights, add them in a fork, and keep the kill switches boolean so the matrix stays honest.

from dataclasses import dataclass
from enum import Enum


class Lane(str, Enum):
    FREE_SHARED = "free_shared"
    PAID_API = "paid_api"
    SELF_HOSTED = "self_hosted"


@dataclass(frozen=True)
class Fit:
    assumption_named: bool
    evidence_is_local_only: bool
    pack_includes_private_context: bool
    replay_audience: str  # solo | team | compliance
    needs_network_secrets: bool
    must_own_disk: bool
    airgapped: bool


class UnnamedAssumptionError(ValueError):
    """Raised when the agent is allowed to invent the problem statement."""


def classify(fit: Fit) -> Lane:
    if not fit.assumption_named:
        raise UnnamedAssumptionError(
            "Name the assumption before you pick compute. No lane fixes this."
        )
    if fit.airgapped or fit.must_own_disk or fit.pack_includes_private_context:
        return Lane.SELF_HOSTED
    if (
        fit.replay_audience in {"team", "compliance"}
        or fit.needs_network_secrets
        or not fit.evidence_is_local_only
    ):
        return Lane.PAID_API
    return Lane.FREE_SHARED

# test_paper_trail.py  — proposed tests, not a production suite I executed on live traffic
import pytest
from paper_trail import Fit, Lane, UnnamedAssumptionError, classify


def test_unnamed_assumption_blocks_every_lane():
    fit = Fit(False, True, False, "solo", False, False, False)
    with pytest.raises(UnnamedAssumptionError):
        classify(fit)


def test_solo_local_evidence_stays_free():
    fit = Fit(True, True, False, "solo", False, False, False)
    assert classify(fit) is Lane.FREE_SHARED


def test_team_replay_moves_to_paid_api():
    fit = Fit(True, True, False, "team", False, False, False)
    assert classify(fit) is Lane.PAID_API


def test_private_pack_forces_self_host():
    fit = Fit(True, False, True, "solo", False, False, False)
    assert classify(fit) is Lane.SELF_HOSTED

Run the tests before you open a billing console. The command is boring on purpose, because lane choice should be a failing assertion, not a slide.

python -m pytest test_paper_trail.py -q
git add assumption_contract.txt agent_evidence.jsonl paper_trail.py test_paper_trail.py

If you want a durable pointer without stuffing JSON into the commit message, hang the latest pack on a git note. That still does not replace a real audit log, and I would not pretend it does.

git notes --ref=agent-evidence add -F agent_evidence.jsonl HEAD
git notes --ref=agent-evidence show HEAD

Where a free model path actually participates

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I am using MonkeyCode here only as one example of a lane that offers free model access and a free server option, which matters if Step 1 and Step 2 still pass. I will not invent model names, quotas, hardware, duration, or permanence, because those claims go stale the moment a dashboard changes. Open source is relevant only insofar as you can inspect how a workspace starts, not as a substitute for the fit matrix.

Stay on that free lane while evidence packs remain local and the agent cannot reach production. Graduate when a row in the matrix breaks, then pick paid API versus self-hosting by log ownership. If you evaluate MonkeyCode against this test, run it on one private-repo scenario before you keep the shared box for anything that needs a paper trail.

Limitations, and who should not use this

This heuristic underfits teams that already have a platform mandate, a data-residency rule, or a procurement list that forbids shared workspaces. It also underfits agents that deploy, migrate data, or touch production, because blast radius is then a safety problem rather than a billing problem. Do not use the classifier as proof that free compute is “safe enough” for regulated source.

I also would not use it as a performance bake-off. There are no tokens-per-second numbers here, and a passing pytest file does not mean a shared server will retain your session the way a disk you own will. If your real constraint is latency, offline models, or GPU placement, this article is the wrong artifact; measure those things directly. If your real constraint is an agent that assumes pytest while the repo uses a Makefile, start with Step 1 and stay there.

The conclusion does not change when a new agent glossary starts circulating. Keep the free shared lane until you cannot falsify the diff, then buy evidence, not vibes. Paid and self-hosted are how you own the paper trail. They are not how you look busy in a usage dashboard.