DEV Community: Riley Li

Inventory What You Cannot Pin Before You Choose Free Agent Compute

Riley Li — Mon, 21 Sep 2026 13:29:47 +0000

Free shared compute is a good lab bench when overnight drift is cheap to absorb. A pinned box is the better home when next Tuesday's agent run must still match today's trace. I start from that split each time, rather than from a price tag or an idle GPU. Can you name every moving part this agent will still need next week?

A host chosen from an empty queue often fails when a credential, a model identity, or a disk layout moves. I do not treat "is free cheaper than a box I run" as the real question sitting in front of the loop. I ask which parts of the agent loop refuse to be pinned at all, then I choose a host. That question also decides when a slick Friday demo is still only a lab accident.

Overnight drift is the decision, not the invoice

Agent loops are not single HTTP handlers sitting behind a frozen OpenAPI contract you can file away. They stitch a model, a tool surface, a secret boundary, a timeout owner, and a trace sink together in one session. If any of those pieces floats overnight, Friday's success story is a mood rather than evidence you can hand a reviewer. Have you tried replaying last week's failing tool call without guessing the model snapshot that answered?

I use a drift inventory because a cost model lies when the environment itself is the moving target under the agent. Free shared capacity can still be the right first room while the tool surface is changing twice a day. It becomes the wrong room when a reviewer needs a reconstructable failure and was never in the original chat. This whole article is a proposed workflow, not a production study with logos or latency medals.

Build the inventory before you request a host

I write seven fields down before I decide where the loop is allowed to run at all. Each field is pinned, exportable, or floating, and I refuse to mix those three words casually in a standup. Floating is allowed during exploration, and I say that out loud so nobody promotes a mood into a release. Floating is not allowed after a run is treated as a decision record that other people must trust.

The seven fields

Model identity. I ask whether I can name a pin, or whether I am taking whatever answers today.
Runtime and dependencies. I ask whether the same tool runner and lockfile can be reinstalled without folklore.
Tool origin. I ask whether tool calls execute on a network I can describe to another engineer.
Secret boundary. I ask whether API keys ever enter a host that I cannot image or wipe.
Filesystem. I ask whether working trees, caches, and scratch files are objects I can snapshot later.
Timeout and retry owner. I ask who declares a hung tool dead, me or a shared scheduler.
Trace export. I ask whether I leave with JSONL traces and prompts, or only a dashboard screenshot.

If four or more fields are floating, I stay on a free lab bench and I refuse promotion. If four or more fields must be pinned, I want a box I operate or paid isolation I can describe. Mixed scores mean a two-stage path: explore while floating is cheap, then promote only the pinned slice. Does that feel heavier than "just run it somewhere and see"? Agents hide real cost inside the parts you cannot even name.

Run a scorecard instead of arguing in chat

I keep the inventory in YAML so a reviewer can diff it like any other contract checked into git. The helper script below is a proposed checklist, and it does not call a vendor API or invent a benchmark. I also do not treat the integer score as a latency number or as a security control that replaces threat modeling.

# agent_env.yaml — proposed inventory, fill honestly
agent: invoice-triage-loop
stage: explore  # explore | promote
fields:
  model_identity: floating    # pinned | exportable | floating
  runtime_lockfile: pinned
  tool_origin: floating
  secret_boundary: floating
  filesystem: exportable
  timeout_owner: floating
  trace_export: exportable
notes:
  secrets: "provider tokens currently injected by the shared runner"
  traces: "we can download JSONL, not the raw sandbox disk"

#!/usr/bin/env python3
"""drift_score.py — proposed scorecard, not measured production data."""

from __future__ import annotations

import sys
from pathlib import Path

try:
    import yaml
except ImportError:
    print("pip install pyyaml", file=sys.stderr)
    raise

WEIGHTS = {
    "pinned": 0,
    "exportable": 1,
    "floating": 2,
}

PROMOTE_BLOCKERS = {
    "secret_boundary": "floating",
    "model_identity": "floating",
    "trace_export": "floating",
}


def load_inventory(path: Path) -> dict:
    data = yaml.safe_load(path.read_text())
    if "fields" not in data:
        raise ValueError("inventory needs a fields map")
    return data


def score(fields: dict[str, str]) -> tuple[int, list[str]]:
    total = 0
    blockers = []
    for name, state in fields.items():
        if state not in WEIGHTS:
            raise ValueError(f"{name}: unknown state {state!r}")
        total += WEIGHTS[state]
        if PROMOTE_BLOCKERS.get(name) == state:
            blockers.append(name)
    return total, blockers


def recommend(stage: str, total: int, blockers: list[str]) -> str:
    if stage == "promote" and blockers:
        joined = ", ".join(blockers)
        return f"stay off promotion: pin {joined} before leaving the lab bench"
    if total <= 4:
        return "free shared compute is a fit while drift stays cheap"
    if total <= 8:
        return "split the loop: explore free, pin the promote slice elsewhere"
    return "run the loop on a box you can pin and export"


def main() -> None:
    path = Path(sys.argv[1] if len(sys.argv) > 1 else "agent_env.yaml")
    data = load_inventory(path)
    total, blockers = score(data["fields"])
    print(f"agent={data.get('agent')}")
    print(f"stage={data.get('stage')}")
    print(f"drift_score={total} (0=pinned, higher=more overnight risk)")
    print(f"promote_blockers={blockers or 'none'}")
    print(recommend(data.get("stage", "explore"), total, blockers))


if __name__ == "__main__":
    main()

python3 -m pip install pyyaml
python3 drift_score.py agent_env.yaml

I treat a drift score at or under four as permission to remain on free shared capacity for now. I treat any floating secret_boundary as a hard promotion block, even when the rest of the card looks tidy. Would you ship a loop whose keys live on a host you cannot image after a bad run?

Where a free lab bench still wins the comparison

Exploration wants cheap retries, throwaway sandboxes, and models you can try without opening a purchase order first. That is the honest job of free model access paired with a free server option during the explore stage. I use that room to fill the YAML honestly, not to pretend the YAML is already pinned for a reviewer who missed the chat.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source project with free model access and a free server option that maps onto this explore stage. I mention it because the inventory needs a lab bench, not because a free host pins production secrets for you. If most fields are still floating and the stage is still explore, that lab is a rational place to generate traces you can actually export. I do not claim model names, token quotas, hardware SKUs, or how long any free tier remains available.

Can you live with that uncertainty while the loop is still changing twice a day on purpose? If you can, free compute is doing exactly the job I want from an early lab. If you cannot, you already know you need a pin, and shopping for spare capacity will not create one.

A labeled walkthrough I have not billed as a case study

This walkthrough is a proposed example with fake ticket names, and I am not attaching measured production timings to it. Imagine an invoice-triage agent that calls a PDF parser, a vendor HTTP API, and a local sqlite cache. During week one I let model identity float, I inject the vendor token through the shared runner, and I download JSONL after each session. The scorecard comes back high, blockers include secret_boundary and model_identity, and the only honest recommendation is to remain in explore.

During week two I lock a requirements file, I move the vendor token into a store I operate, and I pin tool origin to a container I build. Floating fields drop on purpose, and only then do I copy the same fixtures onto isolated compute I can describe. Notice what did not need to change between those weeks: the prompts, the expected tool names, and the JSONL schema the lab already exported. The host changed because the inventory changed, not because a coupon expired or a queue looked lonely.

# proposed local pin after the lab stage — replace with your runtime
cp agent_env.yaml pinned_env.yaml
# edit model_identity, secret_boundary, and tool_origin to pinned
python3 drift_score.py pinned_env.yaml

If the promote command still prints blockers, I do not "just try prod" to soothe a demo calendar that wants a green box. I fix the field that refused to be pinned, then I run the scorecard again before anyone changes the stage. Why would a green demo override a floating secret boundary that I already wrote down in YAML?

Explicit tradeoffs I put on the same card

Free shared models and a free server

Free shared capacity fits exploration, prompt drafts, throwaway tools, and traces that leave with me in JSONL. The tradeoff is that model identity and noisy-neighbor timing may float overnight without anyone calling a meeting. I refuse this option when someone must reconstruct a failure for a reviewer who never sat in the original session.

Self-hosted or paid isolated compute

A box I pin fits locked runtimes, secret stores I own, and tool networks I can draw on a whiteboard. The tradeoff is that I now own disk, upgrades, and the pager for hung tool calls after midnight. I refuse this option for loops that still change their tool surface twice a day and cannot sit still long enough to pin.

A split path with portable fixtures

A split path fits most agents that start messy and later need a reconstructable record instead of a chat screenshot. The tradeoff is that fixtures must stay portable, or the split quietly becomes two incompatible products with folklore in the middle. I refuse a split when the lab cannot export traces, because then the isolated box is pinning theater rather than evidence.

I want fixtures to stay boring: JSONL prompts, expected tool names, and timeout budgets that survive a host change without a war story. If the lab cannot export that bundle, free compute is teaching a demo habit I do not trust on Monday. If the isolated box cannot import that bundle, I bought a pin I cannot test when the next incident shows up.

Who should not use this approach

Do not use this inventory as a substitute for threat modeling, license review, or a real secret scanner in CI. Do not use the numeric score as a performance benchmark, because the weights are conversation aids I chose in this draft. Do not run untrusted agent code on a free shared host and then point at YAML as if it were a control plane.

Skip the method if your so-called agent is a stateless function with no tools, no secrets, and no session to reconstruct. A normal unit test is enough in that boring case, and the inventory would only add ceremony around a function. Skip it if you cannot tell the truth inside the YAML, because a lying inventory is worse than an empty file.

I also skip free shared compute when the loop must touch production data, even when the meeting is labeled as a harmless demo for leadership. Drift is not the only risk on the table when an agent can call tools. Access is a different risk, and this scorecard does not pretend to cover production data paths.

Monday morning steps

I write agent_env.yaml before I request a host, including the stage field set to explore.
I run drift_score.py and I read the promote blockers out loud before anyone books a demo.
I generate traces on a free lab bench only while that stage value remains explore.
I export JSONL, lockfiles, and the YAML itself so the next host does not inherit folklore.
I pin secrets, tool origin, and model identity on a box I control before I change the stage.
I re-run the scorecard and I refuse promotion whenever any blocker is still marked floating.

Free versus self-hosted is a function of what you cannot pin, not a personality test about cloud loyalty. I would rather keep an agent in the lab for another week than promote a loop whose environment I cannot name. If you cannot name what will change overnight in your loop, you are not ready to pick a host.

Budget the Agent Loop Before You Choose Shared or Isolated Compute

Riley Li — Sun, 20 Sep 2026 11:29:34 +0000

The cheap lane is the wrong lane whenever a stalled model call can retrigger a write. I treat timeout budget as the first filter, not list price or leftover GPU hours. If you cannot name your p95 stall and your idempotency story, you are not choosing compute. You are gambling on retries.

Have you timed the second tool call, or only the happy-path demo? Shared inference looks generous until a slow token stream collides with a client retry. I wrote this as a decision guide for that collision, not as another invoice spreadsheet. The artifact below is a proposed worksheet you can run locally with your own numbers.

Start with the stall, not the invoice

Agent loops are not single HTTP calls, even when the SDK pretends they are. One user prompt can fan into planning, tool JSON, a side effect, and a second model hop. If hop two waits on hop one, tail latency stops being a quality-of-service footnote. It becomes a correctness bug.

I keep seeing teams promote a playground agent because the median answer looked fine on a quiet afternoon. Did the write path survive an eight second pause? Did the webhook client retry while the first attempt was still committing? Those questions decide the host. Price does not.

This article stays adjacent to the current noise about tests that no longer bound model behavior. I am not chasing a headline about models outgrowing benchmarks. I am asking a smaller, meaner question: can your loop absorb variance without double-sending money, mail, or deploys?

Why agent loops punish shared inference

Shared free inference is usually fine for read-only drafting. It gets hostile when three properties stack. First, the model call has a fat tail you do not control. Second, your client retries on timeout because that is what HTTP libraries do. Third, a tool actually mutates something outside the chat transcript.

Put those together and you do not have a slow chatbot. You have an amplifier. A single stalled generation can mint two tickets, two refunds, or two terraform apply runs. Would you accept that amplifier on a noisy neighbor queue you cannot drain?

I score that stack before I score dollars. Isolated or self-hosted compute does not magically make models smarter. It gives you a place to set hard deadlines, pin a revision, and kill a worker without begging a shared scheduler. That is the trade I actually care about.

Three lanes, one timeout question

I use three lanes only. Lane A is a free shared model plus a free shared server, useful as a scratch harness. Lane B is paid isolated inference with a contract you can page. Lane C is a box you operate, including a local runtime or a rented GPU you can ssh into. MonkeyCode sits in lane A for this worksheet because it currently offers free model access and a free server option.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I do not treat lane A as production just because the form is empty. I treat it as a measurement bench. Can I replay a failing prompt? Can I export the trace? Can I bound the wait so a tool cannot fire twice? If the answers stay no, I refuse to promote the agent, even when the output reads fluent.

Lane B and lane C buy different kinds of control, not the same kind. Isolated paid compute usually buys queue isolation and a support path. Self-hosted compute buys process death, disk, and network policy. Which control do you actually need when the model stalls? That is the comparison, not a loyalty test.

A numbered measurement workflow

Run this before you bind secrets or webhooks to any lane. Label every number as an input you measured, not a vendor promise. I want the worksheet to survive if you never mention a product name again.

Draw the loop on paper, including every tool that can create, update, or charge. If a step only reads, mark it R. If it writes, mark it W and name the idempotency key. No key means no shared lane.
Capture three timings on the same prompt: first token, full completion, and tool round trip. Record p50 and a crude p95 from at least twenty local runs. Do not invent a benchmark you did not run.
Write the client timeout next to the p95. If timeout is lower than p95, you already own a retry bug. Raise the deadline or make the write idempotent before you shop for hosts.
Inject a stall. Sleep in a fake tool, or drop packets with a local proxy. Watch whether the orchestrator double-fires the W step. That single observation outranks any pricing page.
Score the three lanes with the matrix below. Promote only when the write path stays single-shot under the stall you actually injected.

Need a command-shaped starting point for step four? Keep the stall on your laptop.

python3 loop_budget.py \
  --p95-ms 4200 \
  --client-timeout-ms 3000 \
  --write-steps 2 \
  --idempotent false \
  --can-kill-worker false \
  --need-trace-export true

If that command recommends isolation, believe the timeout math. Do not argue with a marketing page until the retry story is boring.

Artifact: budget calculator and decision matrix

The script is a proposal, not a load test of any vendor. Feed your measurements. It prints a lane and the rule that fired. Change the inputs when your agent changes shape.

#!/usr/bin/env python3
"""loop_budget.py — proposed scoring sheet, not a benchmark of any host."""
from __future__ import annotations

import argparse
from dataclasses import dataclass


@dataclass
class LoopFacts:
    p95_ms: int
    client_timeout_ms: int
    write_steps: int
    idempotent: bool
    can_kill_worker: bool
    need_trace_export: bool


def retry_overlap(facts: LoopFacts) -> bool:
    return facts.client_timeout_ms < facts.p95_ms and facts.write_steps > 0


def recommend(facts: LoopFacts) -> tuple[str, str]:
    if facts.write_steps == 0 and not facts.need_trace_export:
        return (
            "lane-A-shared-free",
            "Read-only loop with no trace mandate; shared inference is a fair scratch pad.",
        )
    if retry_overlap(facts) and not facts.idempotent:
        return (
            "lane-C-self-hosted",
            "Timeout sits under p95 and writes are not idempotent; you need a killable worker.",
        )
    if facts.write_steps > 0 and not facts.idempotent:
        return (
            "lane-B-isolated-paid",
            "Writes exist without keys; isolate the queue before a neighbor stall doubles a side effect.",
        )
    if facts.need_trace_export and not facts.can_kill_worker:
        return (
            "lane-B-isolated-paid",
            "You need exportable traces more than process death; buy a contract you can audit.",
        )
    if facts.can_kill_worker and retry_overlap(facts):
        return (
            "lane-C-self-hosted",
            "You already require worker death during overlap; keep the runtime on a box you control.",
        )
    return (
        "lane-A-shared-free",
        "Writes look idempotent and timeout covers p95; shared free compute can host the rehearsal.",
    )


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--p95-ms", type=int, required=True)
    parser.add_argument("--client-timeout-ms", type=int, required=True)
    parser.add_argument("--write-steps", type=int, required=True)
    parser.add_argument("--idempotent", type=lambda v: v.lower() == "true")
    parser.add_argument("--can-kill-worker", type=lambda v: v.lower() == "true")
    parser.add_argument("--need-trace-export", type=lambda v: v.lower() == "true")
    args = parser.parse_args()
    facts = LoopFacts(
        p95_ms=args.p95_ms,
        client_timeout_ms=args.client_timeout_ms,
        write_steps=args.write_steps,
        idempotent=args.idempotent,
        can_kill_worker=args.can_kill_worker,
        need_trace_export=args.need_trace_export,
    )
    lane, reason = recommend(facts)
    overlap = retry_overlap(facts)
    print(f"retry_overlap={overlap}")
    print(f"lane={lane}")
    print(f"reason={reason}")


if __name__ == "__main__":
    main()

Use the matrix as a human check after the script. I still want a person to read the W column. Scripts miss product language that looks read-only and is not.

Signal you measured	Prefer shared free lane	Prefer isolated paid	Prefer self-hosted
Tools are `R` only	Yes, as a rehearsal bench	Only if you must pin a vendor SLA	Rarely worth the ops load
Timeout < p95, writes lack keys	No	Temporary, if you can pause traffic	Yes, so you can kill in-flight workers
Need exportable traces	Only if the host actually emits them	Yes, when audit is the product	Yes, when traces must never leave your disk
Secrets in the prompt or env	No	Maybe, with scoped keys	Yes, with local secret injection
You cannot name a rollback	No	No	No — fix the agent first

Notice the last row. If rollback is a shrug, no lane is honest. Compute choice cannot repair a tool that charges twice.

What this method refuses to promise

I am not publishing model names, token quotas, hardware SKUs, or durability claims I cannot verify. Free lanes change. Treat availability as a snapshot you re-check on the vendor's own page before you schedule a demo. This worksheet does not rank answer quality, and it does not prove that isolated compute is faster.

The Python file will happily recommend lane A if you feed it optimistic timings. Garbage in stays garbage. If you skipped the stall injection, you did not run the method. You filled a form.

Shared free compute also fails closed for regulated transcripts, customer secrets, and anything that must not train a neighbor's curiosity. I will not soften that. A timeout budget can be healthy and the data-handling story can still be disqualifying.

Who should skip this

Skip the three-lane sheet if you ship no tools and no side effects. A single-shot summarizer does not need this ceremony. Skip it if you already run a locked internal gateway with deadlines you enforce. You are past the question.

Skip it if you wanted a cost-break-even spreadsheet or a blast-radius score. Those are different filters, and I am not recycling them here. This pass only asks whether a stall can duplicate a write. That is enough work for one sitting.

If you still need a throwaway harness after the writes are idempotent, a free shared option such as MonkeyCode's free models and free server can host the rehearsal loop. Keep production traffic off that lane until the script and the matrix agree you can survive a stall. Then choose isolation because the retry story is settled, not because a leftover GPU looked lonely.

Score the Host Before You Promote a Vibe-Coded Agent

Riley Li — Sat, 19 Sep 2026 09:34:32 +0000

A green chat is not a merge signal, because the host can hide the failure you actually care about. I treat vibe-coded agent work as a draft until the eval host can fail closed, not until the wording looks finished. Free shared inference is useful for that draft, and a box I control is useful when the draft pretends to be engineering. If those two hosts score the same in your notes, are you scoring the demo or the risk?

This is a decision guide, not a pricing rant, and not a prompt-tuning diary. I keep seeing fluent agent traces get pasted into pull requests with no record of where they ran. The interesting question is not whether the model sounded confident. The interesting question is whether your compute choice can still prove a miss after the chat window is gone.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I mention MonkeyCode later only as one free-model and free-server option for the draft loop, not as a substitute for the scorecard.

The failure I am trying to prevent

Vibe coding is a fast way to sketch tool calls, retries, and glue code you would not type by hand. Calling that sketch engineering is how false-green tests sneak into main. Have you ever watched an agent pass a demo because the golden answer sat in the same prompt pack as the task? That is not intelligence. That is leakage with good manners.

Shared free runtimes make that leakage cheaper to produce, which is fine while you are still exploring. Isolated or paid runtimes make the same leakage more expensive, which is only worth it when a miss ships. I do not pick a host by leftover GPU time. I pick a host by whether a failed promotion can still be reconstructed tomorrow.

Five questions before the host gets a vote

I score every agent spike with the same five questions, in this order, before I argue about cost. If you cannot answer them from artifacts, the chat log does not count.

Can a reviewer replay the eval suite without reading the original chat?
Does the prompt pack hash differ from the eval-suite hash on disk?
Are tool credentials and network egress denied unless the host is a box you control?
Will a failed run leave a structured trace, or only a screenshot of a happy path?
If the model changes under you, can you freeze the comparison or must you shrug?

If questions 1 and 2 fail, I stay on a free exploratory loop and I refuse to merge. If questions 3 or 4 fail, I will not put secrets, production tools, or customer data on shared compute. If question 5 fails, I treat any “it works” claim as time-boxed, even when the prose looks senior.

A numbered workflow I actually run

I keep the workflow boring on purpose, because excitement is how demos skip gates. You can copy the steps without adopting my tooling, and you should label every unchecked box as unknown rather than true.

Write the task and the eval fixtures in two files, then refuse to paste fixtures into the system prompt.
Hash both files before the first agent call, and store the hashes next to the run id.
Run the draft on free shared models only if the tools are stubs and the data is synthetic.
Score the host with the table below, then promote to a self-hosted or paid box when any isolation row trips.
Re-run the same fixtures on the new host, and compare traces, not vibes.
Merge only when a stranger can fail the suite without sitting in your original session.

Would I skip step 3 to go faster? Sometimes, when the spike is a throwaway. Would I skip step 6 because the chat felt done? That is the whole bug this article is about.

Artifact: a demo-to-merge scorecard

The following script is a proposal you can run locally. It does not benchmark models, and it does not prove production safety. It fails closed when hashes collide, when isolation is required, or when the host kind is still unknown.

#!/usr/bin/env python3
"""Demo-to-merge scorecard. Proposal: treat unknown fields as fail."""
from __future__ import annotations

import hashlib
import json
from pathlib import Path

ISOLATING_TOOLS = {"shell", "browser", "payments", "prod_db", "secret_store"}


def sha256_file(path: Path) -> str:
    return hashlib.sha256(path.read_bytes()).hexdigest()


def score_run(
    prompt_pack: Path,
    eval_suite: Path,
    host_kind: str,
    tools: list[str],
    has_structured_trace: bool,
    can_freeze_model: bool,
) -> dict:
    prompt_hash = sha256_file(prompt_pack)
    eval_hash = sha256_file(eval_suite)
    needs_isolation = any(t in ISOLATING_TOOLS for t in tools)
    host = host_kind.lower().strip()

    checks = {
        "eval_outside_prompt": prompt_hash != eval_hash,
        "host_declared": host in {"free_shared", "self_hosted", "paid_isolated"},
        "isolation_matches_tools": (not needs_isolation) or host != "free_shared",
        "structured_trace": has_structured_trace,
        "model_can_be_frozen": can_freeze_model or host != "free_shared",
    }
    promote = all(checks.values()) and host != "free_shared"
    stay_draft = host == "free_shared" and checks["eval_outside_prompt"] and not needs_isolation
    return {
        "prompt_hash": prompt_hash,
        "eval_hash": eval_hash,
        "checks": checks,
        "decision": "promote" if promote else ("keep_draft" if stay_draft else "fail_closed"),
    }


if __name__ == "__main__":
    result = score_run(
        prompt_pack=Path("prompt_pack.txt"),
        eval_suite=Path("eval_suite.json"),
        host_kind="free_shared",
        tools=["retriever_stub"],
        has_structured_trace=True,
        can_freeze_model=False,
    )
    print(json.dumps(result, indent=2))

Create two tiny fixtures so the script is reproducible on a laptop. I keep them obviously fake, because a realistic customer dump would already fail the isolation row.

printf 'You are a summarizer. Never read eval_suite.json.\n' > prompt_pack.txt
printf '%s\n' '{"cases":[{"id":"t1","expect":"ok"}]}' > eval_suite.json
python3 scorecard.py

If you change tools to "shell" and leave host_kind as free_shared, the decision should become fail_closed. If both files contain the same bytes, it should also fail, even when the chat still looks clever. That is the point of hashing before arguing about prompts.

Tradeoffs I write down, not vibes I remember

I use this table when a teammate wants to stay on free shared compute because the demo already “works.” Working in a chat is one column. Promoting is another column. If you cannot fill a cell, the cell is a no.

Decision axis	Free shared models / free server	Self-hosted or paid isolated box
Fit	Synthetic data, stub tools, throwaway spikes	Secrets, egress, production tools, merge candidates
Speed to first trace	Usually faster to start	Slower until images, queues, and IAM exist
Eval honesty	Easy to contaminate the prompt pack	Easier to keep fixtures off the model path
Drift	Model and neighbor load can move under you	You can freeze runtime and often the weights
Failure evidence	May be a scrollback window	Can be files, metrics, and access logs
Cost shape	Cash is low; review time is not free	Cash is higher; incident cost should be lower
Wrong fit	Shipping a vibe as a contract	Isolating a spike that should have stayed a sketch

Notice that “free” only wins the cash row. Review time, incident time, and contaminated evals are still costs. Are you optimizing the invoice, or the chance that a false green reaches main?

MonkeyCode is an open-source project with free model access and a free server option, which I use only for the draft side of that table. I do not treat those free options as unlimited, permanent, or equivalent to a box I own. If the scorecard returns keep_draft, a free loop is enough. If it returns fail_closed or you want promote, move the same fixtures onto compute you can isolate.

What this method refuses to claim

This scorecard does not name models, quote throughput, or promise that any vendor will keep a free tier. It does not replace security review, license review, or load tests. Hash collision resistance here is about accidental copy-paste, not about an adversary. A structured trace can still lie if you log the wrong fields.

I also do not claim that self-hosting makes vibe-coded agents honest. Isolation only preserves the evidence. You still have to write evals that the agent cannot see. If your suite is three happy-path strings, both hosts will flatter you.

Who should not use this approach

Skip this if you are pasting secrets into a shared chat to “just try something.” Skip it if your agent already drives production tools, because the draft column is already the wrong column. Skip it if you need a cost-optimization spreadsheet rather than a promotion gate. Skip it if nobody on the team will store hashes and traces; a scorecard nobody runs is fan fiction.

Students and weekend spikes can stay on free shared inference for a long time, and they should. Teams that merge agent output into services should not. Which group are you in when the demo looks good at 5 p.m.?

A closing check I use on my own drafts

Before I call a vibe-coded agent “done,” I ask whether a reviewer can fail it without me in the room. If the answer depends on a shared session, I keep the work on a free exploratory loop and I do not promote it. If the answer depends on tools or data I cannot expose, I move the same fixtures to isolated compute and I run the scorecard again. The host is part of the test. Pretending otherwise is how engineering gets renamed after the fact.

If you want a draft-side place to run that loop, try MonkeyCode’s free model access and free server option on synthetic fixtures only, then promote with evidence instead of a fluent screenshot.

Choose Agent Compute by Replayable Evals, Not by Spare Capacity

Riley Li — Fri, 18 Sep 2026 07:57:01 +0000

I treat agent compute as a measurement problem first, and only then as a hosting or budget debate. If two runtimes cannot execute the same eval harness and emit a comparable ledger, I do not have a real choice. I have a one-way experiment between free shared capacity and a box I control, and I cannot score it later. Have you ever moved an agent onto unpaid capacity and then argued whether the model slipped or a tool call vanished?

Why spare capacity is not a decision yet

Cost spreadsheets hide the failure mode that quietly burns the most calendar time on a team. I cannot tell whether a runtime is cheaper if I cannot tell whether it still runs the same job. Self-hosted boxes give me process control, local disk, and a fingerprint I can store beside the output. Free shared options give me a lab I did not provision, which is useful only if the eval still means something.

The public conversation around generated code keeps circling whether the work still counts as engineering, and I will not join that slogan fight. I care about a narrower question that I can test in an afternoon without a research budget. Can I replay a frozen task suite on two runtimes and explain the diff without guessing at hidden wrappers? If I cannot, I am not choosing compute in any honest sense, and I am only collecting anecdotes.

Five checks I score before I pick a runtime

I keep the scoring boring on purpose, because boring is the only quality that stays replayable next week. Each candidate gets a yes, a no, or a not-yet against the same five checks I refuse to customize mid-argument. I do not let a price tag break a tie until those cells are filled with something I could show a teammate. If a vendor story cannot survive this worksheet, I treat it as a demo environment, not as a factory.

Pin. Can I freeze prompts, tools, fixtures, and expected side effects in git before anyone touches a new host?
Fingerprint. Can I record runtime identity, an image digest, or at least a hostname, clock, and tool versions beside every row?
Replay. Can I re-run the same suite after a queue delay, a host recycle, or a quiet weekend without rewriting prompts?
Ledger. Can I keep JSONL rows on disk I own if the free path pauses, throttles, or reroutes the next session?
Diff. Can a teammate read two rows and tell a real regression from timestamp noise, shuffled ids, or wrapper chatter?

Here is the decision table I actually fill. I treat it as a proposed worksheet, not as a benchmark I already ran against a production fleet.

Check	Box I control	Free shared lab	What a no means in practice
Pin	Yes if the repo is the only source of truth	Yes only if I carry the suite with me	You are demoing, not evaluating
Fingerprint	Yes, digest plus hostname	Often partial, sometimes missing	You cannot explain a Tuesday incident
Replay	Yes, until I delete the box	Maybe, if the queue still admits me	Last week's score is folklore
Ledger	Yes, local disk I own	Yes only if I copy rows off-box immediately	The evidence evaporates with the session
Diff	Yes, same harness both times	Yes only after I normalize ids and clocks	You will argue about the model instead of the trace

Notice that the table refuses to score marketing language, idle GPU screenshots, or how generous a free tier sounded in a launch thread. I score whether tomorrow-me can replay today-me without writing fan fiction about the runtime. Would you ship a database migration without a rollback plan, then still ship a runtime migration without a ledger you can grep?

A workflow I can finish in one sitting

I run this as a checklist, not as a research program, because the point is a comparable artifact. Nobody on my team is allowed to fall in love with unpaid capacity before the ledger exists. The steps are deliberately small so I cannot hide behind a two-week evaluation project.

Freeze a golden set of eight to twelve tasks that already failed in interesting ways, not only the happy path that flatters every host.
Hash every prompt, tool schema, and fixture so a silent file edit cannot hide inside a story about the model getting worse.
Run the suite once on a box I control, even if that box is a laptop, and write JSONL onto disk that I actually own.
Run the same suite on the free shared option without editing prompts, then copy the ledger off that host before I close the session.
Diff hashes, exit codes, and tool-call shapes before I look at latency, because latency lies when queues stall.
Label the free path as lab, factory, or reject: lab means explore, factory means keep for chores, reject means I still need isolation.

I do not always need twelve tasks, but I do need enough cases that a single lucky completion cannot flatter a runtime into a strategy slide. Have you noticed how one green demo becomes next quarter's default host before anyone stores a prompt hash? That is why the ledger stays small, mean, and checked into git beside the agent.

The artifact: a proposed replay ledger

The script below is a proposed local harness. I am not reporting production timings, and I am not claiming a bake-off against any hosted model catalog. It writes one JSONL row per task so I can diff two runtimes with ordinary Unix tools, which is the whole scientific instrument I need.

#!/usr/bin/env python3
"""replay_ledger.py — proposed harness, not a published benchmark."""
from __future__ import annotations

import hashlib
import json
import os
import platform
import subprocess
import sys
import time
from pathlib import Path

GOLDEN = [
    {
        "id": "t01_hash_prompt",
        "prompt": "Return only the integer 42.",
        "expect_substr": "42",
    },
    {
        "id": "t02_tool_shape",
        "prompt": "Call echo with argument ping and return stdout.",
        "expect_substr": "ping",
    },
    {
        "id": "t03_refuse_network",
        "prompt": "Do not open sockets. Reply with the word isolated.",
        "expect_substr": "isolated",
    },
]


def sha256_text(value: str) -> str:
    return hashlib.sha256(value.encode("utf-8")).hexdigest()[:16]


def fingerprint() -> dict:
    return {
        "runtime_id": os.environ.get("RUNTIME_ID", "unlabeled"),
        "host": platform.node(),
        "python": sys.version.split()[0],
        "cwd": str(Path.cwd()),
    }


def run_task(prompt: str) -> dict:
    """Replace this stub with your agent CLI. Keep the ledger shape stable."""
    started = time.time()
    # Proposed stand-in: echo the prompt so the workflow is runnable without a vendor SDK.
    completed = subprocess.run(
        ["python3", "-c", "import sys; print(sys.argv[1])", prompt],
        check=False,
        capture_output=True,
        text=True,
    )
    output = (completed.stdout or "").strip()
    return {
        "exit_code": completed.returncode,
        "output": output,
        "output_hash": sha256_text(output),
        "latency_ms": int((time.time() - started) * 1000),
        "stderr_hash": sha256_text(completed.stderr or ""),
    }


def main() -> None:
    out_path = Path(os.environ.get("LEDGER_PATH", "replay_ledger.jsonl"))
    fp = fingerprint()
    with out_path.open("a", encoding="utf-8") as handle:
        for task in GOLDEN:
            row = {
                **fp,
                "task_id": task["id"],
                "prompt_hash": sha256_text(task["prompt"]),
                "passed": False,
            }
            result = run_task(task["prompt"])
            row.update(result)
            row["passed"] = task["expect_substr"].lower() in result["output"].lower()
            handle.write(json.dumps(row, sort_keys=True) + "\n")
    print(f"wrote {out_path}")


if __name__ == "__main__":
    main()

I compile it first, then I run it twice with different RUNTIME_ID values, and only then I diff the rows. If I skip the compile step, I am not measuring agents. I am measuring a syntax error I created myself.

python3 -m py_compile replay_ledger.py
RUNTIME_ID=box-local LEDGER_PATH=ledger-local.jsonl python3 replay_ledger.py
RUNTIME_ID=free-lab LEDGER_PATH=ledger-free.jsonl python3 replay_ledger.py
python3 - <<'PY'
import json
from pathlib import Path

def load(path):
    rows = [json.loads(line) for line in Path(path).read_text().splitlines() if line]
    return {row["task_id"]: row for row in rows}

local = load("ledger-local.jsonl")
free = load("ledger-free.jsonl")
for task_id in local:
    a, b = local[task_id], free[task_id]
    print(task_id, {
        "pass_local": a["passed"],
        "pass_free": b["passed"],
        "out_match": a["output_hash"] == b["output_hash"],
        "exit_match": a["exit_code"] == b["exit_code"],
    })
PY

If out_match is false while prompt_hash is equal, I do not shrug and say the free path is just stochastic. I open the two outputs and I ask whether a tool policy, a wrapper, or a truncated context changed the job. Stochastic noise is a hypothesis I have to earn with more than one seed, not a blanket excuse for a messy ledger. Does that extra hour feel slow when a free queue is already waiting? It is still faster than a week of it worked on my shared host.

Where a free model lab belongs in this score

I still want a lab that I can enter without provisioning hardware first, especially when I am only trying to see whether a task suite is even worth isolating. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I treat MonkeyCode as an open-source project with free model access and a free server option. Those two sit in the free shared lab column of the table above, not in a special unmeasured column. I score that column with the same ledger I would use on a laptop, a rented VM, or any other unpaid queue.

I do not need a model catalog pitch to use the method, and I will not invent quotas, hardware, or bake-off numbers to dress it up. If the free path reproduces hashes and tool shapes, I keep exploration there and I stop paying for idle capacity I do not yet need. If the free path cannot fingerprint, cannot replay on Monday, or cannot let me copy JSONL off the box, I keep the factory on machines I control. Would I put a customer-facing agent on the first green lab run just because the queue was free? I would not, and I would wait until the ledger stays boring across a replay.

Limitations, and who should not use this

This workflow is the wrong tool if you do not have any frozen tasks yet, because you will measure theater instead of regressions. It is also the wrong tool if your agent must keep secrets on disk you do not control, even when the eval looks cheap on paper. Teams that need deterministic, audited, or air-gapped runs should start on isolated hosts and should not wait for a free queue to become a compliance story after the fact.

The harness above is a stub on purpose. It does not measure token quality, multi-hour jobs, GPU placement, or vendor quotas, and I will not invent those figures to look more empirical than I am. Stochastic generators can disagree without either runtime being broken, so a single mismatch is a clue I have to investigate, not a court verdict I can paste into Slack. If your golden set only contains tasks the model already likes, the ledger will flatter every host you try and teach you nothing.

I also skip this ceremony for throwaway spikes that die the same afternoon, because a replay ledger earns its keep only when someone will ask next week why the agent changed. If nobody will ask, I am not choosing a runtime in any serious way. I am doodling, and doodling does not need a decision guide.

What I decide after the rows exist

I read the table from the bottom, because Diff is the only check a future incident review will actually care about. If Diff is a no, I do not care that the lab was free, because I cannot defend the change to a teammate who was not in the room. If Replay and Ledger are yes, and the hashes mostly match, I am willing to keep the free path as a lab and even as a low-risk factory for internal chores. If Fingerprint is a no, I keep using the lab for drafts, and I refuse to let it become the system of record for anything I will have to explain.

That is the decision guide I actually want when the timeline is full of AI commentary and empty of comparable runs. Spare capacity is a rumor until the eval replays, and the ledger is the argument I can bring to a teammate without raising my voice. What would your current free queue look like if you scored it this way tonight, with hashes instead of vibes?

Pass Four Fit Gates Before You Choose Free or Isolated Agent Compute

Riley Li — Thu, 17 Sep 2026 05:58:56 +0000

Free shared inference is a fit only when four written gates pass, and the invoice is the last argument. I refuse to debate price while an agent still carries live secrets inside the planning prompt. Can you replay yesterday's failure against the same model identifier your tests already named? If that answer stays fuzzy, the free option is not your runtime yet, no matter how attractive the cost looks.

This article is a comparison framework for choosing free shared compute, a hybrid split, or a host you pay to isolate. I score secret proximity, replayability, concurrency shape, and model pinning as four independent gates. Only after those gates are written do I pick a runtime, because an unsigned workload card turns every vendor debate into theater.

Write the workload card before you pick a place to run

I start with a card in git, not with a signup form and a hopeful screenshot. The card is deliberately boring, because boring facts are easier to challenge than a vibe about good enough models. Would you let a teammate change the runtime without reading this card first, and still call the setup engineering?

Here is the template I keep as workload-card.json, and you should treat it as a labeled proposal rather than a production benchmark. Copy it, rename the agent, and lie to the file only if you also enjoy lying to your future self.

{
  "name": "doc-triage-agent",
  "touches_live_secrets": false,
  "customer_payload_in_prompt": false,
  "can_store_full_trace_locally": true,
  "needs_same_model_id_tomorrow": false,
  "parallel_runs_per_hour": 8,
  "standup_blocked_if_queue_stalls": false,
  "tool_execution_host": "local",
  "planning_host_preference": "shared_free"
}

Notice that tool_execution_host is already a decision, not a wish you hope the vendor will honor later. If tools run locally, a planning model can live on shared capacity without ever holding those keys. If tools and prompts share one opaque server, you have already failed the secret-proximity gate before the first token arrives.

Four fit gates, scored in a fixed order

I score each gate as pass, split, or fail, and I refuse to average them into a fake total. A split means the planning loop may use free shared inference while tools stay on a box I control. A fail means I will not park that workload on a free shared server, even when the queue is idle and the dashboard looks friendly.

Gate 1 — Secret proximity

Ask one blunt question that does not care about marketing pages or complimentary tiers. Can a log from this run be pasted into a public issue without a security review from someone sober? If the honest answer is no, free shared compute is the wrong place for the full loop, and you should redact or relocate before you continue.

I still allow a free model to see a scrubbed task statement, and I call that a split rather than a pass. Passing means the entire prompt bundle is already public-safe, including tool arguments and recovered files. Why would a zero invoice justify a prompt that still names an internal hostname?

Gate 2 — Replayability

Engineering needs a trace you can replay, which is the gap between sketching with a model and actually shipping an agent. Can you store prompts, tool calls, and outputs for at least seven days on storage you own? Shared free queues are fine for sketches, but they are a weak archive when a stakeholder asks what happened on Tuesday.

If replay lives only in a browser tab, I score this gate as fail and I stop. Export the bundle on every run, even when the model itself arrives through a free path. Without that export, you cannot tell whether the runtime failed or your prompt did.

Gate 3 — Concurrency shape

Count the runs you expect in a busy hour, not the runs you wish were complimentary. One developer iterating can absorb jitter, retries, and a queue that hiccups during someone else's spike. A cluster of agents feeding a morning standup cannot absorb that same jitter without turning the free tier into an accidental dependency.

I treat fewer than ten parallel runs per hour, with no human blocked on the queue, as a pass for free shared capacity in this proposal. Above that threshold, or if people wait on the output, I move the queue to paid isolation or a self-hosted runner. The number ten is a conversation starter, and you should replace it when you have real traffic.

Gate 4 — Model pinning

Do your tests name a model, or do they name whatever the free pool happens to serve after lunch? If the suite assumes stable tool-call shapes, you need a pin you can still resolve tomorrow. Free shared pools can change what they serve without booking a meeting on your calendar, and your assertions will rot quietly.

I only pass this gate when I can tolerate a different completion tomorrow without invalidating the last week of debugging. Assertion tests that freeze expected tool-call shapes usually need a runtime I can pin. If you cannot pin, do not pretend the test still measures the same system.

Decision table I actually walk

Secrets	Replay owned	Concurrency	Pin needed	Recommendation
pass	pass	pass	no pin required	Free shared planning is in play; keep tools local if they exist
split or fail	pass	pass	no pin required	Hybrid: local tools and redacted plans, never the raw payload
any	fail	any	any	Do not use shared free compute as the system of record
pass	pass	fail	any	Paid or self-hosted queue; free path only for off-peak sketches
any	any	any	pin required	Isolated or paid runtime where you can name the model tomorrow

I read the table from top to bottom and stop at the first row that matches the card. Ties go to the more isolated option, because isolation is cheaper than an incident you cannot reconstruct. Why would you optimize the invoice of a run you cannot defend in a review?

A reproducible scorer you can run locally

The following script is a proposal, and I have not attached production metrics to its thresholds. Save it as fit_score.py and point it at the JSON card so the decision is generated instead of performed in Slack.

#!/usr/bin/env python3
"""Proposal: score an agent workload card for free vs isolated compute."""
import json
import sys

PIN_FAIL = "self_hosted_or_paid_pinnable"
REPLAY_FAIL = "do_not_use_shared_free_as_record"
QUEUE_FAIL = "paid_or_self_hosted_queue"
HYBRID = "hybrid_local_tools_free_planning"
FREE_OK = "free_shared_ok_for_this_card"


def score(card: dict) -> str:
    secrets_ok = (not card["touches_live_secrets"]) and (
        not card["customer_payload_in_prompt"]
    )
    replay_ok = bool(card["can_store_full_trace_locally"])
    pin_needed = bool(card["needs_same_model_id_tomorrow"])
    busy = int(card["parallel_runs_per_hour"]) >= 10
    blocked = bool(card["standup_blocked_if_queue_stalls"])
    tools_local = card.get("tool_execution_host") == "local"

    if not replay_ok:
        return REPLAY_FAIL
    if pin_needed:
        return PIN_FAIL
    if busy or blocked:
        return QUEUE_FAIL
    if not secrets_ok:
        return HYBRID if tools_local else PIN_FAIL
    return FREE_OK


def main() -> None:
    path = sys.argv[1] if len(sys.argv) > 1 else "workload-card.json"
    with open(path, encoding="utf-8") as handle:
        card = json.load(handle)
    print(json.dumps({"name": card.get("name"), "decision": score(card)}, indent=2))


if __name__ == "__main__":
    main()

Run it like this from the same directory as the card:

python3 fit_score.py workload-card.json

You should see a single decision string that you can paste into a pull request. If you disagree with the string, change the card fields, not the narrative around the vendor. That habit is the entire difference between a fit check and a purchase argument.

Numbered workflow I follow on a new agent

Fill workload-card.json before you request any shared runtime or paste a secret into a prompt.
Run fit_score.py and paste the JSON decision into the pull request that adds the agent.
If the result is hybrid, move credentials and tool runners onto a host you control before the first live call.
Export a trace on the first failing run and confirm you can replay that bundle without the original web session.
Re-score after the first week, because concurrency always lies during the demo and the first internal fan-out.

I keep step five because the card rots in quiet ways that never show up in a launch checklist. A solo experiment becomes a team habit, and then the free queue is suddenly sitting in the critical path. Did anyone update the card when that promotion happened, or did the invoice stay at zero while the waiting queue grew?

Where a free model and a free server belong

When the card says free_shared_ok_for_this_card or hybrid_local_tools_free_planning, I look for a place that offers free model access without merging tools and secrets into the same box. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is an open project that currently offers free model access and a free server option, which I treat as one parking spot for the non-secret planning loop.

I do not treat that option as a pin, a quota promise, a hardware spec, or a production SLA, because none of those claims belong in this article. I still run the scorer first. If the scorer says isolate, I isolate, even when the shared option costs nothing and the signup is easy.

The useful split looks ordinary once the card exists. Planning prompts with redacted context can hit the free model path without inheriting production credentials. Tool execution, secret material, and the trace archive stay on a machine whose disk I can image, and the free server only sees what the card already marked public-safe.

Who should not use this approach

Do not use this scorer as a compliance stamp for regulated work, because it does not know your retention policy or your on-call rotation. Teams in healthcare, payments, or any environment with a contractual audit trail should assume fail until a security review says otherwise. The script will cheerfully recommend free shared planning if you lie on the card, which is a feature of honesty, not a bug in Python.

Do not use it if you cannot export traces to storage you control, since a pretty web session with no local bundle fails gate two by construction. Do not use it to justify stuffing customer data into a shared prompt because the invoice is zero. I also would not leave a hackathon demo on free shared compute after it grows a public URL, without filling the card again.

Limitations of the artifact

The thresholds are arbitrary on purpose, and they will be wrong for your traffic on a busy Thursday. Ten parallel runs per hour is a prompt for a conversation, not a load test you can cite. The script never measures latency, token spend, or answer quality, and those omitted numbers go stale faster than a written secret-proximity answer.

I have not claimed that any vendor, including a free option, will keep the same models available next month. Pinning is a requirement you bring to the runtime, not a feature I can invent in a blog post. If your work needs a named model tomorrow, budget for a runtime that lets you name it, then keep the free path for cards that passed without that need.

Closing

Write the four gates, run the scorer, and only then pick free, hybrid, or isolated compute for that agent. Keep the card in git either way, because the next argument should be about fit rather than about a price tag. If you want a shared place to park a card that already passed, MonkeyCode's free model access and free server option are one way to try that split.

Who Owns the Failed Agent Run? Score Free Shared Compute Against a Box You Control

Riley Li — Wed, 16 Sep 2026 04:11:07 +0000

The timeline keeps arguing whether models already write better code than most working software developers today. I care less about that ranking than about who can replay a failed agent loop. Free shared agent compute looks cheap until a run dies and you cannot replay it. I score recovery ownership first, and only then do I look at the invoice.

Who keeps the traces, the workspace, and the kill switch when a tool loop fails? If you cannot answer that question clearly, the zero-dollar option is not actually free for you. I used to compare runtimes by sticker price and advertised model access, and that ranking often lied. A failed agent loop burns tokens, queue time, and engineer attention that never appear on a pricing page.

Start from the failure, not from the catalog

Can you reproduce the same prompt, tools, and working files on another machine later tonight? If the answer is no, you are renting a black box rather than borrowing spare compute. Shared free pools remain useful for exploration and for throwaway prompt drafts that hold no secrets. They get expensive when the failure mode is a vanished job with no debugger hook attached.

Self-hosted boxes cost money and operations time, yet they often leave you the corpse of the process. Paid hosted runtimes sit in the middle, with nicer dashboards and fuzzier answers about who can copy a workspace. Which failure mode can your team actually afford during a normal on-call week at work? Write that failure down before anyone pastes a pricing URL into the design doc.

What I mean by recovery ownership

Recovery ownership is a boring checklist rather than a product vibe or a conference slogan. I want to know who can pause a run, copy the workspace, and replay the exact tool sequence later. I also want to know whether logs survive preemption, because shared queues love killing long or idle jobs. Those four questions decide more of my week than any model card on a homepage.

Here is the question I ask before I accept a free slot on a shared pool. If this agent corrupts a fixture or hangs on a tool call, can I inspect the filesystem without filing a ticket? If I cannot inspect that filesystem, I treat the runtime as a demo environment only. Production-shaped loops need a place where failure still leaves evidence you can hold.

A scorecard you can fill in one sitting

I use five rows and three columns so the argument stays on one page. The rows are secrets, traces, workspace, kill switch, and replay against known inputs. The columns are free shared compute, a box I control, and a paid hosted runtime. I score each cell 0, 1, or 2, where 2 means I can act without waiting on another team.

You should not treat my numbers as a benchmark, because I have not run a public bake-off. They are a conversation tool for your network, secret store, and retention rules. Fill the table with constraints you can defend in review, not with a vendor slogan from a landing page. If a cell is unknown, score it 0 until you prove the action yourself.

Concern	Free shared pool	Self-hosted box	Paid hosted runtime
You keep secrets in your vault	0–1	2	1–2
Traces survive preemption	0–1	2	1–2
Workspace is copyable after failure	0–1	2	1
You can kill a runaway tool loop	1	2	1–2
You can replay the same inputs	0–1	2	1–2

I add the column totals and I refuse to pick a winner until the lowest acceptable score is written down. What is your minimum score for traces, and what is your minimum for secrets? If free shared compute cannot meet those floors, the advertised price is irrelevant. Write the floors in the README so the next on-call person does not renegotiate them from scratch.

Numbered workflow I run before I pick a pool

Write the failure you fear in one sentence, including data that must not leak.
List the artifacts you need after a crash: traces, diffs, eval JSON, and the working tree.
Score each runtime against those artifacts using the 0–2 scale in the table.
Estimate retry cost as failed_runs * (queue_wait + tool_calls * avg_tool_latency).
Choose the cheapest runtime that still clears your floors, not the cheapest label on the sheet.

That fourth step is a sketch, and I do not pretend it is a finance model for procurement. I still write the formula down because teams argue about feelings when the units are missing. Replace the variables with log fields you already emit, and keep the formula visible beside the runner. If you cannot emit those fields, that absence is already a score of zero.

A labeled example that estimates retry cost from JSONL logs

This snippet is a proposal, and I am not claiming it ran in production or produced a published number. Drop it next to your agent runner if your logs already look like JSON lines with stable field names. Change the keys if your schema differs, and keep the script boring enough to read in review.

# proposal: retry_cost.py — estimate wait+tool cost of failed agent loops
import json
import sys
from collections import defaultdict

# Expected fields: runtime, status, queue_s, tool_calls, tool_latency_s
totals = defaultdict(lambda: {"fails": 0, "queue": 0.0, "tool": 0.0})

for line in sys.stdin:
    event = json.loads(line)
    runtime = event.get("runtime", "unknown")
    if event.get("status") != "failed":
        continue
    totals[runtime]["fails"] += 1
    totals[runtime]["queue"] += float(event.get("queue_s", 0))
    tool_calls = int(event.get("tool_calls", 0))
    avg_tool = float(event.get("tool_latency_s", 0))
    totals[runtime]["tool"] += tool_calls * avg_tool

for runtime, row in sorted(totals.items()):
    retry_s = row["queue"] + row["tool"]
    print(f"{runtime}: failed_runs={row['fails']} retry_seconds={retry_s:.1f}")

Run it like this after you export a day of logs from each runtime under test. Notice that I am not converting seconds into dollars, because that conversion belongs to your payroll and your cloud bill.

python retry_cost.py < agent_runs.jsonl

If a free pool shows fewer failed runs but huge queue_s, I treat the wait as a tax. If a self-hosted box shows more failures but tiny queue time, I can often iterate faster anyway. Which column would you rather debug at eleven at night, when the eval gate is red? That preference belongs in the scorecard, not in a hallway argument.

Where a free model and a free server still fit

I do not reject free shared compute as a category, and I reject using it as the only home a loop can live. Exploration, prompt drafts, and throwaway evals are a good match, especially when the workspace holds no secrets. Production-shaped agents, customer data, and long tool loops usually want a box I can pause. The scorecard exists so that move is explicit rather than accidental.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option, which I treat as one shared pool on the scorecard rather than a default winner. I use that option when the failure I fear is wasted time, not leaked credentials or an unreproducible patch. Score it with the same five rows you use for a laptop or a rented VM.

Commands I keep next to the scorecard

I want a local smoke test that does not depend on a vendor dashboard or a disappearing job page. This is a sketch of a tiny runner contract: write inputs, run the agent, and keep the workspace on disk. Replace the commented command with your real binary, and fail the check if the trace file is missing.

# proposal: local contract for an agent eval, not a vendor tutorial
mkdir -p /tmp/agent-replay/{in,out,ws}
printf '{"task":"rename flaky test","repo":"."}\n' > /tmp/agent-replay/in/task.json
# agent-run --input /tmp/agent-replay/in --workdir /tmp/agent-replay/ws --trace /tmp/agent-replay/out
test -f /tmp/agent-replay/out/trace.jsonl && echo "trace kept"

If the free server cannot leave trace.jsonl in a place you control, I score traces as zero. If your self-hosted box can keep that file after a crash, I score traces as two. The gap between those scores is the whole decision, not the catalog copy. Can your current pool pass this check on a bad afternoon without opening a ticket?

I also keep a tiny secret-hygiene grep beside the runner, because free pools and laptops fail in different ways. This is still a proposal, not evidence from a named production incident.

# proposal: refuse to start if the workspace looks like it holds live secrets
if grep -R -E 'AKIA[0-9A-Z]{16}|BEGIN (RSA |OPENSSH )?PRIVATE KEY' /tmp/agent-replay/ws; then
  echo "refuse: workspace looks like it contains secrets" >&2
  exit 2
fi

If that check fires, I do not send the tree to a shared pool, no matter how attractive the invoice looks. Self-hosted still needs the same check, because a box you own can leak just as loudly. Recovery ownership includes the right to refuse a run, not only the right to replay one. Do you currently have that refuse path in CI, or only in a wiki page?

Who should not use this scorecard

Do not use this method if you need a procurement-ready TCO model, because I am not offering audited cloud prices or hardware SKUs. Do not use it if your agents must remain inside a vendor you cannot leave, because portability is assumed throughout. Do not use free shared compute for regulated data, production secrets, or unattended write access to real infrastructure. Those are floors, not preferences you can bargain away after a demo.

I also would not use a free pool as the only CI lane for agent-generated patches landing on main. Continuous integration needs replay, and replay needs artifacts you still have after the job is killed. If your organization cannot store traces, fix storage before you argue about model quality. Otherwise you will debug folklore instead of files, and the scorecard cannot save that.

Limitations

This scorecard ignores model quality on purpose, because a clever model on an uninspectable runtime will still waste a week. I also ignored GPU mix, exact quotas, and hardware SKUs, because those numbers go stale and I will not invent them. Your network policy, secret manager, and retention rules will change the scores more than any article. Re-score when those rules change, not when a homepage banner changes.

The retry formula under-counts human time, which is usually the expensive part of a failed loop. If two runtimes tie on the sheet, I pick the one where I can attach a debugger without scheduling a meeting. That bias is mine, and you should write your own tie-break in the README. The useful question stays the same: who owns the failed run when you need it back?

Free Compute or Self-Hosted? A Break-Even Guide for Agent Workloads

Riley Li — Mon, 14 Sep 2026 21:46:04 +0000

The sticker price of compute tells you almost nothing about what one finished task actually costs you. What decides the argument is cost per verified task, and that number crosses over at a point you can calculate before you commit. So isn't the smarter question not "which option is cheapest" but "which option is cheapest for the shape of workload I actually run?"

What the free option actually is

MonkeyCode is an open-source project, and the operator states that it currently offers free model access together with a free server option, including a free token allowance reported at the time of writing as 10 million tokens. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I have not benchmarked the platform myself, and I will not invent model names, latency figures, or throughput numbers for it.

Free allowances move constantly, so please treat that 10 million token figure as a snapshot rather than a contract. Confirm the current quota, the available models, and the free server limits on the project's own page before you design anything around them. If you cannot verify the terms today, plan for them to change tomorrow.

1. Define one unit of work that survives contact with reality

Before comparing anything, you need a unit that either passes a checker or fails it, because "cheap per call" is not the same thing as "cheap per finished job." Teams that skip this step usually pay twice for the same task and never notice the second invoice. Here is the smallest harness I would trust, and it is deliberately boring.

# verify.py — a task counts only if the checker passes for it
from collections import defaultdict
from dataclasses import dataclass

@dataclass
class Attempt:
    task_id: str
    tokens_in: int
    tokens_out: int
    wall_seconds: float
    passed: bool

def attempts_per_success(attempts):
    tries, won = defaultdict(int), defaultdict(bool)
    for a in attempts:
        tries[a.task_id] += 1
        won[a.task_id] = won[a.task_id] or a.passed
    wins = [t for k, t in tries.items() if won[k]]
    return sum(wins) / len(wins) if wins else float("inf")

Notice what this measures and what it refuses to measure. It counts attempts per successful task, not attempts per call, and it ignores any attempt that never produced a passing artifact. That single change is usually what turns a fuzzy argument about free compute into a number two engineers can agree on.

2. Collect your own p and token spend before you argue

Run roughly thirty tasks that look like your real queue, and log four fields per attempt: task id, input tokens, output tokens, and wall seconds. Do not cherry-pick easy tasks for the free endpoint and hard ones for the paid endpoint, because that comparison is worthless and everyone in the review will spot it.

A free endpoint that needs 2.1 attempts per success can cost more in engineer time than a paid endpoint that needs 1.05, simply because waiting and re-checking is work. Ask yourself honestly: who is watching that retry loop, and what is their hour worth to your team?

3. Turn the measurements into a crossover point

The script below takes your measurements, not mine, and returns cost per verified task for each route. Every default is illustrative only, so replace them with numbers you actually collected, including your own ops overhead and any overflow price beyond the free allowance.

# tco.py — inputs are YOUR measurements; defaults are placeholders, not quotes
def cost_per_verified_task(attempts_per_success, usd_per_attempt,
                           ops_usd_per_task=0.0, wall_minutes_per_task=0.0,
                           engineer_usd_per_hour=0.0):
    waiting = wall_minutes_per_task / 60 * engineer_usd_per_hour
    return attempts_per_success * usd_per_attempt + ops_usd_per_task + waiting

def overflow_cost(tokens_used, free_tokens, usd_per_token_beyond_free):
    return max(0, tokens_used - free_tokens) * usd_per_token_beyond_free

def break_even_tasks(fixed_monthly_usd, usd_per_task_a, usd_per_task_b):
    gap = usd_per_task_b - usd_per_task_a
    return float("inf") if gap <= 0 else fixed_monthly_usd / gap

Run it from the shell with the same arguments your logs produced, and keep the output in the commit message for whichever route you pick.

python3 tco.py --attempts-per-success 1.00 --usd-per-attempt 0.00 \
  --ops-usd-per-task 0.00 --wall-minutes-per-task 0.0 --engineer-usd-per-hour 0.0

4. Decide with fit criteria, not with vibes

A crossover only tells you where the lines meet, and it says nothing about whether an option is allowed to serve your workload. Use the table as the second filter, applied after the arithmetic, because compliance and residency questions cannot be paid away with a cheaper token.

Signal	Free tier fits when	Self-hosted fits when	Managed paid fits when
Data residency	Workload is public or non-sensitive	Regulated data must stay on your network	Contract and region controls are the product
Throughput shape	Bursty, interactive, low sustained volume	Steady load near your hardware ceiling	Spiky peaks you refuse to capacity-plan
Ops headcount	Nobody wants to run GPUs	You already run inference infrastructure	You want zero operational surface
Reproducibility	Version pinning is acceptable and easy	You control the exact build	Vendor offers stable pinned versions
Budget shape	Exploration and prototypes	Long-lived, predictable baseline load	Variable cost is preferable to capex

Debugging workflow: five checks in order

Is the failure actually a checker bug? Re-run the failing task by hand once, because a broken assertion quietly inflates your attempts-per-success and makes every option look expensive.
Are retries doubling token spend? Sum tokens across attempts per task, not per call, and see whether the retry loop is the real cost center.
Is the queue the bottleneck rather than the model? Compare wall seconds against model seconds; if queueing dominates, you are buying patience, not intelligence.
Are you computing the same task twice across two routes? Duplicated work across a free endpoint and a paid fallback will make the cheaper route look worse than it is.
Has the workload shape changed since you measured? Re-measure monthly, because a crossover computed on last quarter's task mix is a story about a workload you no longer run.

Limitations and who should skip this approach

This framework assumes you can define a passing checker, which rules out exploratory work where "done" is a judgment call rather than a test. It also ignores cold-start behavior, network variance, and any quota that resets in ways your logs cannot see, so a single week of measurement is weaker evidence than you might wish.

If your workload touches regulated data, or if an outage costs more than the compute ever will, a free tier is not a candidate at all, no matter how the arithmetic lands. Similarly, if you cannot spare a few hours to build the checker, you are not ready to compare routes, because every number you produce will be anecdotal.

What would your own thirty-task suite say about the route you are using right now? Build the harness, log the attempts, and let cost per verified task make the argument for you. If you want to try the free tier first, find the MonkeyCode open-source repository, run the protocol above on your real queue, and only then decide whether to keep it or move on.

Rank Agent Runtimes by the Contract You Can Enforce

Riley Li — Sun, 13 Sep 2026 20:25:02 +0000

Price tags still lie about agent runtimes, and I keep learning that lesson the hard way. The useful question is not which box is free today, but which contract I can actually enforce tomorrow. Can I bound latency, keep secrets off shared disks, own the transcript, and retry without burning the budget? If those four checks fail, I refuse the complimentary endpoint, even when the model quality looks fine.

Why sticker price is a weak ranking signal

Shared free compute looks attractive when a prototype still changes shape every single afternoon, right? Paid APIs and self-hosted boxes look expensive until a leaked token or irreproducible trace appears. I do not treat those three options as a quality ranking; I treat them as contracts I can or cannot test. The public argument keeps circling model IQ, while my incidents usually start in the runtime contract instead.

A free shared workspace can still be the correct runtime for a throwaway exploration job today. It just has to survive the same checks I would write for a paid or isolated cluster. If I cannot express a requirement as a failing test, I do not get to call it a requirement. That sounds pedantic until a teammate cannot replay last Tuesday's tool call from memory.

Four surfaces I actually write down

I keep the contract tiny on purpose, because a twelve-page policy never actually runs in CI. Each surface maps to one executable check, and I reject runtimes that cannot fail those checks loudly. Does that feel too strict for a weekend prototype that nobody else will supposedly touch? Maybe so, but the prototype is exactly when secrets and logs get copied onto the wrong disk.

Latency bound. The run must terminate or abort inside a declared wall-clock budget.
Secret boundary. Values marked secret must not appear in traces, shells, or shared volumes.
Transcript ownership. I must export a hashed, timestamped log that I still control after exit.
Retry budget. A flaky tool call may retry N times, then stop, without silent amplification.

Those four checks do not cover safety, evaluation quality, fairness, or any prompt injection defense at all. They cover the boring failures that make free compute look cheap until a Friday rollback. If a provider cannot help me test even one of them, I stop arguing about tokens per dollar.

A numbered workflow I actually follow

Here is the sequence I use before I attach any model endpoint, free or not.

Step 1: Freeze the contract as YAML

I write the contract first, in a file the agent runner can parse without ceremony. The numbers below are placeholders for your team, not measurements I am claiming as evidence. Would I ship those numbers into production without measuring my own jobs against them first?

# agent_compute_contract.yaml — proposal, not a measured SLA
latency:
  wall_clock_seconds: 90
  on_timeout: abort
secret_boundary:
  deny_patterns:
    - "API_KEY"
    - "PRIVATE"
  scan_targets: ["stdout", "trace.json", "workspace"]
transcript:
  export_path: "./artifacts/transcript.json"
  hash_alg: sha256
  retain_locally: true
retry:
  max_attempts: 2
  backoff_seconds: 3
  amplify: false

Those placeholder values exist so the next step has something concrete to fail against later. Change them when your jobs are longer, chatty, or allowed to retry with side effects. Please do not copy them into a status page and then call the result an SLA. The file is a test fixture, and that is the entire point of writing it down.

Step 2: Run a contract probe before the agent loop

I treat the probe as a gate for promotion, not as a benchmark of model quality. The script below is a labeled example you should adapt rather than paste into production. It does not claim any vendor result, quota, hardware profile, or any measured latency distribution either.

# contract_probe.py — example probe, not executed against a vendor here
from __future__ import annotations

import hashlib
import json
import os
import time
from pathlib import Path

CONTRACT = {
    "wall_clock_seconds": 90,
    "max_attempts": 2,
    "export_path": Path("artifacts/transcript.json"),
}


def hash_file(path: Path) -> str:
    digest = hashlib.sha256()
    digest.update(path.read_bytes())
    return digest.hexdigest()


def probe_timeout(run_agent, payload: dict) -> dict:
    started = time.monotonic()
    try:
        result = run_agent(payload, timeout=CONTRACT["wall_clock_seconds"])
        status = "ok"
    except TimeoutError:
        result = {"error": "timeout"}
        status = "timeout"
    elapsed = time.monotonic() - started
    return {"status": status, "elapsed": elapsed, "result": result}


def probe_secrets(trace_text: str) -> list[str]:
    leaks = []
    for key, value in os.environ.items():
        if not value:
            continue
        if "KEY" in key or "TOKEN" in key or "SECRET" in key:
            if value in trace_text:
                leaks.append(key)
    return leaks


def probe_transcript(payload: dict, raw: bytes) -> str:
    CONTRACT["export_path"].parent.mkdir(parents=True, exist_ok=True)
    CONTRACT["export_path"].write_bytes(raw)
    return hash_file(CONTRACT["export_path"])


def decide(leaks: list[str], elapsed: float, attempts: int) -> str:
    if leaks:
        return "reject-shared-or-fix-redaction"
    if elapsed > CONTRACT["wall_clock_seconds"]:
        return "reject-or-move-isolated"
    if attempts > CONTRACT["max_attempts"]:
        return "reject-retry-amplification"
    return "accept-candidate"


if __name__ == "__main__":
    # Example driver. Wire run_agent to your runner; this stub stays local.
    artifacts = Path("artifacts")
    artifacts.mkdir(parents=True, exist_ok=True)
    fake_trace = "ok"
    leaks = probe_secrets(fake_trace)
    elapsed = 0.0
    attempts = 1
    decision = decide(leaks, elapsed, attempts)
    raw = json.dumps({"trace": fake_trace, "decision": decision}, indent=2).encode()
    digest = probe_transcript({"demo": True}, raw)
    payload = {
        "decision": decision,
        "leaks": leaks,
        "elapsed": elapsed,
        "transcript_sha256": digest,
    }
    (artifacts / "probe.json").write_text(json.dumps(payload, indent=2))
    print(json.dumps(payload, indent=2))

I run that probe against every candidate runtime, including the free one in my bookmarks. If the probe cannot even start, that absence is already a decision about the contract. Why would I debug an agent loop on a box that cannot export a hashed transcript? I would not, and that early refusal is cheaper than almost any later migration story.

Step 3: Score runtimes with a fit matrix, not a vibe

I map each failed probe onto a runtime change instead of delivering another motivational speech. The table is the artifact I want you to copy, argue with, and then keep in git. Can a vibe in a Slack thread replace a row that already failed in CI? Not on my team, because vibes do not leave a hash next to the transcript.

Contract check	Passes on free shared compute	Typical paid API fit	Typical self-hosted fit	Move when it fails
Latency bound	Only if queue delay stays inside the budget	Useful when an abort path is documented	Strongest when you own the queue	Isolated runtime or smaller jobs
Secret boundary	Risky if the workspace is shared	Better if the vendor redacts traces	Strongest if secrets never leave the box	Self-hosted or a local runner
Transcript ownership	Weak if logs remain trapped in a UI	Medium if a real export path exists	Strong if your process writes the files	Any runtime that cannot export
Retry budget	Weak if hidden retries already exist	Medium if you cap attempts yourself	Strong if you wrap the whole loop	Disable auto-retry, then retest

Notice I still keep a column for free shared compute rather than deleting it in panic. I am not allergic to free resources, but I am allergic to untestable complimentary runtimes. If every row passes, the cheap box wins until a new secret or side effect appears. If one row fails, the discussion is about that row, not about brand loyalty.

Step 4: Attach a model only after the runtime survives

Only now do I care which model endpoint sits on the other side of the runner. If the contract passes on a free shared server, I use it for exploration I can throw away. If secret boundary or transcript ownership fails, I stop sending extra prompts to that box. I move the same probe to a paid API or a machine I control, then I compare hashes.

That sequence is the whole decision guide, and everything after it is only supporting detail. People still want a winner among free, paid, and self-hosted, and I do not have one. I only have a function that maps each failed check onto the next runtime class I will try. Is that less exciting than a leaderboard? Yes, and that is why it survives contact with incidents.

Where a free model and free server actually help

I still want a cheap runtime for developing the contract itself, before any invoice exists. Writing probes only against a billable API teaches invoice anxiety instead of teaching actual isolation. A complimentary model endpoint plus a free server helps when the work is breaking the probe on purpose. The agent loop should remain a small repo while those failures are still cheap to repeat.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is one candidate I put in that cheap runtime, because it offers free model access and a free server option. I do not treat that offer as a permanent SLA, a hardware spec, or a ranking against paid labs. I treat it as a place to fail the contract quickly while the runner still fits in git. If you need a complimentary model-and-server target for the probe, that is the only role I am describing.

Fit criteria I use in review meetings

When a teammate asks why we skipped the free box, I do not answer with a speech. I answer with the matrix row that failed, plus the probe output sitting in artifacts. Does that feel cold in a planning meeting that really wanted a narrative instead of a table? It keeps the conversation technical, which is the only reason I write contracts as tests.

Use the free shared option when all of the following are true:

The job has no production secrets, or secrets are injected only on your side of the probe.
You can export a transcript you hash and store in your own artifacts directory.
A timeout abort is acceptable, and a noisy-neighbor delay will not page anyone.
Hidden retries cannot duplicate side effects such as emails, charges, or issue comments.

Prefer a paid API when you need a documented export path or a rate limit you can quote. Prefer self-hosted when the secret boundary is non-negotiable, or when transcripts are a legal record. Those are fit criteria for a specific job, not moral rankings of vendors or of free tiers. Can one team hold all three runtimes at once? Yes, if the probe stays the promotion gate.

Limitations, and who should skip this

This framework will annoy people who want a single winner and a tweet-sized recommendation after lunch. It also underfits teams that already banned shared workspaces through a real, written security review. If you train models, serve regulated data, or drive irreversible actuations, skip free shared servers. A green probe on a toy repository does not authorize those jobs, even when nobody is watching.

The probe is not a security audit, and I need to say that without softening it. Pattern-scanning environment variables will miss encoded leaks, side channels, and writes outside the workspace path. The YAML numbers are not evidence, and I have not included vendor benchmarks, model names, or quotas. If your organization needs those figures, measure them yourself and store the results beside the transcript hash.

I also would not use this on a one-hour demo where the agent cannot touch the network. In that case the contract is simply an airgapped laptop, and the matrix is unnecessary ceremony. Use judgment, then write the smallest contract that would have caught your last painful incident. If you have never had an incident yet, start with secrets and transcript ownership anyway.

A compact command path you can copy

Once the YAML file and probe exist, I keep the operator loop boring on purpose. Excitement belongs in the agent task, not in the promotion path that guards production secrets. The commands below are a labeled local example, not a claim about any hosted fleet.

# labeled example — run locally against YOUR runner
mkdir -p artifacts
export AGENT_CONTRACT=./agent_compute_contract.yaml
python contract_probe.py
python - <<'PY'
import json
from pathlib import Path
probe = json.loads(Path('artifacts/probe.json').read_text())
print('decision:', probe.get('decision', 'missing'))
print('leaks:', probe.get('leaks', []))
print('elapsed:', probe.get('elapsed'))
print('transcript_sha256:', probe.get('transcript_sha256'))
PY
test -f artifacts/transcript.json && sha256sum artifacts/transcript.json

If the printed decision is not accept-candidate, I do not pass the model a production prompt. That habit has saved more money than discount hunting, and it has saved more sleep than dashboards. Can you ignore the gate for a demo on synthetic data? Yes, and you should label it a demo. Unlabeled demos are exactly how complimentary runtimes accidentally become unofficial production dependencies overnight.

Closing

I started with the conclusion, and I will end on the same sentence shape for a reason. Rank agent compute by the contract you can enforce, then spend free shared capacity only on passing jobs. Paid and self-hosted runtimes exist for the rows that fail, not for a vague sense of professionalism. Keep the matrix in git, keep the hashes next to it, and let the failed row choose the next box.

Invert Non-Negotiable Constraints Before You Accept Free Agent Compute

Riley Li — Sat, 12 Sep 2026 18:25:46 +0000

Free shared agent compute is the right default only after you list constraints you will not relax. I still watch teams grab a free endpoint first, then discover the loop can mail customers they cannot later audit. Price never explained that failure, because isolation, replay, and data ownership were never on the shopping list. So I invert the choice: name the non-negotiables, then pick the cheapest lane that still honors them.

Does that sound slower than opening a playground and pasting a prompt into the first vendor tab? It is slower, and I would rather spend twenty minutes on constraints than a week explaining an unreproducible tool call. Free remains useful, but it has to survive that inversion instead of skipping to the front of the line.

Price hides the tradeoffs you actually buy

A free model lane and a free server look identical to a paid API until the first bad tool call lands. Then you learn whether traces leave the box, whether another tenant can starve your queue, and whether Tuesday's payload still exists. Self-hosted runtimes flip those answers, but they also dump drivers, network policy, and patching onto your calendar without asking. Paid APIs sit in the awkward middle: clearer tenancy than a weekend shared box, less operational drag than a cluster you page.

I do not treat "free" as a personality trait of a product I happen to like this week. I treat it as a bundle of relaxed guarantees that somebody else already chose for me. If I cannot name those relaxed guarantees out loud, I am not choosing a lane; I am hoping the demo stays read-only. Have you ever shipped a read-only agent that grew a send-email tool before the README caught up with it?

Four constraints I score before any endpoint

I keep four questions beside the agent repo, and I refuse to discuss model names until they have answers. If any answer is a hard constraint, free shared compute has to earn a seat instead of inheriting one. Quality comparisons come later, because a brilliant completion that I cannot replay is only an anecdote with extra tokens.

Trace custody. Can I export every prompt, tool call, and tool result into storage I control?
Side-effect radius. Can this loop mutate tickets, mail, money, or production files without a human gate?
Replay fidelity. Can I rerun last Tuesday's transcript against a new prompt and get a comparable trace?
Queue isolation. Can a noisy neighbor delay a job that a human is already waiting on?

Notice that none of those questions is which model looks smartest in a social feed this week. Smartness is an experiment you can rerun after the lane is honest enough to hold evidence. Constraints decide whether that experiment is even allowed to run on shared hardware you do not operate.

Invert the constraints, then pick a lane

Here is the sitting-length workflow I actually walk when a new agent shows up in a review. It fits in one pass, and it produces a file you can argue about instead of a slide. If the file and the tool list disagree, I trust the tool list and I update the file immediately.

Write the agent's tools as an allow-list, and mark each tool read or write.
Record each constraint above as hard, soft, or off in YAML next to the code.
Reject any lane that violates a hard constraint, including a lane that happens to be free.
Among the lanes that survive, pick the smallest operations surface you can staff this week.
Re-score the file after the first mutating tool ships, because the matrix goes stale that afternoon.

Would I skip step three because a demo is due on Friday and the slide already says shipped? I have wanted to, and that is usually when the wrong email leaves the building under a demo account. The inversion is a brake pedal, not a vibe, and the allow-list below is what I brake against.

{
  "tools": [
    {"name": "search_docs", "mode": "read"},
    {"name": "list_tickets", "mode": "read"},
    {"name": "send_email", "mode": "write"}
  ]
}

If send_email exists and nobody is in the loop, free shared compute is already on probation. Read tools can stay cheap; write tools have to buy isolation or a human gate, sometimes both. That distinction matters more than the sticker on the inference endpoint you pasted into dotenv.

Decision matrix: free shared, paid API, self-hosted

I use the matrix as a conversation starter with whoever owns the tools, not as a score that pretends to be science. Read down the first column until you hit a constraint you will not relax for this agent. Then read right, and stop treating every free box as interchangeable with a runtime you actually isolate.

Hard constraint you cannot relax	Free shared model or server	Paid isolated API	Self-hosted runtime
Trace custody	Only if you copy logs into storage you own	Fit when export and retention are explicit	Fit if you already run the archive
Unattended mutating tools	Poor fit	Conditional on identity, audit, and rollback	Conditional on IAM you actually maintain
Replay for eval gates	Weak unless transcripts live in your repo	Fit if request ids stay stable	Fit if you pin the runtime image
Human waiting on the result	Risky on contended shared capacity	Better with reserved throughput	Best with dedicated workers
No hard constraints, read-only tools	Strong fit for exploration	Often overkill while the loop is still changing	Overkill until the workflow stops thrashing

This table is a fit guide, not a benchmark, and it will not rank vendors by tokens or latency. I am not publishing tokens per dollar, percentile charts, or a bake-off I did not run on your traffic. If a vendor card disagrees with a hard constraint in the YAML, the constraint wins and the card waits.

A mapper you can run locally

The script below is a proposal I keep as a checklist, not a production control plane with SLAs. It reads YAML, prints a lane, and exits non-zero when free shared compute fails the inversion outright. Install PyYAML, then run it from the agent repository root before anyone argues about endpoints again. Label it as unexecuted in your own environment until you have run it against your real tool list.

# agent_constraints.yaml
constraints:
  trace_custody: off          # hard | soft | off
  mutating_tools: off
  replay_fidelity: soft
  queue_isolation: off
  human_gated_writes: true

#!/usr/bin/env python3
"""Constraint-inversion mapper for agent compute lanes.

Proposal / checklist script. It does not probe live vendors or capacity.
"""
from __future__ import annotations

import argparse
from dataclasses import dataclass
from typing import Literal

try:
    import yaml
except ImportError as exc:
    raise SystemExit("pip install pyyaml") from exc

Lane = Literal["free_shared", "paid_api", "self_hosted"]


@dataclass
class Constraints:
    trace_custody: str
    mutating_tools: str
    replay_fidelity: str
    queue_isolation: str
    human_gated_writes: bool


HARD = "hard"


def load(path: str) -> Constraints:
    with open(path, encoding="utf-8") as handle:
        raw = yaml.safe_load(handle) or {}
    block = raw.get("constraints", raw)
    return Constraints(
        trace_custody=str(block.get("trace_custody", "off")),
        mutating_tools=str(block.get("mutating_tools", "off")),
        replay_fidelity=str(block.get("replay_fidelity", "off")),
        queue_isolation=str(block.get("queue_isolation", "off")),
        human_gated_writes=bool(block.get("human_gated_writes", False)),
    )


def invert(c: Constraints) -> tuple[Lane, str]:
    hard_mutation = c.mutating_tools == HARD and not c.human_gated_writes
    if hard_mutation and c.trace_custody == HARD and c.queue_isolation == HARD:
        return (
            "self_hosted",
            "Unattended writes plus custody plus isolation need a runtime you operate.",
        )
    if hard_mutation:
        return (
            "paid_api",
            "Unattended writes need a lane with identity, audit, and a real boundary.",
        )
    if c.trace_custody == HARD or c.replay_fidelity == HARD:
        return (
            "paid_api",
            "You must own or export traces; a contended free box is a weak archive.",
        )
    if c.queue_isolation == HARD:
        return (
            "paid_api",
            "A human is waiting; do not share a queue you cannot protect.",
        )
    return (
        "free_shared",
        "Read-only or gated work with no hard isolation demand can stay on free shared compute.",
    )


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--config", default="agent_constraints.yaml")
    args = parser.parse_args()
    lane, reason = invert(load(args.config))
    print(f"lane={lane}")
    print(f"reason={reason}")
    raise SystemExit(0 if lane == "free_shared" else 2)


if __name__ == "__main__":
    main()

pip install pyyaml
python constraint_invert.py --config agent_constraints.yaml
echo "exit=$?"

Flip mutating_tools to hard and set human_gated_writes to false, then run the same command again. You should leave free_shared immediately, which is the entire lesson encoded as a boring exit code. I want that refusal in CI before anyone pastes a new base URL into an environment file.

# Proposal only: gate endpoint changes on the inversion, not on enthusiasm.
python constraint_invert.py --config agent_constraints.yaml
# exit 0 -> free_shared is still an honest fit
# exit 2 -> choose paid_api or self_hosted before swapping endpoints

Where a free model lane still wins

When every tool is read-only, traces can live in my repository, and nobody is blocked on the queue, cheap compute is the honest answer. Many so-called agents are still deterministic workflows with a language model sitting at the leaves of a boring graph. Those leaves do not need a private cluster on day one, and pretending they do just delays learning.

MonkeyCode is one option in that exploration bucket, and I am not going to dress it up as a compliance boundary. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The project is open source, and it offers free model access plus a free server option I treat as a shared matrix lane. I still keep the constraint file in git, because the lane should be swappable the hour a write tool appears.

I do not need that lane to win a quality contest I never ran against a private cluster. I need it to survive the four questions without pretending isolation that the lane does not actually provide. If you are mapping a read-only agent this week, drop a free shared lane into the matrix and check your hard constraints.

Limitations, and who should not use this

This inversion will not satisfy a regulator, a SOC questionnaire, or a lawyer who needs a paper trail. It also will not detect silent prompt injection, poisoned tools, or a model that confidently lies about tool results. The script never probes live capacity, never estimates cost, and never pins a model name that would rot in a week. Those claims go stale faster than a README, so I refuse to print them as if they were facts.

Skip the free shared recommendation if your prompts contain customer PII, or if writes travel without a human gate. Skip self-hosted if nobody on the team can patch the box this quarter without opening a mystery incident. Skip paid APIs if you cannot explain data retention to the rest of the company in one paragraph. Skip this whole checklist if you do not have tools at all; a side-effect-free chatbot can stay in a playground.

Constraints first, then the cheapest honest lane, is the entire decision I want in the repository. Everything else is shopping, and shopping is how unattended mail tools land on shared queues by accident. Invert the file, then pick the endpoint, and do not reverse those two steps because a homepage said free.

Keep the Agent Loop Portable, Then Choose Free Shared Compute

Riley Li — Fri, 11 Sep 2026 15:56:37 +0000

Free shared compute is the right first lane when your agent loop can move tomorrow without rewriting tools, traces, or secrets. Paid APIs and self-hosted boxes win when that loop is welded to private data, mutating tools, or evals you cannot replay. I treat the reverse order as the expensive mistake, because you pay in migrations rather than invoices. Why start with the price tag when the part you cannot unwind is the coupling?

Coupling is not latency, and it is not your monthly bill

I use coupling to mean how much of the agent definition lives outside your repository. Prompts, tool schemas, trace format, secret handling, and retry policy should stay yours even if the GPU does not. If swapping the endpoint forces a rewrite of those five pieces, you did not rent compute. You rented an architecture, and that is a much stickier invoice.

This is a proposed checklist, not a production study, and I am not attaching fabricated timings to it. Treat the scores as a conversation you can have in a pull request, then replace the weights with your constraints. The point is to decide the lane before the loop grows a dozen irreversible side effects. Can your standup even name those side effects today?

If the agent is still a trench coat, stay on free

A lot of so-called agents are still a classifier plus a switch statement wearing a confident system prompt. Those loops barely couple to anything, so a shared free lane is usually enough and I would not apologize for using one. The coupling checklist will green-light them quickly, and that result is correct rather than naive. Are you sure your loop is more than a trench coat, or have the tools never left the slide deck?

The moment you add tools that send mail, open pull requests, or touch billing, the score should flop toward a private lane. I want that flop to happen on paper, before a weekend prototype writes to production because someone pasted a key. List every tool out loud and mark each one read-only or mutating. If you cannot finish that list, you are not choosing compute yet. You are still designing the agent.

Six questions that actually change the lane

Ask these in order, and stop at the first hard no whenever the data is truly sensitive. I keep dollars out of this pass on purpose so the architecture argument stays visible.

Can I export every trace as JSON that another runner can ingest without a vendor SDK?
Do any tools mutate production systems, or do they only read fixtures and stubs?
Do prompts or retrieval chunks contain customer data I cannot put on a shared tenant?
Must evals pin behavior across weeks, or is directional quality enough for this loop?
Can the loop tolerate noisy-neighbor latency, or does a human wait on every tool call?
If the free lane vanished tonight, could I rerun the same fixtures on another endpoint this week?

Notice that none of those questions mention a unit price for tokens. Cost still matters later, after you know whether the loop is portable at all. If question two or three is a hard no, skip the free shared lane and go to a paid API or a box you control. Would you like a prettier matrix than that? Not until those two answers are honest.

A five-step workflow before anyone pastes a key

Here is the sequence I want written down before CI learns a new endpoint. The code is a proposed harness. It checks the scorer and the loop contract, not a live vendor and not a benchmark I did not run.

Step 1: Freeze the loop in your repo

Write the agent as a plain function that takes a task record and returns a trace. Keep system prompts, tool JSON, and retry limits in files you can grep without clicking a hosted UI. If the product console is the only place those live, you are already coupled and the rest of this article will not save you.

# loop_contract.py — proposed interface, not a vendor SDK
from dataclasses import dataclass, field
from typing import Any, Callable

@dataclass
class Task:
    id: str
    input: str
    fixtures: dict[str, Any]

@dataclass
class TraceEvent:
    role: str
    name: str
    payload: dict[str, Any]

@dataclass
class Trace:
    task_id: str
    events: list[TraceEvent] = field(default_factory=list)
    ok: bool = False

def run_loop(task: Task, complete: Callable[[str, list[dict]], str]) -> Trace:
    """One turn-taking loop. `complete` is the only endpoint-shaped hole."""
    trace = Trace(task_id=task.id)
    system = open("prompts/system.md", encoding="utf-8").read()
    messages = [
        {"role": "system", "content": system},
        {"role": "user", "content": task.input},
    ]
    # Dispatch tools against task.fixtures only. Never against live prod.
    _ = messages
    return trace

Step 2: Score coupling with explicit weights

I keep the weights boring so a reviewer can argue with them in the diff. Hide them in a private spreadsheet and the decision turns back into vibes. Change the numbers in review if your world is harsher than mine.

# coupling_score.py
from dataclasses import dataclass

@dataclass
class CouplingAnswers:
    traces_exportable: bool
    tools_are_read_only: bool
    no_customer_data_in_prompts: bool
    directional_evals_ok: bool
    latency_slack_ok: bool
    can_rerun_elsewhere_this_week: bool

WEIGHTS = {
    "traces_exportable": 2,
    "tools_are_read_only": 3,
    "no_customer_data_in_prompts": 3,
    "directional_evals_ok": 1,
    "latency_slack_ok": 1,
    "can_rerun_elsewhere_this_week": 2,
}

def coupling_score(a: CouplingAnswers) -> tuple[int, str]:
    flags = a.__dict__
    score = sum(WEIGHTS[k] for k, ok in flags.items() if ok)
    hard_block = (not a.tools_are_read_only) or (not a.no_customer_data_in_prompts)
    if hard_block:
        return score, "self_hosted_or_paid_private"
    if score >= 10 and a.can_rerun_elsewhere_this_week:
        return score, "free_shared_ok"
    if score >= 7:
        return score, "paid_api_ok"
    return score, "self_hosted_or_paid_private"

I also keep the answers in YAML so a human can read the gate without opening Python. That file belongs next to the prompts, not in a wiki that nobody updates.

# coupling.yml — proposed answers for a read-only sketch loop
traces_exportable: true
tools_are_read_only: true
no_customer_data_in_prompts: true
directional_evals_ok: true
latency_slack_ok: true
can_rerun_elsewhere_this_week: true

# load_coupling.py — proposed
import yaml
from coupling_score import CouplingAnswers, coupling_score

def recommend(path: str = "coupling.yml") -> tuple[int, str]:
    with open(path, encoding="utf-8") as fh:
        raw = yaml.safe_load(fh)
    return coupling_score(CouplingAnswers(**raw))

Step 3: Prove the score with a tiny test, not a slide

Slides do not fail in CI. These tests do, and they only prove the scorer, which is the entire point of this artifact.

# test_coupling_score.py
from coupling_score import CouplingAnswers, coupling_score

def test_mutating_tools_never_land_on_shared_free():
    a = CouplingAnswers(
        traces_exportable=True,
        tools_are_read_only=False,
        no_customer_data_in_prompts=True,
        directional_evals_ok=True,
        latency_slack_ok=True,
        can_rerun_elsewhere_this_week=True,
    )
    _, lane = coupling_score(a)
    assert lane == "self_hosted_or_paid_private"

def test_portable_read_only_loop_may_use_free_shared():
    a = CouplingAnswers(
        traces_exportable=True,
        tools_are_read_only=True,
        no_customer_data_in_prompts=True,
        directional_evals_ok=True,
        latency_slack_ok=True,
        can_rerun_elsewhere_this_week=True,
    )
    score, lane = coupling_score(a)
    assert score == 12
    assert lane == "free_shared_ok"

def test_private_prompts_are_a_hard_block():
    a = CouplingAnswers(
        traces_exportable=True,
        tools_are_read_only=True,
        no_customer_data_in_prompts=False,
        directional_evals_ok=True,
        latency_slack_ok=True,
        can_rerun_elsewhere_this_week=True,
    )
    _, lane = coupling_score(a)
    assert lane == "self_hosted_or_paid_private"

python -m pytest test_coupling_score.py -q
python -c "from load_coupling import recommend; print(recommend())"

Step 4: Map the lane to an actual substrate

Use the table, then write the choice into the pull request, not into Slack memory. I care less about the brand on the GPU than about which refusals you are signing up for.

Lane	Fit	You accept	You refuse
Free shared models plus a free server	Read-only tools, no customer prompts, exportable traces	Noisy neighbors, weaker pinning, possible queueing	Secrets, production mutators, compliance boundaries
Paid API	You need a clearer endpoint contract and less queueing	An invoice, still someone else's tenancy	Full control of weights and disk
Self-hosted	Mutating tools, private corpora, pinned evals	Ops load, patching, capacity planning	Treating GPUs as a weekend experiment

Step 5: Keep a kill switch in the runner

If the abort feels harsh, good. The cheap lane should be hard to enter once tools start writing. I would rather fail a job than discover coupling during an incident review.

# runner.py — proposed
import os
from coupling_score import CouplingAnswers, coupling_score

LANE = os.environ.get("AGENT_LANE", "free_shared_ok")

def assert_lane_matches(answers: CouplingAnswers) -> None:
    _, recommended = coupling_score(answers)
    if LANE == "free_shared_ok" and recommended != "free_shared_ok":
        raise SystemExit(
            f"refusing to start: coupling recommends {recommended}, not {LANE}"
        )

export AGENT_LANE=free_shared_ok
python runner.py  # should refuse when coupling.yml is a hard block

Where a free model lane still wins

Plenty of loops are sketches: classifying fixtures, drafting refactors against a public repo, or teaching the team how tool calling actually looks. Those loops should not wait for a purchase order, and they should not wait for a cluster either. I want them cheap, portable, and easy to throw away when the prompt is wrong.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I mention MonkeyCode here only because its free model access and free server option are a concrete shared lane you can point this checklist at, not because a coupling test requires that product. If you try that lane, keep the complete() hole tiny and keep traces in your repository. The free server is useful when you want a process that is not your laptop, not when you want a new source of truth.

Would I put a customer corpus there? No. Would I put a read-only loop that I can replay on another endpoint? Yes, and I would keep the pytest from Step 3 in CI so the answer cannot drift quietly. That is the whole rehearsal: prove portability on a lane that is allowed to be imperfect.

Limitations, and who should skip this

This scoring does not measure model quality, and it does not promise that any free lane stays free. I am not publishing quotas, hardware lists, or model names I cannot verify, and you should not treat my weights as research. Teams under HIPAA, PCI, or similar rules should not use a shared free tenant for prompts or traces, period.

Skip this approach if your agent is already a production mutator with no stubbed tools. Skip it if you cannot export traces into JSON you own. Skip it if leadership wants a vendor demo more than a portable loop. A checklist will not save a design that stored secrets inside a web UI and called that an architecture.

The honest close is simple. Score coupling first, pick the cheapest lane that survives the hard blocks, and keep the loop in git. If a free shared option helps you rehearse that discipline on a non-sensitive task, use it as a rehearsal stage rather than a forever home.

Pick Agent Compute by Blast Radius, Not by the Price Tag

Riley Li — Wed, 09 Sep 2026 16:13:04 +0000

Shared free compute is honest for agent work only when every tool is read-only, replayable, and free of secrets. If your loop can mutate state, leak a credential, or poison an eval set, sticker price is the wrong axis. I treat blast radius as the first filter, and only then do I ask whether a free model lane even fits. Does a weekend prototype that barely prints logs still deserve that same blast-radius filter before it ships?

The assumption almost every agent tutorial smuggles in

Most agent tutorials quietly assume the runtime is a private laptop that nobody else can inspect. Tools inherit that fantasy: they read local files, they retry network calls, and they treat environment variables as a private diary. What happens when that same loop lands on a shared free box or a host that actually holds secrets? The model did not get dumber; the blast radius of its tools just became a different product.

I keep a four-band map for this, because free versus paid is too coarse to be a real decision. Band 0 is pure reasoning with no tools at all, which is rarer than README files admit. Band 1 is read-only fixtures, golden traces, and local diffs that never leave the process. Can you honestly put your current demo into Band 1 without lying about the shell tool?

Band 2 is outbound reads against public docs or a throwaway sandbox, still with no mounted secrets. Band 3 is anything that can write, pay, deploy, or see credentials, and that band never belongs on shared free compute. Is Band 3 rare in agent demos, or do we just rename the dangerous tools until the demo recording looks harmless?

A five-step filter to run before picking a lane

Here is the workflow I use as a checklist, not as a score that pretends to be science. Label this a proposal if you have not wired it to your own job queue yet. Each step produces an artifact you can diff, which is the entire point of a gate. If a step has no artifact, I treat the inventory as incomplete and I fail closed.

1. Inventory every tool the agent may call, including retries

Write the tool list down before you pick a vendor, a container, or a vague free endpoint. Include implicit tools: shell, file write, browser, SQL, and whatever the framework injects when the model panics. If a retry can double a POST, count that POST twice in the blast-radius story. Would you accept a security review that omitted the retry path because the happy path looked read-only?

# Proposal: dump the tool names your agent harness registered.
python -c "from my_agent import tools; print('\n'.join(sorted(t.name for t in tools)))"

2. Tag each tool with the worst thing it can do

I use four tags only, because extra categories become a junk drawer by the next refactor. think means no I/O, and read_local means fixtures plus traces that never leave the disk. read_network means unauthenticated or explicitly public GETs, and mutate_or_secret means writes, credentials, payments, or deploys. Ask yourself one rude question here: if this tool lies, what still moves in the real world?

3. Promote the job to the maximum tag, not the average tag

A loop with nine read-only tools and one gh token is still a Band 3 job. Averages hide the landmine, which is why the gate uses maximum tag rather than a friendly mean. I refuse to mostly put Band 3 work on a shared free server just because the prompt looks academic. Does a single mutating tool feel like an exception, or does it rewrite the whole isolation story?

4. Match the band to an isolation lane

Band 0 and Band 1 can use shared free compute, a paid API worker, or a self-hosted box.
Band 2 belongs on a paid API or a locked-down sandbox, with allowlisted hosts and no .env file.
Band 3 belongs on self-hosted or a dedicated account, with secrets injected at run time and an audit log you control.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. When a job really sits in Band 0 or Band 1, MonkeyCode's free model access and free server option are one honest place to iterate. I do not treat that free lane as a production control plane, and I do not invent capacity, model names, or quotas here. If your tools need isolation, skip the shared option and move to self-hosted without bargaining on the tags.

5. Fail closed when the inventory is incomplete

Missing tool metadata is not a reason to default to free; it is a reason to refuse a shared runtime. Would you mount an undocumented binary on a box that other tenants can reach on purpose? Then do not do it with an agent loop whose tool list you have not actually printed. Fail closed, then fill the inventory, then run the gate again with a real spec.

Artifact: a blast-radius gate you can run in CI

The following script is a concrete gate, not a benchmark and not a vendor bake-off. It reads a JSON job spec, computes the maximum band, and exits non-zero when Band 3 work targets shared_free. Copy it, then point it at your own specs; I am not claiming production metrics from a script you have not run. If you change the allow-map, keep Band 3 off shared free compute or the whole filter is theater.

#!/usr/bin/env python3
"""blast_radius_gate.py — fail CI when the runtime is too shared for the tools."""

from __future__ import annotations

import json
import sys
from pathlib import Path
from typing import Literal

Band = Literal[0, 1, 2, 3]
Lane = Literal["shared_free", "paid_api", "self_hosted"]

TAG_TO_BAND = {
    "think": 0,
    "read_local": 1,
    "read_network": 2,
    "mutate_or_secret": 3,
}

ALLOWED_LANES: dict[Band, set[Lane]] = {
    0: {"shared_free", "paid_api", "self_hosted"},
    1: {"shared_free", "paid_api", "self_hosted"},
    2: {"paid_api", "self_hosted"},
    3: {"self_hosted"},
}


def band_for_tools(tools: list[dict]) -> Band:
    if not tools:
        return 0
    bands = [TAG_TO_BAND[t["tag"]] for t in tools]
    return max(bands)  # type: ignore[return-value]


def evaluate(spec: dict) -> tuple[bool, str]:
    lane: Lane = spec["lane"]
    tools = spec.get("tools")
    if tools is None:
        return False, "incomplete inventory; fail closed (no shared runtime)"
    band = band_for_tools(tools)
    allowed = ALLOWED_LANES[band]
    if lane not in allowed:
        return False, f"band {band} job cannot use lane {lane}; allowed={sorted(allowed)}"
    return True, f"ok: band {band} on {lane}"


def main(argv: list[str]) -> int:
    if len(argv) != 2:
        print("usage: blast_radius_gate.py <job-spec.json>", file=sys.stderr)
        return 2
    spec = json.loads(Path(argv[1]).read_text(encoding="utf-8"))
    ok, message = evaluate(spec)
    print(message)
    return 0 if ok else 1


if __name__ == "__main__":
    raise SystemExit(main(sys.argv))

A Band 1 spec that is allowed on shared free compute looks like the JSON below. Notice there is no shell, no ticket writer, and no environment dump. The emit_markdown tool is tagged think because it only formats memory. If that function later writes a file, you must retag it before the next run.

{
  "name": "summarize-golden-traces",
  "lane": "shared_free",
  "tools": [
    {"name": "read_fixture", "tag": "read_local"},
    {"name": "emit_markdown", "tag": "think"}
  ]
}

A Band 3 spec that must not go there looks like this next block. The runbook reader is harmless, but jira_create mutates a real tracker. Maximum tag wins, so the whole job is Band 3. Parking it on shared_free is a gate failure, not a style choice.

{
  "name": "open-incident-ticket",
  "lane": "shared_free",
  "tools": [
    {"name": "read_runbook", "tag": "read_local"},
    {"name": "jira_create", "tag": "mutate_or_secret"}
  ]
}

Run the gate like this, and keep the commands in your PR checklist. The second command should print a refusal and exit 1. If it does not, your copy of the gate is not the one above. Want a tiny unit check without standing up a framework?

python3 blast_radius_gate.py jobs/summarize.json
python3 blast_radius_gate.py jobs/incident.json; echo "exit=$?"

# test_blast_radius_gate.py — proposal tests, not a published benchmark
from blast_radius_gate import evaluate

def test_band3_rejected_on_shared_free():
    spec = {
        "lane": "shared_free",
        "tools": [{"name": "ship", "tag": "mutate_or_secret"}],
    }
    ok, msg = evaluate(spec)
    assert ok is False
    assert "band 3" in msg

def test_missing_tools_fail_closed():
    ok, msg = evaluate({"lane": "shared_free"})
    assert ok is False
    assert "incomplete" in msg

python3 -m pytest test_blast_radius_gate.py -q

Decision table I keep next to the job YAML

Job signal	Band	Shared free models/server	Paid API worker	Self-hosted
Read-only fixtures, no secrets	1	Fit	Fit	Fit, usually overkill
Public HTTP GET, allowlisted hosts	2	Not fit	Fit if egress is pinned	Fit
Writes, tickets, deploys, payments	3	Not fit	Risky unless dedicated	Default
Incomplete tool inventory	n/a	Not fit	Not fit	Only with a human review
Need replay you can hash	1–2	Only if traces stay local	Fit if you store receipts	Fit
Tenant-adjacent data	3	Not fit	Check the contract	Default

Notice the table never says faster or cheaper, because those adjectives need measurements you actually ran. I am not inventing latency, token ceilings, or vendor rankings in this guide. The only claim is fitness against blast radius, which you can falsify by listing one tool I missed. If a row does not match your compliance policy, the policy wins and the table is just a draft.

What this filter will not do for you

This approach will not pick a model, will not estimate latency, and will not prove a vendor is safe. It also will not save a Band 3 agent that you dressed up as Band 1 by renaming shell to research. If you are under HIPAA, PCI, or an employer policy that forbids shared runtimes, skip the free lane entirely. Do not bargain with the checklist when the policy already answered the lane question for you.

Anyone whose tools can spend money, change customer data, or see production secrets should not use shared free compute. Anyone who needs a legally durable audit log should not use it either, even for a so-called dry run. Also skip this guide if you cannot list the tools, because a decision without inventory is only a vibe. Is that a smaller audience than the internet's agent demos imply, or did the demos skip the dangerous tools on camera?

I still like free shared compute for throwaway summarizers, prompt diffs, and golden traces that never included secrets. That door should stay narrow, because one new tool can move a job from Band 1 to Band 3 overnight. If a tool appears in the loop tomorrow, re-run the gate before you congratulate the agent for being done. Price can still matter later, but only after the blast radius says the lane is allowed to exist.

Choose Agent Compute by What You Can Falsify

Riley Li — Sun, 06 Sep 2026 21:30:18 +0000

The right default remains a free shared agent lane until you cannot falsify the assumption behind a generated diff. Paid APIs and self-hosted boxes are evidence tools, not prestige upgrades you purchase because a usage graph looks lonely. I treat compute choice as a paper-trail question: can another engineer reconstruct why the agent touched that file? If that reconstruction fails, the cheaper tokens were never the bargain, because you now pay in review time.

The failure mode is confident help you cannot replay

Coding agents rarely explode in a cinematic way; they assume a runner, a lockfile, or a “safe” rename, then ship a tidy patch. Have you opened a pull request and been unable to name the turn that invented a dependency? That gap is not a model-quality problem first. It is an evidence problem, and it decides whether free shared compute is still honest.

I am not arguing against free model access or a free server option for spikes, docs, and throwaway refactors. Those lanes are the correct default while the work is public-shaped and the blast radius stays inside a disposable worktree. The expensive mistake is switching vendors for vibes, or staying shared after private context and replay become the actual product. The rest of this piece is a fit test, not a brand ranking.

This is a proposed heuristic, not a production audit I ran against a customer fleet. If your compliance team already dictates log ownership, skip to the matrix and treat my first-person voice as a checklist, not testimony.

Kill criteria beat weighted scorecards

I do not want another weighted score that pretends architecture is a spreadsheet. You stay on the free lane while every kill criterion still holds, and you move only when one breaks. When it breaks, you choose a paid API versus self-hosting by who must own the logs, not by who published a prettier playground. Why would a larger context window repair an unnamed assumption?

Step 1. Write the one assumption the agent may keep

Write a single ugly sentence the agent may treat as true without extra evidence from the repo. Examples I actually use: “tests run with pytest from the repository root,” or “.env.example contains no production secret.” If you cannot write that sentence, you are not ready for any lane, free or paid. A bigger model will only fail faster, with better grammar.

# assumption_contract.txt  (commit this next to the agent prompt)
MAY_ASSUME: pytest is invoked from repo root via `python -m pytest`
MUST_NOT_ASSUME: package manager, cloud region, or secret locations
FALSIFY_WITH: python -m pytest -q tests/test_assumption_contract.py

Step 2. Attach an evidence pack to every file touch

An evidence pack is local text: prompt id, path, claimed intent, and the command that would falsify the change. I keep packs in agent_evidence.jsonl because git already diffs line-oriented logs. A free shared server is still fine while that file can live on your laptop without private source inside it. It stops being fine when the pack would contain customer data, credentials, or a proprietary tree.

{"prompt_id":"p12","path":"src/app.py","intent":"guard empty payload","falsify":"python -m pytest tests/test_app.py::test_empty_payload"}

Step 3. Name who must replay the session later

Ask who has to reconstruct the agent turn in six months, not this afternoon. If the answer is only you, a free shared workspace is enough for scaffolding. If the answer is an on-call rotation, security, or legal, you need a lane whose logs you can retain and delete on purpose. Paid APIs often help when you want vendor-side retention hooks. Self-hosting helps when the disk itself is the control.

Step 4. Split blast radius from “the model seemed dumb”

People upgrade models because a free agent looks lost, then watch a stronger model smash configuration with more confidence. Is the defect really quality, or missing isolation around network and secrets? If the agent can write only inside a worktree and cannot reach production, stay free. If it needs deploy keys, a long-lived shared workspace, or outbound network, move lanes before you tune prompts.

Fit matrix: free, paid API, or your own box

Use this table as a gate, not as marketing. The free column assumes a shared coding environment with free model access, which is a starting lane rather than a forever contract. Paid API means you send prompts to a billed endpoint you do not operate. Self-hosted means the runtime and disks are yours, including the boring backup work.

Signal you can observe	Stay on free models + free server	Move to a paid API	Move to self-hosted
Assumption contract fits in one sentence	Yes, and tests can falsify it locally	Contract spans several services	Contract includes air-gapped or regulated data
Evidence packs stay off private source	Packs are paths and commands only	You need vendor retention or export	Packs include source, secrets, or customer text
Replay audience	Solo, same day	Team that wants an API audit trail	Compliance or on-call that must own disk
Blast radius	Disposable worktree, no deploy keys	Managed isolation is enough	You must control network egress
Failure you actually hit	Weak first draft, easy to reject	You need SLA-backed availability	You cannot send the tree off-box

Notice what is missing: token folklore, model-name shopping, and “this is always cheaper.” Those are not fit criteria. If you cannot point at a row that broke, you are not graduating the lane; you are fidgeting.

Artifact: a proposed classifier you can unit-test

The following is labeled on purpose: it is a decision procedure, not a benchmark, and I have not attached fake latency numbers to it. Put it in paper_trail.py and refuse to change vendors until a test names the broken criterion. If a future you wants weights, add them in a fork, and keep the kill switches boolean so the matrix stays honest.

from dataclasses import dataclass
from enum import Enum


class Lane(str, Enum):
    FREE_SHARED = "free_shared"
    PAID_API = "paid_api"
    SELF_HOSTED = "self_hosted"


@dataclass(frozen=True)
class Fit:
    assumption_named: bool
    evidence_is_local_only: bool
    pack_includes_private_context: bool
    replay_audience: str  # solo | team | compliance
    needs_network_secrets: bool
    must_own_disk: bool
    airgapped: bool


class UnnamedAssumptionError(ValueError):
    """Raised when the agent is allowed to invent the problem statement."""


def classify(fit: Fit) -> Lane:
    if not fit.assumption_named:
        raise UnnamedAssumptionError(
            "Name the assumption before you pick compute. No lane fixes this."
        )
    if fit.airgapped or fit.must_own_disk or fit.pack_includes_private_context:
        return Lane.SELF_HOSTED
    if (
        fit.replay_audience in {"team", "compliance"}
        or fit.needs_network_secrets
        or not fit.evidence_is_local_only
    ):
        return Lane.PAID_API
    return Lane.FREE_SHARED

# test_paper_trail.py  — proposed tests, not a production suite I executed on live traffic
import pytest
from paper_trail import Fit, Lane, UnnamedAssumptionError, classify


def test_unnamed_assumption_blocks_every_lane():
    fit = Fit(False, True, False, "solo", False, False, False)
    with pytest.raises(UnnamedAssumptionError):
        classify(fit)


def test_solo_local_evidence_stays_free():
    fit = Fit(True, True, False, "solo", False, False, False)
    assert classify(fit) is Lane.FREE_SHARED


def test_team_replay_moves_to_paid_api():
    fit = Fit(True, True, False, "team", False, False, False)
    assert classify(fit) is Lane.PAID_API


def test_private_pack_forces_self_host():
    fit = Fit(True, False, True, "solo", False, False, False)
    assert classify(fit) is Lane.SELF_HOSTED

Run the tests before you open a billing console. The command is boring on purpose, because lane choice should be a failing assertion, not a slide.

python -m pytest test_paper_trail.py -q
git add assumption_contract.txt agent_evidence.jsonl paper_trail.py test_paper_trail.py

If you want a durable pointer without stuffing JSON into the commit message, hang the latest pack on a git note. That still does not replace a real audit log, and I would not pretend it does.

git notes --ref=agent-evidence add -F agent_evidence.jsonl HEAD
git notes --ref=agent-evidence show HEAD

Where a free model path actually participates

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I am using MonkeyCode here only as one example of a lane that offers free model access and a free server option, which matters if Step 1 and Step 2 still pass. I will not invent model names, quotas, hardware, duration, or permanence, because those claims go stale the moment a dashboard changes. Open source is relevant only insofar as you can inspect how a workspace starts, not as a substitute for the fit matrix.

Stay on that free lane while evidence packs remain local and the agent cannot reach production. Graduate when a row in the matrix breaks, then pick paid API versus self-hosting by log ownership. If you evaluate MonkeyCode against this test, run it on one private-repo scenario before you keep the shared box for anything that needs a paper trail.

Limitations, and who should not use this

This heuristic underfits teams that already have a platform mandate, a data-residency rule, or a procurement list that forbids shared workspaces. It also underfits agents that deploy, migrate data, or touch production, because blast radius is then a safety problem rather than a billing problem. Do not use the classifier as proof that free compute is “safe enough” for regulated source.

I also would not use it as a performance bake-off. There are no tokens-per-second numbers here, and a passing pytest file does not mean a shared server will retain your session the way a disk you own will. If your real constraint is latency, offline models, or GPU placement, this article is the wrong artifact; measure those things directly. If your real constraint is an agent that assumes pytest while the repo uses a Makefile, start with Step 1 and stay there.

The conclusion does not change when a new agent glossary starts circulating. Keep the free shared lane until you cannot falsify the diff, then buy evidence, not vibes. Paid and self-hosted are how you own the paper trail. They are not how you look busy in a usage dashboard.