Who Owns the Failed Agent Run? Score Free Shared Compute Against a Box You Control

The timeline keeps arguing whether models already write better code than most working software developers today. I care less about that ranking than about who can replay a failed agent loop. Free shared agent compute looks cheap until a run dies and you cannot replay it. I score recovery ownership first, and only then do I look at the invoice.

Who keeps the traces, the workspace, and the kill switch when a tool loop fails? If you cannot answer that question clearly, the zero-dollar option is not actually free for you. I used to compare runtimes by sticker price and advertised model access, and that ranking often lied. A failed agent loop burns tokens, queue time, and engineer attention that never appear on a pricing page.

Start from the failure, not from the catalog

Can you reproduce the same prompt, tools, and working files on another machine later tonight? If the answer is no, you are renting a black box rather than borrowing spare compute. Shared free pools remain useful for exploration and for throwaway prompt drafts that hold no secrets. They get expensive when the failure mode is a vanished job with no debugger hook attached.

Self-hosted boxes cost money and operations time, yet they often leave you the corpse of the process. Paid hosted runtimes sit in the middle, with nicer dashboards and fuzzier answers about who can copy a workspace. Which failure mode can your team actually afford during a normal on-call week at work? Write that failure down before anyone pastes a pricing URL into the design doc.

What I mean by recovery ownership

Recovery ownership is a boring checklist rather than a product vibe or a conference slogan. I want to know who can pause a run, copy the workspace, and replay the exact tool sequence later. I also want to know whether logs survive preemption, because shared queues love killing long or idle jobs. Those four questions decide more of my week than any model card on a homepage.

Here is the question I ask before I accept a free slot on a shared pool. If this agent corrupts a fixture or hangs on a tool call, can I inspect the filesystem without filing a ticket? If I cannot inspect that filesystem, I treat the runtime as a demo environment only. Production-shaped loops need a place where failure still leaves evidence you can hold.

A scorecard you can fill in one sitting

I use five rows and three columns so the argument stays on one page. The rows are secrets, traces, workspace, kill switch, and replay against known inputs. The columns are free shared compute, a box I control, and a paid hosted runtime. I score each cell 0, 1, or 2, where 2 means I can act without waiting on another team.

You should not treat my numbers as a benchmark, because I have not run a public bake-off. They are a conversation tool for your network, secret store, and retention rules. Fill the table with constraints you can defend in review, not with a vendor slogan from a landing page. If a cell is unknown, score it 0 until you prove the action yourself.

Concern	Free shared pool	Self-hosted box	Paid hosted runtime
You keep secrets in your vault	0–1	2	1–2
Traces survive preemption	0–1	2	1–2
Workspace is copyable after failure	0–1	2	1
You can kill a runaway tool loop	1	2	1–2
You can replay the same inputs	0–1	2	1–2

I add the column totals and I refuse to pick a winner until the lowest acceptable score is written down. What is your minimum score for traces, and what is your minimum for secrets? If free shared compute cannot meet those floors, the advertised price is irrelevant. Write the floors in the README so the next on-call person does not renegotiate them from scratch.

Numbered workflow I run before I pick a pool

1. Write the failure you fear in one sentence, including data that must not leak.
2. List the artifacts you need after a crash: traces, diffs, eval JSON, and the working tree.
3. Score each runtime against those artifacts using the 0–2 scale in the table.
4. Estimate retry cost as failed_runs * (queue_wait + tool_calls * avg_tool_latency).
5. Choose the cheapest runtime that still clears your floors, not the cheapest label on the sheet.

That fourth step is a sketch, and I do not pretend it is a finance model for procurement. I still write the formula down because teams argue about feelings when the units are missing. Replace the variables with log fields you already emit, and keep the formula visible beside the runner. If you cannot emit those fields, that absence is already a score of zero.

A labeled example that estimates retry cost from JSONL logs

This snippet is a proposal, and I am not claiming it ran in production or produced a published number. Drop it next to your agent runner if your logs already look like JSON lines with stable field names. Change the keys if your schema differs, and keep the script boring enough to read in review.

# proposal: retry_cost.py — estimate wait+tool cost of failed agent loops
import json
import sys
from collections import defaultdict

# Expected fields: runtime, status, queue_s, tool_calls, tool_latency_s
totals = defaultdict(lambda: {"fails": 0, "queue": 0.0, "tool": 0.0})

for line in sys.stdin:
    event = json.loads(line)
    runtime = event.get("runtime", "unknown")
    if event.get("status") != "failed":
        continue
    totals[runtime]["fails"] += 1
    totals[runtime]["queue"] += float(event.get("queue_s", 0))
    tool_calls = int(event.get("tool_calls", 0))
    avg_tool = float(event.get("tool_latency_s", 0))
    totals[runtime]["tool"] += tool_calls * avg_tool

for runtime, row in sorted(totals.items()):
    retry_s = row["queue"] + row["tool"]
    print(f"{runtime}: failed_runs={row['fails']} retry_seconds={retry_s:.1f}")

Run it like this after you export a day of logs from each runtime under test. Notice that I am not converting seconds into dollars, because that conversion belongs to your payroll and your cloud bill.

python retry_cost.py < agent_runs.jsonl

If a free pool shows fewer failed runs but huge queue_s, I treat the wait as a tax. If a self-hosted box shows more failures but tiny queue time, I can often iterate faster anyway. Which column would you rather debug at eleven at night, when the eval gate is red? That preference belongs in the scorecard, not in a hallway argument.

Where a free model and a free server still fit

I do not reject free shared compute as a category, and I reject using it as the only home a loop can live. Exploration, prompt drafts, and throwaway evals are a good match, especially when the workspace holds no secrets. Production-shaped agents, customer data, and long tool loops usually want a box I can pause. The scorecard exists so that move is explicit rather than accidental.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option, which I treat as one shared pool on the scorecard rather than a default winner. I use that option when the failure I fear is wasted time, not leaked credentials or an unreproducible patch. Score it with the same five rows you use for a laptop or a rented VM.

Commands I keep next to the scorecard

I want a local smoke test that does not depend on a vendor dashboard or a disappearing job page. This is a sketch of a tiny runner contract: write inputs, run the agent, and keep the workspace on disk. Replace the commented command with your real binary, and fail the check if the trace file is missing.

# proposal: local contract for an agent eval, not a vendor tutorial
mkdir -p /tmp/agent-replay/{in,out,ws}
printf '{"task":"rename flaky test","repo":"."}\n' > /tmp/agent-replay/in/task.json
# agent-run --input /tmp/agent-replay/in --workdir /tmp/agent-replay/ws --trace /tmp/agent-replay/out
test -f /tmp/agent-replay/out/trace.jsonl && echo "trace kept"

If the free server cannot leave trace.jsonl in a place you control, I score traces as zero. If your self-hosted box can keep that file after a crash, I score traces as two. The gap between those scores is the whole decision, not the catalog copy. Can your current pool pass this check on a bad afternoon without opening a ticket?

I also keep a tiny secret-hygiene grep beside the runner, because free pools and laptops fail in different ways. This is still a proposal, not evidence from a named production incident.

# proposal: refuse to start if the workspace looks like it holds live secrets
if grep -R -E 'AKIA[0-9A-Z]{16}|BEGIN (RSA |OPENSSH )?PRIVATE KEY' /tmp/agent-replay/ws; then
  echo "refuse: workspace looks like it contains secrets" >&2
  exit 2
fi

If that check fires, I do not send the tree to a shared pool, no matter how attractive the invoice looks. Self-hosted still needs the same check, because a box you own can leak just as loudly. Recovery ownership includes the right to refuse a run, not only the right to replay one. Do you currently have that refuse path in CI, or only in a wiki page?

Who should not use this scorecard

Do not use this method if you need a procurement-ready TCO model, because I am not offering audited cloud prices or hardware SKUs. Do not use it if your agents must remain inside a vendor you cannot leave, because portability is assumed throughout. Do not use free shared compute for regulated data, production secrets, or unattended write access to real infrastructure. Those are floors, not preferences you can bargain away after a demo.

I also would not use a free pool as the only CI lane for agent-generated patches landing on main. Continuous integration needs replay, and replay needs artifacts you still have after the job is killed. If your organization cannot store traces, fix storage before you argue about model quality. Otherwise you will debug folklore instead of files, and the scorecard cannot save that.

Limitations

This scorecard ignores model quality on purpose, because a clever model on an uninspectable runtime will still waste a week. I also ignored GPU mix, exact quotas, and hardware SKUs, because those numbers go stale and I will not invent them. Your network policy, secret manager, and retention rules will change the scores more than any article. Re-score when those rules change, not when a homepage banner changes.

The retry formula under-counts human time, which is usually the expensive part of a failed loop. If two runtimes tie on the sheet, I pick the one where I can attach a debugger without scheduling a meeting. That bias is mine, and you should write your own tie-break in the README. The useful question stays the same: who owns the failed run when you need it back?