Most "AI fixes your issues" demos are a session: you run a CLI, it edits files, you look at the diff. I wanted the opposite: a standing service that sits next to a repository, picks up issues labeled bug, docs or chore, and opens a pull request — and that I can leave running without watching my LLM bill.
That's fly-swe, built on mini-swe-agent. Here's how it's put together, and what it cost on a real TypeScript monorepo.
One Machine, one process pair, one SQLite file
The whole thing is a FastAPI webhook receiver and a worker, run together by honcho on a single shared-cpu-1x Fly Machine with a 1 GB volume. Webhooks are verified (HMAC, timing-safe, fail-closed on an empty secret), de-duplicated by delivery ID, and written to a SQLite queue in WAL mode on the volume. The worker polls it.
No Postgres, no Redis, no broker. If the Machine restarts mid-task, orphan recovery picks the task back up: nothing lost, nothing run twice. The infra costs around $6/month.
The LLM never touches git
This is the invariant I care most about. The coding agent edits files and leaves them in the working tree. Then it emits a completion signal, and everything after that belongs to the harness: git add, commit, push and PR. The agent's shell gets a tokenless remote, no push credential, no gh, and an env allowlist with no secrets.
A side effect: when the target repo's AGENTS.md says "commit and open a PR", the agent follows it for style and tests but not for git, so nothing goes sideways.
Three budgets, enforced in code
-
max_iterations: a hard cap on agent steps -
max_cost_per_task_usd: per task -
max_daily_cost_usd: per repo per day
cost_tracking: strict goes one step further: if a step can't be priced, it isn't run.
When a run hits a limit, it doesn't just die. It force-pushes its partial work to a WIP branch, and the next trigger resumes from there, along with a summary of what was already tried. That summary costs zero LLM tokens.
"How I tested" is what the harness saw, not what the model said
After the agent's loop ends, the harness re-runs the test command the agent says it used, in the same environment. The PR's "How I tested" checkbox reflects that run. Anything the agent couldn't run (say, a suite that needs a database the sandbox lacks) is listed separately, labeled as the agent's report, and never checked off.
If verification fails, the harness says so on the issue and retries automatically. That happened on 3 of the 10 issues below.
Numbers from a real repo
10 consecutive issues (bugs and docs) on a TypeScript monorepo with an SDK and a docs site, using GLM-5.3:
- median cost per issue: ~$0.31; worst case $1.85 (225 iterations, two verification retries)
- typical run: ~80 iterations
- all 10 PRs were reviewed by a human before merge. fly-swe never merges.
What it isn't (yet)
Execution isn't sandboxed: the agent's commands run as a subprocess in the same container, mitigated by the env allowlist and the absence of credentials (SECURITY.md). A per-task sandbox is next on the roadmap. Ephemeral Fly Machines / Sprites are an obvious candidate.
GitHub is the only forge today, but it sits behind a provider plugin with a conformance suite. GitLab or Gitea would be a good first contribution.
Deploy in ~10 minutes: Quickstart · AGPL-3.0
Top comments (2)
Having the outer harness independently re-run the test command is the cleanest invariant here. Most agent loops run off the rails because the model grades its own work, confusing a truncated stdout or a silent exit with a passing suite. Separating execution from verification also explains your cost distribution. The jump from the thirty-one cent median to the dollar-eighty-five ceiling is almost entirely retry search. Once verification is deterministic, spending that budget on bounded retries becomes an actual calculation rather than an open-ended gamble.
Thanks, that's exactly the reason the harness owns verification. A model reading its own test output will rationalize a truncated log or a non-zero exit, and a separate re-run can't do that.
You're right about the outlier, and the issue thread shows it. The $1.85 run was three attempts at roughly the cost of a normal run each. Every time the re-run failed, the harness posted the failing command and its output on the issue and started a new attempt with that output as context. The third attempt passed. 225 iterations split across three attempts is about 75 each, which is right around the median.
One nuance, though: in that run, the first verification failure was the environment (a module the test runner couldn't resolve), not the fix itself. That's the weak spot of bounded retries: an attempt can be spent on something the agent can't fix from inside the sandbox. That's why retries are capped at 3 and the per-task cost cap still applies. The worst case is a known number, not an open-ended one.
Better environment diagnosis before spending an attempt is on my list. A per-task sandbox with the right toolchain preinstalled should eliminate most of that class of failure.