Favur ran unattended for two days on Meta Muse Spark 1.1, working from a Statement of Work for a 2048 game. It produced 131 files with tests across six sessions, with no human intervention at any point.
Unattended runs past a few hours fail in a small number of ways. A better model raises the ceiling on output quality but removes none of them, because they are properties of running a model in a loop for a long time rather than properties of the model. This covers what the harness does about each, including one it does not address.
False completion
A step reports success without having done the work. In a supervised run this costs a minute. In an unattended run downstream steps build on the result and the error surfaces hours later, past any clean rollback point.
Completion is gated on artifacts rather than on reports. Each step type has exactly one tool that can advance it, and validators run before the step closes.
The validators are the substantive part. Verification must cite at least two context documents. A commit is rejected without a commit hash and a non-empty list of committed files. Acceptance criteria are returned as structured fields and machine-checked rather than as prose asserting success. A claim is not accepted anywhere an artifact is available, because a model can assert anything but cannot produce a hash for a commit it did not make.
Work and judgment are separated. The agent running the development workflow writes no code; it sequences sub-agents and enforces the gates. Test creation is a distinct delegated step ahead of implementation, and the implementing step exists to make those tests pass. Failed verification routes back to implementation rather than forward.
Non-convergence
The failure that ends long runs is not a crash, which is detectable, but a loop, which presents as normal operation. The process is live, requests are being issued, tokens are being spent, and output is being produced.
The common form is not unbounded retry but oscillation, where an agent alternates between two candidate solutions that are both wrong and never terminates.
This is bounded at three levels. Each step carries a retry budget; exhausting it marks the step failed rather than retrying, and the terminal state cannot return to pending, so a failed step cannot re-arm and restart the cycle. A failed step routes to a declared failure path rather than terminating the run. Where a step can route back to an earlier planning stage, re-entry is counted and reaching the cap escalates. Each agent additionally carries circuit breakers counting cumulative failures across the whole run, which separates repeated failure of a single step from the same error recurring across different steps.
Context exhaustion
Running out of context partway through a unit of work leaves a half-implemented feature that the next session cannot safely resume, and the usual outcome is discarding work that was nearly complete.
Conversation history is summarized or dropped in tiers before each request as it approaches the model's context limit, escalating with fill level. Near the ceiling a request is hard-blocked rather than issued in a state where it cannot complete.
Task drift
Task drift is the one failure here the harness does not address. There is no semantic check on whether output remains relevant to the original task, and nothing evaluates the agent's work for topical relevance.
The constraints on drift are structural. A step cannot advance until its validators pass. The advancing tool is fixed per step type. Step requirements are injected on entry, before output is produced, so the evidence a step owes is known in advance rather than assessed afterwards. Strategy knowledge from completed planning steps persists for the remainder of the run, which removes re-derivation of settled decisions as a drift source.
Review quality
The practical failure in review is not that an automated reviewer approves bad code. It is that human review of a large generated diff degrades to skimming, so the verification burden moves rather than disappearing when code generation gets cheap.
That improves only if the automated review is trustworthy enough to rely on, which requires the reviewer to carry a cost for being wrong in either direction.
The reviewer must read the files before it can produce findings. Every finding carries a severity. A false-positive circuit breaker trips after three findings that are not real issues, which prices over-flagging rather than leaving it as a free hedge.
Concealed capability gaps
Capability-justified abandonment carries no scoring penalty. An agent that halts and reports a missing capability is not marked down for halting.
The reasoning is the incentive structure rather than the mechanism. Penalizing honest abandonment causes agents to conceal gaps, and a concealed gap is filled with fabricated work.
State across sessions
A context reset begins a session with no memory of prior state. Over a two-day run this means half-implemented work that is not identified as such, and a new agent resuming from a state it cannot interpret.
This run spanned six sessions. Checkpoints are written periodically and immediately on any agent completion or failure, and that state persists across session boundaries, so each session resumed with the set of passed and failed steps intact. Checkpointing does not block the run.
Output
The SOW specifies a creative variant rather than a clone of 2048, so the theme was model-selected. Tiles run from cool to unstable.
The run launched the built game and played it, producing moves, merges and achievement events, rather than reporting the test suite green. On an unattended run that is a stronger completion signal than a passing suite.
The repo is open source at https://github.com/awesoftsolutions/idea_meta-muse-2048-game
The full run recording is at https://www.youtube.com/watch?v=o0fQ7WQRTjo
The replay can be driven and stepped through at https://favur.dev/go/devto/the2048
Top comments (0)