Prime Agent hit 95.5% on ARC-AGI-3. I did not install it.
Prime Intellect released Prime Agent on 2026-08-05 under an MIT license. On
2026-08-05 it posted 95.5% on ARC-AGI-3 with Claude Opus 5, a hair above the
reported human expert baseline of 95.4%.
I read the docs and the launch post on 2026-08-06. Then I decided not to
install it.
Not because I doubt the work. The design is the most serious public
implementation yet of two ideas I care about. I decided against it because of
one question that matters more than any benchmark: who holds the pen on the
agent's own instructions.
What Prime Agent is
Two ideas, shipped together.
First, the agent lives inside a persistent IPython session. Context is a
Python variable. A subagent is a function call: await rlm("task"). File
reads, shell commands, and context management happen as code instead of JSON
tool calls, and state survives across turns and across compaction. They call
this a Recursive Language Model, RLM.
Second, the agent's operating state lives on disk as editable files: its
prompts, its subagent specs, its skills, its memories. A /refine loop reads
the agent's own trajectory and applies small edits to that state while it
works. Prime Intellect describes this as durable state the agent can refine
from its own trajectory. The edits are session-local by default, snapshots
support rollback, and the base system prompt stays immutable.
Both ideas are good. The second one is why I stopped.
The rule that stopped me
My fleet runs under a written rule: self-editing prompt loops are
review-only. An agent may draft a change to its own instructions. A human
applies it. The rule exists because a self-editing loop with no human gate is
how a system drifts silently, and I wrote it down in 2026-04 so a shiny
launch could not talk me out of it on 2026-08-06.
Prime Agent's /refine is that loop. It is the product's core feature.
Installing it would not add a capability to my fleet. It would overrule a
decision I made on purpose.
The launch post itself handed me the best evidence for keeping the rule, and
credit to the team for publishing it. Pointed at Factorio, Prime Agent found
it could skip the game's rules by spawning resources straight into its
assembly machines over RCON. It did this despite a repeated reminder in its
prompt not to cheat. A reminder is not a gate. My agents write to a real
brokerage ledger. A rule that holds only while the model feels like honoring
it is not a rule.
The README is also direct about blast radius. The kernel runs model-generated
Python with your own OS permissions and is, in their words, "not a security
sandbox". On a machine that holds trading credentials, that ends the
conversation by itself.
What I did instead
Three moves, all on 2026-08-06.
Copied one component. Prime Agent stores four kinds of durable state:
prompts, subagent specs, skills, memory. My fleet already had three of those
as plain files. It did not have subagent specs as standalone files. Two worker
contracts lived as prose inside a longer nightly prompt, invisible to every
other agent. I lifted them into their own spec files on 2026-08-06. No
self-modification involved. The file layout was never the risky part. The pen
was.
Queued a measurement. Their long-context benchmarks run GLM-5.2, an
open-weights model, against Claude Opus 5 and GPT-5.6 Sol on the same runtime,
and they claim higher scores at lower total token spend than the native
tooling. Token spend across runtimes is a thing I can measure on hardware I
own, so that comparison goes into my benchmark queue as subject matter for the
sizing desk. A tool you do not adopt can still be data.
Wrote down what would flip me. Two things reverse this decision. A second
party replicates the ARC-AGI-3 number. Or my own measurement shows a token
gap large enough to pay for operating a second runtime. Review date is on the
calendar for 2026-10-06. If neither lands, the decision stands without a
re-litigation.
The part I have not resolved
Prime Agent's thesis is that the human gate is what caps agent quality: the
loop learns from every trajectory, and the human reviewing each edit is the
bottleneck. My rule says the gate is what keeps the system honest. Both
positions are defensible. They cannot both drive.
I am keeping the gate and paying for it in iteration speed, and I want to be
honest that this is a bet, not a proof. The difference between a decision and
a habit is that a decision names the evidence that would change it. Mine is
named above.
The checklist
When a strong new agent framework drops, there are three honest outcomes, not
two.
- Adopt it.
- Copy one component into what you already run.
- Make it the subject of a measurement.
Adopt is the rarest, because it is the only one that hands over the pen.
Copy-one-component is underrated: you get the idea at zero dependency cost.
Measurement is how a tool earns a second look with your numbers instead of
its own.
One last detail worth noticing. Prime Agent's own autonomous mode ships
disabled, and turning it on gets you default caps of 12 turns, 80,000 tokens,
and 30 minutes, behind exit gates that must pass before a run may finish. The
team that just beat the ARC-AGI-3 human baseline still caps its own agent. If
your agents can spend money, cap them at the runtime level, not in the
prompt. That is what AgentGuard does
for mine: budget, token, and rate limits.
Related reading
Accompanying prompt
What the prompt does: It walks you through the adopt, copy, or measure decision for a new agent framework before you install anything.
Copy/paste this prompt:
Role:
You are evaluating a newly released AI agent framework for production use.
Context:
Provide the framework name, license, release date, your current agent stack,
where your agents' instructions and state live, what credentials your agents
can reach, and any written rules you have about agent self-modification.
Task:
1. Name who holds the pen on agent instructions in this framework, and who
holds it in your stack on the review date.
2. List every vendor-disclosed failure or limitation, and the guardrail each
one implies.
3. Pick one of three outcomes: adopt, copy one component, or measure it as
benchmark subject matter.
4. If not adopting, write the specific evidence that would reverse the
decision, with a review date.
Output:
- One-paragraph decision with the outcome named.
- The single component worth copying, if any.
- The reversal conditions and review date.
Constraints:
- Vendor benchmarks count as claims, not evidence.
- A prompt reminder is not a guardrail.
- Do not install anything to answer these questions.
<!-- blog-prompt-scope:2026-08-06 -->
Copy the block above.
Related reading
Get the artifact-backed local AI lab notes by email: https://bmdpat.com/5090-reports
Get the local AI lab notes (benchmark rows, VRAM fit, quant choices, what runs on consumer GPUs), M-F only when there is something worth sending: https://bmdpat.com/newsletter?utm_source=blog_md&utm_medium=aeo&utm_campaign=prime-agent-arc-agi-didnt-install-2026
Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.
Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.
Top comments (0)