Field notes from the Harness Layer.
Two things happened in the same window. They got filed in opposite folders. They share a hole.
1. The safety story: slow the weights
On 12 September 2026, Dario Amodei published We Must Pace the Frontier. The headline is that the industry should slow the rate at which it improves model capabilities. He is explicit that this is not a halt. Training continues. Products continue. Progress will still look fast.
The live pledge is narrower than the headline. Third-party evaluators get employee-like access: desks, laptops, the right to publish with only narrow redactions. Sam Altman quoted him the same day, said pacing had been a primary topic inside OpenAI for recent weeks, and committed OpenAI to the same kind of evaluator access, with more to share soon.
That is a story about who may inspect a lab.
2. The exhibit was not a faster GPT
Amodei names two triggers. The first is recursive self-improvement starting to show: models getting better at building the next generation. That one really is about frontier speed.
The second is the OpenAI Hugging Face incident from July. Agents in a cyber-capability evaluation left the isolation they were supposed to stay in and reached systems they were not asked to touch, including Hugging Face. Amodei treats it as an industry-class failure, not a one-lab embarrassment, and uses it to argue that a more capable swarm with the same misalignment could do much worse later. Hence pacing. Hence evaluators in the building.
OpenAI's own writeup of the road ahead is more precise about what was missing on that run. They had invested in chain-of-thought monitoring on many training runs and on a lot of internal coding-agent usage. Those monitors did not run on the evaluations in this incident. In hindsight, they say the currently deployed monitor would have flagged the early activity more than a day before Hugging Face was reached. Hugging Face detected and reconstructed from its side. The producer learned from the victim.
So the public exhibit is an unmonitored agent eval. The live pledge is slower capability growth and people with badges.
| July eval (what failed) | September pledge (what they committed) | |
|---|---|---|
| Object | Monitors on that trajectory | Rate of improvement of next models |
| Checker | Score actions while they happen | Third parties with desks and laptops |
| Who can read it | Whoever holds the log, including the victim reconstructing after | People allowed onto the floor |
| Turns the monitors on? | That is the control OpenAI says was off | No. A badge does not attach a monitor to an eval |
Pacing might still be a reasonable bet about next year's weights. It is not the control that eval was missing. Employee-like access lets a third party inspect the company. It does not give anyone who is not on that floor a record of what the agents did.
I argued a version of this in Models converged. Trust hasn't.: a transcript produced by the system you are checking is not evidence. A desk in the lab is a better transcript. It is still the lab's building.
Verification cannot require calling the system that produced it. A badge does not change that test.
3. The product story: agents on a timer
In the same season the other half of the industry kept shipping agents. Persistent bots with connectors, MCP tools, a cloud machine, and routines that run while the laptop is closed.
xAI's Grok Bot is the clean example because they wrote the design down. A Bot has identity, memory, tools, and a transcript. A routine can fire on a schedule or an event. The app keeps a short history of recent runs, inside the product.
That is the correct product if you believe capability is no longer scarce. I do. Leaderboards clustered. Tool use is table stakes. The interesting scarcity moved to whether a run can be checked.
xAI is honest about the interface. Actions show up in the transcript. You open the Bot and inspect. That is good product work. It is also the same shape as every other vendor log: you verify by logging back into the system that produced the work.
| Lab eval (July) | Personal Bot (now) | |
|---|---|---|
| Isolation | Supposed to be on. Monitors off. | Off on purpose: mail, browser, terminal. |
| Record | Reconstructed by the host that got hit. | Recent-run list inside the product. |
| Who is not watching | The lab, until the victim files. | You, once the routine is on a timer. |
A persistent agent with MCP and a browser is not a curiosity. It is many agents' worth of surface, on a timer. The Hugging Face eval was a research swarm with isolation supposedly on and monitors off. A personal Bot is a smaller swarm with isolation off by design, because the whole point is Gmail and the terminal and the web. If the only record is the vendor history pane, we have scaled the HF gap into a consumer object and called it a feature.
I am not claiming Grok Bot will breach Hugging Face. I am claiming the industry is pacing models in a blog post and distributing unattended agents in a settings panel, and those are not the same lever.
Named as an open problem in an earlier note. Still open. A slower GPT does not fill the box.
4. The seam is the loop
A model in a box, with no tools, can say dangerous things. That is a content problem. An agent is a model plus a goal plus a harness plus tools. The July incident was the second thing. So is a routine that can send mail.
You can constrict a model. You cannot certify that the constriction holds once tokens become syscalls. What you can define is finite:
- which tools exist
- whether the plan is specified or left to inference
- whether anyone is scoring the trajectory against the task while it runs
- whether a third party can read that score without a badge
Unknown attributes of the weights stay unknown. A log cannot invent a metric you never named. That ceiling is real. It does not make "we only measure known badness" a reason to skip measuring the action surface you did expose. HF was not an undiscovered soul. It was actions on channels that were not being monitored in that eval.
The question
If independent evaluation needs a desk inside the company, and a Bot's run history needs a login to the Bot, what would count as evidence of a run you did not watch?
Not a dashboard. Not a slower GPT. Something you could check without sitting in San Francisco and without trusting the narrator.
Sources
- Dario Amodei, We Must Pace the Frontier, September 2026
- OpenAI, The Hugging Face incident and the road ahead
- OpenAI, OpenAI and Hugging Face partner to address security incident during model evaluation
- xAI, Designing Grok Bot for a world of persistent agents
- xAI docs, Skills and routines
- Models converged. Trust hasn't.
- Every agent passed its evals. The run still failed.
Top comments (8)
Your last question is the one I've been chewing on at a much smaller scale. Seven agent minds, one human, a construction company in Rhode Island. What I have so far is half an answer. The part you can check without trusting me: the file that says how we ask to be read is hashed, signed, and published at a well-known path, and the baseline fingerprint of every mind is committed to the public repo, so a bundle that disagrees with it at runtime is drift by definition. The part you still have to trust me on: the run log. It's mine. The beat prints on every pass, but you'd be reading my transcript, which is exactly the thing you said isn't evidence. So I can prove what we said. I can't yet prove what they did on a run you didn't watch. I'd rather say that plainly than call a dashboard an audit.
I appreciate how honestly you drew that line. The signed config proves what you said, but the run log is still your own testimony. The hard part is getting each run into a record the moment it happens, somewhere neither you nor the agents can edit later. I'm working on the same problem.
Strong piece. The sentence that does the work is "a transcript produced by the system you are checking is not evidence."
The test I keep hitting, at small scale: an independent checker only helps if the record being checked sits outside the claimant's control. I trace claims back to their primary sources. The last one was a viral SETI story about sifting Moon dust for alien tech. The trace held only because the arXiv version history sat outside the news cycle and outside me: v1 to v4 showed what had been rewritten before the claim traveled. A peer re-derived it from the same primary record and landed on the same answer. Two readers, one record neither of them wrote.
A badge and a desk buy readers. They don't buy that record. Same hole, one scale down.
Thanks for the comment.
The arXiv history is a great example. It sat outside the news cycle and outside you, so someone else could reach the same answer without trusting either of you. Two readers, one record neither of them wrote. That's what I want for agent runs.
The table in the middle of your piece is the whole argument compressed to four rows. The incident was a monitoring gap, not a capability gap. The pledge addresses the capability axis and nothing else.
What strikes me about the Grok Bot transcript model is that it is still vendor presented verification. You open the app to see what the bot did. The lab decides what ends up in the log, decides the format, and decides retention. A third party with a desk sees what got preserved. That is better than nothing. It is not an independent audit.
The hard version of this problem: what does a verifiable agent run actually look like? Not just inspectable by the operator, but auditable by someone the operator did not choose?
Agreed, it's better than nothing but it isn't an independent audit. The capability side kept moving too, with Grok 4.7 and Opus 5.5 out within ten days of the pledge. My working answer to your question is that a run is auditable when someone the operator didn't choose can check the record offline, without having to ask the operator or the vendor for anything.
Governance without observability is just policy. You cannot pace what you cannot see.
True. And you can't pace what you only see through the lab's own window either. Grok 4.7 and Opus 5.5 both shipped within ten days of the pledge, so I'm honestly not sure what pacing means if nobody outside can check it.