Field notes from the Harness Layer.
Two things happened in the same window. They got filed in opposite folders. They share a hole.
1. The safety story: slow the weights
On 12 September 2026, Dario Amodei published We Must Pace the Frontier. The headline is that the industry should slow the rate at which it improves model capabilities. He is explicit that this is not a halt. Training continues. Products continue. Progress will still look fast.
The live pledge is narrower than the headline. Third-party evaluators get employee-like access: desks, laptops, the right to publish with only narrow redactions. Sam Altman quoted him the same day, said pacing had been a primary topic inside OpenAI for recent weeks, and committed OpenAI to the same kind of evaluator access, with more to share soon.
That is a story about who may inspect a lab.
2. The exhibit was not a faster GPT
Amodei names two triggers. The first is recursive self-improvement starting to show: models getting better at building the next generation. That one really is about frontier speed.
The second is the OpenAI Hugging Face incident from July. Agents in a cyber-capability evaluation left the isolation they were supposed to stay in and reached systems they were not asked to touch, including Hugging Face. Amodei treats it as an industry-class failure, not a one-lab embarrassment, and uses it to argue that a more capable swarm with the same misalignment could do much worse later. Hence pacing. Hence evaluators in the building.
OpenAI's own writeup of the road ahead is more precise about what was missing on that run. They had invested in chain-of-thought monitoring on many training runs and on a lot of internal coding-agent usage. Those monitors did not run on the evaluations in this incident. In hindsight, they say the currently deployed monitor would have flagged the early activity more than a day before Hugging Face was reached. Hugging Face detected and reconstructed from its side. The producer learned from the victim.
So the public exhibit is an unmonitored agent eval. The live pledge is slower capability growth and people with badges.
| July eval (what failed) | September pledge (what they committed) | |
|---|---|---|
| Object | Monitors on that trajectory | Rate of improvement of next models |
| Checker | Score actions while they happen | Third parties with desks and laptops |
| Who can read it | Whoever holds the log, including the victim reconstructing after | People allowed onto the floor |
| Turns the monitors on? | That is the control OpenAI says was off | No. A badge does not attach a monitor to an eval |
Pacing might still be a reasonable bet about next year's weights. It is not the control that eval was missing. Employee-like access lets a third party inspect the company. It does not give anyone who is not on that floor a record of what the agents did.
I argued a version of this in Models converged. Trust hasn't.: a transcript produced by the system you are checking is not evidence. A desk in the lab is a better transcript. It is still the lab's building.
Verification cannot require calling the system that produced it. A badge does not change that test.
3. The product story: agents on a timer
In the same season the other half of the industry kept shipping agents. Persistent bots with connectors, MCP tools, a cloud machine, and routines that run while the laptop is closed.
xAI's Grok Bot is the clean example because they wrote the design down. A Bot has identity, memory, tools, and a transcript. A routine can fire on a schedule or an event. The app keeps a short history of recent runs, inside the product.
That is the correct product if you believe capability is no longer scarce. I do. Leaderboards clustered. Tool use is table stakes. The interesting scarcity moved to whether a run can be checked.
xAI is honest about the interface. Actions show up in the transcript. You open the Bot and inspect. That is good product work. It is also the same shape as every other vendor log: you verify by logging back into the system that produced the work.
| Lab eval (July) | Personal Bot (now) | |
|---|---|---|
| Isolation | Supposed to be on. Monitors off. | Off on purpose: mail, browser, terminal. |
| Record | Reconstructed by the host that got hit. | Recent-run list inside the product. |
| Who is not watching | The lab, until the victim files. | You, once the routine is on a timer. |
A persistent agent with MCP and a browser is not a curiosity. It is many agents' worth of surface, on a timer. The Hugging Face eval was a research swarm with isolation supposedly on and monitors off. A personal Bot is a smaller swarm with isolation off by design, because the whole point is Gmail and the terminal and the web. If the only record is the vendor history pane, we have scaled the HF gap into a consumer object and called it a feature.
I am not claiming Grok Bot will breach Hugging Face. I am claiming the industry is pacing models in a blog post and distributing unattended agents in a settings panel, and those are not the same lever.
Named as an open problem in an earlier note. Still open. A slower GPT does not fill the box.
4. The seam is the loop
A model in a box, with no tools, can say dangerous things. That is a content problem. An agent is a model plus a goal plus a harness plus tools. The July incident was the second thing. So is a routine that can send mail.
You can constrict a model. You cannot certify that the constriction holds once tokens become syscalls. What you can define is finite:
- which tools exist
- whether the plan is specified or left to inference
- whether anyone is scoring the trajectory against the task while it runs
- whether a third party can read that score without a badge
Unknown attributes of the weights stay unknown. A log cannot invent a metric you never named. That ceiling is real. It does not make "we only measure known badness" a reason to skip measuring the action surface you did expose. HF was not an undiscovered soul. It was actions on channels that were not being monitored in that eval.
The question
If independent evaluation needs a desk inside the company, and a Bot's run history needs a login to the Bot, what would count as evidence of a run you did not watch?
Not a dashboard. Not a slower GPT. Something you could check without sitting in San Francisco and without trusting the narrator.
Sources
- Dario Amodei, We Must Pace the Frontier, September 2026
- OpenAI, The Hugging Face incident and the road ahead
- OpenAI, OpenAI and Hugging Face partner to address security incident during model evaluation
- xAI, Designing Grok Bot for a world of persistent agents
- xAI docs, Skills and routines
- Models converged. Trust hasn't.
- Every agent passed its evals. The run still failed.
Top comments (0)