DEV Community

stepinto.dev
stepinto.dev

Posted on Originally published at stepinto.dev

The Heater Room So Far, in Seven Posts

There are seven posts about the heater room so far, and the eighth is out on Monday. This is the short version of the seven, with links for anyone who wants the detail. It is enough to start at the eighth.

The idea

A world model answers one question: if things are like this and I do that, what happens next? In code it is a function, predict(state, action), that returns the next state. Call it in a loop and you have a rollout, which is how you ask "what if?" without touching the hardware. The model is learned from logs of state, action and next state, which is what telemetry with the commands joined on already is.

The room

Every post since uses one simulated room: a room temperature, an outside temperature and a heater that is on or off. The log is 4,800 rows of those three fields, with outside temperatures between 2 and 12 degrees. From the fifth post on, the room has a door that opens and closes, and the log does not record it. The test room is at minus four outside, colder than anything in the log.

The four models

Model Weights What happened
Hand-written, from Forty Lines none I guessed the constants. It is 2.77 degrees out over thirty steps of the test room
Four weights, from Learning the Heater From Its Own Logs 4 Fitted by gradient descent. It recovers the room's constants and is 0.20 degrees out. It cannot follow the door
Latent, from Latent State Is the Fields You Never Logged 13 One private field, trained by walking each episode backwards. The field tracks the door from the step after it moves. Away from its log the model is wrong: with the heater off for twenty steps at minus four it says 5.25 degrees and the room is at 1.42
Physics-shaped, from Planning Against the Model 9 The same field, with the prediction limited to a leak times the gap between inside and outside, plus a constant from the heater. The field adds to the leak. On the same test it says 1.15

Claude and the room

How a World Model Differs From an LLM, and Why You Want Both gave Claude Opus 5.5 the same readings and questions. Claude fitted the room's physics from the readings and forecast the open door more closely than the latent model in two runs of three. Given the latent model as a tool, it chose a heating plan the model said would keep the room at 12 degrees. The real room fell to 10.6 in all 200 runs, because the model was wrong outside its log and nothing in its output said so. The physics-shaped model is the shape Claude described, and with that model as the tool the plan held in all 200.

A twenty-step rollout of either model takes about two microseconds. A forecast from Claude with the log pasted in takes 25 seconds and costs 35 cents. The post puts the world model and a planner on the device, and the LLM outside them to take the goal in and explain the result.

The planner

Planning Against the Model scores heating schedules by rolling the model forward under each one. The cross-entropy method finds the best twenty-step schedule in about 1,200 rollouts, under a millisecond, and the planner plans again at every step with the newest reading. Over 200 evenings it matches a thermostat tuned for that night's outside temperature and beats one tuned for a different night. With the door opening, the two are level.

What is still open

The model gives the planner one number per step and no measure of how far to trust it, so the planner plans to the target with no margin. At 3 degrees outside it leaves 1.3 readings an evening short of 18. On the minus four question it chose sixteen heater steps where the true minimum is fifteen. Fifteen fails in 75 of 200 runs, so sixteen is the better plan, but the model chose it because it is six tenths of a degree pessimistic at that point. It had no way to know fifteen was close.

Monday's post, When the Model Doesn't Know, fits eight copies of the model on resampled logs and uses how far they disagree as that measure.

The playgrounds run all of it in the browser: /heater/ for the four-weight and latent models, and /heater/2/ for the physics-shaped model and the planner.

Top comments (0)