The sparse-reward excuse is an authoring problem

Every eval conversation I've had this year hits the same wall. Someone wants to measure an agent on a long-horizon task — 40, 60, 100 steps — and someone else says you can't score that densely. Grading every step means a human or a judge model reading the whole trajectory, and that's expensive, slow, and noisy. So you run the task, check the final assertion, and get one bit back. Pass or fail.

I've built evals that way. They're cheap and they're nearly useless. When the agent fails at step 47, you have a terminal error and a transcript, and you spend the afternoon reading it like a detective. The signal you needed was at step 12, and you didn't collect it because collecting it looked expensive.

Here's what I've started to believe: dense scoring was never the expensive part. Authoring was. We've been blaming the wrong cost.

Think about what dense reward actually requires. To say "step 14 was progress," something has to know what progress means at step 14. If all you have is a task prompt and a final assertion, nothing knows that. So you hand the trajectory to a judge model and ask it to guess. That's expensive, and it's guessing, because the judge has no ground truth either. It's reading the same transcript you are, with the same missing information, and charging you per token for the privilege.

Tev makes the other move, and it's the one I keep coming back to. Instead of scoring the trajectory after the fact, you write the hidden ground-truth world spec first — the actual state of the world, the preconditions, the milestones, the terminal conditions — and then you render the environment from that spec. The environment isn't hand-built and then described in a doc that drifts. The spec is the source, and the environment is a projection of it. Once that's true, progress at any step is a function of the spec. You don't judge it. You compute it. Dense per-step rewards fall out as a byproduct, because you already wrote down what "closer" means.

Paper's here if you want the actual method: https://arxiv.org/abs/2610.18868v1

The reason this lands for me isn't the benchmark numbers. It's that it's the same discipline I keep failing to apply in production. I run long-lived agents. They make tool calls, they hold state, they run for minutes at a time. When one goes sideways, my instrumentation tells me it errored. It does not tell me it was drifting at step 9 and unrecoverable by step 20. I've got logs, I've got traces, and I've got no notion of expected state to compare against, so all of it is description, not diagnosis. I can tell you exactly what the agent did. I can't tell you where it went wrong.

The fix isn't a smarter judge model watching the agent in real time. I tried that. It's slow, it costs per step, and it's confidently wrong in exactly the places I need it to be right — the ambiguous middle, where the agent is doing something plausible but off-spec. The fix is writing down the world spec: what the state should look like, which invariants should hold, what a milestone looks like. Then checking observed state against it. That check is cheap. It's a comparison, not an inference. And it gives me a progress trace instead of a postmortem.

So the myth I want to bust is this: dense agent evals are too expensive to run. They're not. They're too expensive to improvise. If you don't have a spec, every step is a judgment call, and judgment calls don't scale — not in money, and not in consistency. If you do have a spec, every step is a lookup.

What I'd steal from Tev, concretely:

1. Write the world spec before the task prompt. State, invariants, milestones, terminal conditions. If you can't write it, you don't understand the task well enough to eval it, and no amount of judge model will fix that.
2. Render the environment from the spec. One source of truth. Hand-built environments drift from their descriptions, and then your rewards are scoring a world that doesn't exist.
3. Emit progress as a byproduct, not a second pass. If scoring is a separate pipeline, it rots the moment the environment changes. If it's a projection of the spec, it stays in sync for free.
4. In prod, make the same spec your instrumentation contract. Every tool call gets checked against expected state. You get a curve, not a vibe.

Where I'm skeptical: rendering from a spec is only as good as the spec. Under-specify it and you get dense rewards that are confidently wrong, and an agent that optimizes straight into the gap between your spec and reality. That's worse than sparse, because sparse failure is at least honest — it doesn't tell you you're winning. Also, spec authoring is real work. You've moved the cost, not deleted it. But moving it from per-run scoring to one-time authoring is the right trade, because the spec is reusable, reviewable, and versionable. A wrong spec is a bug you can fix. A noisy judge is fog you can't.

The other thing dense rewards buy you is debuggability. Sparse gives you pass/fail and a feeling. Dense gives you a curve, and the curve tells you where the agent stalls. That's the actual product. Not the score — the shape.

Next time someone tells you long-horizon agent evals have to be sparse because dense scoring is too expensive, ask them for the world spec. If they don't have one, that's the cost they're actually paying. They just haven't itemized it.