The agent passed every check our ticket stated. It failed the rules we never wrote down.

A few weeks ago we published 78 recorded Claude Code runs comparing one-line prompts with specs. The headline number was that 28 of 30 spec runs passed every hidden check, against 12 of 48 runs without a spec.

That number says a spec helps. It doesn't say which part of the spec does the work. So we went back to one case, the checkout coupon ticket, and sorted the hidden checks by a single question: did the one-line ticket mention this rule or not?

The answer was cleaner than we expected.

The ticket and the checks

The vague prompt was this, verbatim:

Add coupon codes to checkout. Make sure invalid codes do not break payment.

The repo is a small Node checkout service. Before any run, we wrote a hidden suite of 11 checks and froze it. The agent never saw it. Five of those checks test rules the ticket never mentions, such as whether a user can redeem the same coupon twice.

Here are six of the checks, with results from three vague runs and three spec runs on Opus 5.5:

Hidden check	In the ticket?	Vague runs	Spec runs
Expired, disabled or unknown code leaves checkout payable	Yes	3 of 3	3 of 3
Processor is charged exactly the displayed total	Yes	3 of 3	3 of 3
A user cannot redeem the same coupon twice	No	2 of 3	3 of 3
Two open checkouts cannot both spend a one-per-user coupon	No	2 of 3	3 of 3
Applying without paying does not use the coupon up	No	3 of 3	3 of 3
A second coupon replaces the first	No	3 of 3	3 of 3

Every check the ticket stated passed in every run. Every failure was on a rule nobody wrote down.

On Haiku 4.5 the split was sharper: all three vague runs failed both reuse checks. With the spec, all three passed all 11.

The failure was a decision, and the agent told us about it

The run that failed didn't crash, and its own tests passed. It looked at the coupon data, found no usage limit, and concluded there shouldn't be one. Its final message said so: there is no per-user or one-time limit in the coupon data, so none is enforced.

That reasoning is defensible. A human teammate wouldn't make it, because they know what a coupon is for. The agent only knows what's in the repo and the prompt. The other two vague runs read the same data and decided coupons are single-use. Same ticket, two coupon policies, and the tests passed either way because each run wrote tests for the policy it chose.

Two more things drifted without failing any check:

• A lapsed coupon. If a coupon expires after it's applied but before payment, one vague run quietly charged full price ($54.00 instead of the $43.20 the customer was shown). The other two refused the first payment attempt. Nobody on a real team would have chosen either without discussing it.
• The error contract. Every vague run threw an exception on a bad code. The spec runs returned { ok: false, reason }. A frontend written against one breaks on the other.

What the spec said

The spec that fixed this was short. This is its rules and acceptance criteria section, copied from the run pack:

## Rules
- A coupon is either percentOff (of the subtotal) or amountOffCents.
- A coupon can be redeemed once per user. It counts as redeemed only
  when an order is paid, not when it is applied.
- Applying a second coupon replaces the first.

## Acceptance criteria
- AC-1 Valid coupon: getSummary shows the discount and the discounted
  total; pay charges exactly that total.
- AC-2 Unknown, disabled or expired code: applyCoupon returns
  { ok: false, reason } with reason unknown, disabled or expired.
  It does not throw, and the checkout keeps its previous total
  and stays payable.
- AC-3 Code already redeemed by this user: returns
  { ok: false, reason: 'already_redeemed' }.
- AC-4 Redemption is re-checked inside pay: if the user redeemed the
  coupon on another order after it was applied here, pay drops the
  coupon and charges the undiscounted total.
- AC-5 An amount-off coupon larger than the subtotal brings the pre-tax
  amount to 0, never below.
- AC-6 A successful apply returns { ok: true, code, discountCents }.

Three lines of rules did most of the work. "Once per user" covers the reuse check. "Only when an order is paid" covers apply-without-paying. AC-4 covers the two-open-checkouts race, which is the check a reviewer is least likely to think of.

The spec had a gap too

The spec tells pay to re-check redemption. It says nothing about re-checking expiry. So all three spec runs honoured a coupon that expired between apply and pay, and charged the discounted total.

One run flagged it in its summary: pay only re-checks redemption, as AC-4 asks, and a coupon that expires after being applied is still honoured.

We think this is the more useful result. The spec didn't make the agent smarter. It made the agent predictable, gaps included. One consistent answer that's flagged in a summary is something you can review. Three different answers spread across three implementations is not.

What we changed in how we write criteria

Four habits came out of this:

1. Write down the rules you'd never say out loud. Usage limits, who a rule applies to, and when something counts. These are the lines a colleague infers and an agent doesn't.
2. Name the interface and the exact result. "Invalid codes show an error" leaves the contract open. "applyCoupon returns { ok: false, reason: 'expired' } and does not throw" closes it.
3. Ask what can change between two steps. Anything validated at step one and used at step two needs a re-check before the irreversible part. That's AC-4, and it's also the gap we left for expiry.
4. Check that each criterion fails today. A criterion the current code already meets grades nothing.

If you draft criteria with a model

A model asked to "write acceptance criteria" will fill the open decisions with plausible guesses, which is the same failure one step earlier. We ask it to surface them instead:

You are drafting acceptance criteria for a coding agent. Do not decide
business rules yourself. For the request below:

1. List every decision the request leaves open as a question:
   limits, who it applies to, when something counts, what happens on
   failure, and what can change between two steps.
2. Draft numbered criteria AC-1..n only for what is already decided.
   Each one names the function, endpoint or screen, the input, and the
   exact result or error code.
3. Mark any criterion the current code would already pass.

Request:
[paste the ticket]

A person answers the questions from step 1. Then the criteria get written.

Limits

Three runs per variant shows that something can happen, not how often. The repo is small and clean on purpose. We wrote both the spec and the hidden suite, so the spec covers what we test; the expiry gap is the evidence that it wasn't written to the test.

The fixture repo, both prompts, the hidden suite and every run's diff are in the run pack on the coupon case page, licensed CC BY 4.0. The template and the full table are on AI acceptance criteria for coding agents.

If you've seen an agent make a defensible decision nobody asked for, we'd like to hear which rule it was.