Seventy-two days of this series changed what goes into the prompt. This one is about where the reply gets cut, and by whom — not the model, but the four lines of harness between the model and your parser.
Nothing here simulates language ability. The generation is data: 448 records in five output formats with the producer's states enumerated, so every rate below is an exact expectation over 448 × 8 states, never a sample.
Read it: https://dev48.infy.uk/prompt/day73-stop-sequences.html
The centrepiece needs no parameters
You are returning JSON. The advice is universal: set the stop sequence to your format's closing token, }.
} as a stop sequence, on JSON output |
rate |
|---|---|
| matches the reply's text | 100.00% |
| matches the reply's tokens | 14.29% |
| of those token-level fires, are early cuts inside the model's own data | 100.00% |
It fires on exactly the replies it must not touch, and misses every one it was installed for.
The mechanism is the tokeniser, and it is decided by the type of the last field:
{"total": "120.00"} → ends in ONE token: "}
{"total": 120.00} → ends in: }
A string-valued last field ends in "}, a single token that your } stop never matches. A number-valued last field ends in a bare } — which your stop does match, along with every } that closes a nested object earlier in the reply.
Then the universal advice dies
"Set the stop to your format's closing token" delivers 0.00% on all three terminated formats. Not "poorly" — zero.
The reason is not the tokeniser this time. It is the return convention: the commonest API behaviour removes the matched string from the response. So on the rare occasion the stop does fire correctly, at the real end of the document, it deletes the closing brace and hands your parser a truncated object.
The stop sequence that works is the one that fires after something you do not need — a newline following the document, a sentinel you emitted yourself — never the last character of the payload.
The general shape
A stop sequence is a string match on a token stream owned by a third party whose return convention you did not choose. Three layers, each of which can independently make the feature do nothing or do harm, and none of which appear in the code you wrote.
Independently verified: 3,875,933 verifier asserts, 173,090 in-page assertions, 0 failures.
Top comments (0)