Every prompt is sent with three numbers beside it, and the argument about them is always about which values to pick.
This is about what that argument assumes away: the values are applied in an order, two orders ship in mainstream stacks, and the same three numbers do not mean the same thing in both.
Run it: https://dev48.infy.uk/prompt/day76-sampler-order.html
The two orders
order A : temperature -> top_k -> top_p
order B : top_k -> top_p -> temperature
Temperature rescales the distribution. top_p truncates by cumulative mass. Apply temperature first and you change the mass that top_p is measuring, so the same top_p keeps a different set of tokens. Apply it last and top_p has already cut on the raw distribution.
Neither is wrong. They are different functions with the same three parameters, and nothing in an API request tells you which one you are calling.
Nothing here is sampled
Twenty-four next-token situations are data. The settings grid is enumerated. Every kept set is decided by running a real truncator — so each of the 26,208 cells is a count, never an estimate.
Exactly two numbers on the page are declared, both on sliders and both labelled where they are used, and no panel marked parameter-free reads either of them.
Why this matters more than a tuning argument
If you port settings between providers, or reproduce a paper's numbers, or copy a "known good" configuration from a blog post, you are assuming the order matches. When it does not, the failure is silent: the model still generates, the output is still plausible, and your top_p is quietly keeping a different set of candidates than the one the settings were tuned for.
The practical reading: sampler settings are only portable within one implementation. Across stacks they are three numbers with the same names attached to a different function.
Verifier 703,921 assertions, 0 failures.
Top comments (0)