DEV Community

Devanshu Biswas
Devanshu Biswas

Posted on

Deleting Two Thirds of Your Prompt Hits the Same 183 of 14,400 Cases as the Careful Fix, at 74.5 Tokens Against 229.7

Every prompting guide tells you to put your constraints in priority order. None of them says what precedence is for. It is for deciding which of your instructions gets broken — which only matters once no output can satisfy them all, and that is a question you can answer exactly.

Make the answer a small object (length, tone, format, language, code, greeting) and there are 576 of them. Make the request a small object too and there are 36. Every instruction is now a predicate: a set of situations it fires in, and a set of the 576 answers it permits.

function satSet(active, out){
  out.set(ALL_ONES);
  for (const a of active) bsAnd(out, a.m, out);   // commutative, associative
  return out;
}
Enter fullscreen mode Exit fullscreen mode

Feasibility is a set intersection, so it cannot depend on the order of your rules — asserted, not argued, bit-identical under reversal on 2,160 checks. Every number below: https://dev48.infy.uk/prompt/day69-instruction-precedence.html

The dumbest strategy ties the best one, exactly

Twelve rules, 400 prompts, all 36 situations each: 14,400 cases, exhaustive.

strategy tok/call broke a critical rule whole-set obedience collateral
as written 222.5 7.25% 94.3% 0.0%
sort the critical rules last 222.5 2.78% 94.3% 0.0%
print "(priority n)" on all twelve 270.5 3.83% 94.3% 0.0%
delete the conflicting rule 184.2 1.27% 88.5% 20.2%
attach an exception clause instead 229.7 1.27% 92.4% 5.3%
keep only the critical rules 74.5 1.27% 67.6% 68.8%

Three strategies land on the identical 183 of 14,400 cases, to the last one. That is not luck: once the critical rules are intact and satisfiable, a critical rule can only break where the critical rules contradict each other, which is a property of what you marked critical and not of what you did about it. 1.27% is a floor, and the cheapest way to reach it is a two-minute deletion.

Which is why the careful fix has to justify itself on the next column along. A single-column scoreboard would have made "delete nine rules" the best idea on the page.

The free re-order also beat the paid annotation outright — 2.78% against 3.83% at +48 tokens on every call, forever. Stacking them recovers 0.069 points.

What the measurement contradicted

Three things, and I built the page around the first one being wrong.

I wrote the numbering strategy expecting it to win, with re-ordering as the cheap control. It lost twice: worse than the free fix alone, and worth 0.069 points on top of it. The mechanism is legible once measured — a numbered prompt is still in whatever order it grew in, so its printed priorities fight its own recency, while sorting makes the two agree for nothing.

I assumed over-constraint was the failure mode. Twelve rules leave 70.3 of 576 answers standing and pin 0.00% of cases to exactly one answer. The story about long prompts squeezing the space to a point is not what happens. Contradiction is what happens: only 4.47% of rule pairs clash, but twelve rules is 66 pairs and 1 − (1 − 0.0447)⁶⁶ is 95.1%, measured 90.8%.

And an assertion failed once in 5,400. Chasing it turned up something the page had no idea about: the maximum score is shared by more than one answer in 100% of cases, because the instructions that fire never constrain every attribute. So the answer you get is decided by something not in your prompt at all. The assertion asked for index equality; it now asserts score-optimality and counts the ties.

5,920 in-page assertions, 38,773 in the verifier.

Part of a from-scratch series — one prompting technique a day, measured rather than described: https://dev48.infy.uk/promptfromzero.php

Top comments (0)