DEV Community: Devanshu Biswas

Your Withdrawal Rate Moves Time-to-FI by 31 Months. Your Balance, the Number in the Biggest Font, Moves It 3.

Devanshu Biswas — Thu, 17 Sep 2026 12:35:10 +0000

A freedom calculator answers three questions — how long the money lasts, what number ends the job, how far away that is — and they are one recurrence asked from two sides. Build it exactly, measure it honestly, and the finding is not that the arithmetic is wrong. It is that the answer is mostly a statement about assumptions, and the ones that move it most are the ones the genre does not show you.

Weekend Builds Vol 4 · #05 · the finale — Vol 4 is now complete at 5 of 5.

👉 Live, runs in your browser: https://dev48v.infy.uk/agentlab/vol4-05-freedom-calculator.html
👉 PUBLIC, MIT, dependencies = []: https://github.com/dev48v/freedom-calculator

python -m freedom_calculator                # runway, the FI number, time to FI
python -m freedom_calculator --sensitivity  # which input the answer rests on
python -m freedom_calculator --assumptions  # the model, and what it costs
python -m freedom_calculator --float        # why money is not a float here

Declared inputs, derived signs, measured spans

You declare five numbers. The scenario on the page: 42,000 balance, 3,200 a month out, 5,400 a month in, 5% real return, 4% withdrawal rate. From those: an FI number of 960,000 (25× annual spend) and 231 months to reach it.

Every direction below is derived on paper from the recurrence, then measured by bumping the input and watching the answer. The two are kept apart so they can be checked against each other, and the suite asserts they agree for every input at six bump sizes — plus a non-vacuity test, because a calculator that ignored its inputs entirely would pass a sign check perfectly.

±10% on…	months, down / base / up	span
what you spend a month	198 / 231 / 269	71
what you earn a month	271 / 231 / 202	69
withdrawal rate (usually hidden)	248 / 231 / 217	31
real return you assume	241 / 231 / 222	19
balance you already have	233 / 231 / 230	3

The row to sit with is the last one. The genre is arranged around a number worth about a tenth of what the hidden constant is worth.

There is also a row that must be exactly zero: runway does not depend on the withdrawal rate at all. It is a drawdown question. The derived sign is 0 and the measured direction must be 0 too — not small, not usually, exactly, at every bump size on every scenario. That identity is what stops the two halves of the calculator leaking into each other.

The argument people actually have, priced

The 4% rule is a rule about a number under argument — 3% for the cautious, 5% for the cheerful:

withdrawal rate	FI number	months
3%	1,280,000	278
4%	960,000	231
5%	768,000	198

512,000 of spread — 53.3% of the headline number — and 80 months of your life, from one constant most calculators will not let you type.

Forgetting to subtract inflation is worth 43 months: a nominal 8% instead of a real 5% brings the date from 231 months forward to 188, on which date you actually hold ~717,562 against a target of 960,000.

Money is an exact rational because the output is an integer

Not fastidiousness. The output is a month count that turns on a comparison, and a comparison decided by the last bit of a binary approximation is a month count decided by the last bit of a binary approximation.

Over 120,000 balances that are exactly n months of spend, floor(balance / spend) in doubles answers n−1 in 12,736 of them — 10.61%. Exact rationals: 0. Whole months, not cents.

The zero-net-burn case is worse. Summed as floats, the same basket of line items lands on a net burn of 2.27e-13 instead of zero, and a calculator that divides reports 43,980,465,111,040,000 months with a straight face. This package reaches the unbounded answer with a comparison and no division at all:

// runway and time-to-FI are the SAME recurrence, asked from two sides
//     b[m+1] = b[m] * (1 + r) + flow          flow = income - spend
// the zero-net-burn identity, decided BEFORE any division ever happens
if (cmp(add(mul(balance, r), flow), ZERO) >= 0) return { status: "unbounded" };

A test walks the syntax tree of runway() and asserts there is no division operator in it, with docstrings stripped first so it cannot pass on the paragraph explaining that the division is not there.

Same returns, reversed, a 17× different answer

The model has no sequence-of-returns risk in it. That is a limitation, and here is what it costs — with no RNG anywhere. One declared list of 30 yearly real returns, mean exactly the same 5% the smooth model uses, applied forward, then reversed. Start at the FI number, draw 4%:

order	ending balance
forward (crash at the front)	~127,381
reversed	~2,163,974
the smooth model's answer	~1,625,807

17.0× from order alone. Same numbers, same mean, same multiset, same product — and the control proves the product is identical: withdraw nothing and the two orders end on the identical cent, 4,176,967.52, because multiplication commutes. The entire gap is order, and the smooth model's answer sits between the two giving no hint which one you got.

The sequence was built to put the crash at the front. Stated, not hidden — the size of the crash is not the point. The point is that reversing a list changes the answer at all, which the smooth model says is impossible.

What this is not

Deterministic arithmetic on declared inputs. No RNG, no Monte Carlo, no historical data. Constant real returns, constant burn in today's money, no taxes, no lumpy years, no property, no state pension. The monthly rate is the annual rate over twelve rather than its twelfth root — the usual convention, and slightly generous. Anything past a 100-year horizon is reported as beyond the horizon rather than as a number, because a model that answers "3,847 months" is not being more precise, it is being less honest.

The JavaScript on the page is exact rationals over BigInt, checked against the Python's reference.json numerator for denominator, and against a third implementation in the verifier that simulates month by month instead of using the closed form.

50 pytest · 239 verifier asserts · 44 in-page self-checks · 0 failures. MIT, Python standard library only.

180 RAG Citation Defects Only a Free id Lookup Catches, 60 Only the Grader Does: Neither Subsumes the Other

Devanshu Biswas — Thu, 17 Sep 2026 12:34:28 +0000

A RAG answer says [c091] and everybody relaxes. There are two checks you can run on that bracket, and they catch different things.

Citation resolution is pure code: is c091 actually one of the chunks we assembled for this question? Zero model calls. Faithfulness is model-scored: does the assembled context support the claim?

👉 Live, runs in your browser: https://dev48v.infy.uk/ai/days/day80-citation-verification.html

The 2×2, and why both off-diagonals matter

420 seeded answers, 7 defect classes, 60 each:

	faithfulness says supported	says unsupported
resolution passes	120	60
resolution fails	180	60

If either off-diagonal cell were empty, one check would subsume the other and you could drop it. Both are populated by measurement: 180 answers where the identifier is wrong and the prose is fine, 60 where the identifier is fine and the prose is invented.

A pipeline running one check is not running a cheaper version of the other. It is running a different check whose blind spot is exactly what the other one is for.

The class people get wrong

stale_id: a real store id that was not in this assembled context. It looks real. It resolves perfectly against your vector store.

check	catches
resolution against the assembled context	60 / 60
resolution against the chunk store	0 / 60

Same check, one wrong set. That is the entire difference between a working guard and a decorative one:

// check 1 — resolution. No model. This is the entire thing.
const inContext = new Set(answer.assembled);          // ids WE assembled
const resolves  = answer.cites.every(id => inContext.has(id));
const pass      = answer.cites.length > 0 && resolves;

// the bug: checking the STORE instead of the assembled context
const wrong = answer.cites.every(id => store.has(id)); // c091 is real...
                                                       // ...just not here

Both blind spots, stated as numbers

Resolution never reads the claim. An unsupported sentence carrying an id that genuinely is in the context is a clean pass — 0 of 60 caught.

A context-scoped faithfulness grader is id-blind. Claim in, assembled chunks in, supported-or-not out. An invented id, a stale one, or no id at all does not change its input by one token — 0 of 180 citation defects caught. That is not inferred: the self-check replaces every citation with nonsense and measures the verdict move on 0 of 420 answers.

Neither number moves by tuning a threshold. Both are structural.

The residual neither one catches

60 answers cite a chunk that is in the assembled context and does not support the claim, while some other assembled chunk does. Resolution passes — the id is in the context. Context-scoped faithfulness passes — the context supports the claim. Both answer their own question correctly; neither question is "does this chunk support this sentence".

A citation-level grader does catch all 60, and costs one call per citation rather than per answer — 180 here against 420, and far worse on answers citing five sources each. It also cannot run on a citation that did not resolve: you cannot hand a grader a chunk you could not find. Third layer, never a replacement.

What the free check is worth

	defects caught (of 360)
faithfulness alone	120
resolution alone	240
union	300
plus citation-level	360

Zero false alarms from either check on the clean class. Resolution costs 360 comparisons and 0 model calls over the whole corpus; grading everything costs 420 calls; running the free check first and grading only what survives costs 180 — 240 model calls removed with the union catch rate unchanged.

One cost that is not money: everything short-circuiting skips sits in the cell where the answer text was fine. For those the repair is re-citing, not re-answering, and an ordering that dumps them into one undifferentiated "failed" bucket sends correct answers back to be rewritten.

The limitation to read first

The entailment check here is a stand-in, not a model — a stated token rule over 12 disjoint topic vocabularies, which separates supported from unsupported with no overlap at all. Real graders are wrong a great deal, and that error is part of why the free check is worth keeping. So read the faithfulness column as a ceiling and the resolution column as exact, because that one is code. The corpus is synthetic, one claim per answer, one citation per claim, no multi-hop, no reranker, no abstention. And nothing here asks whether the source is any good.

38 in-page checks, 138 verifier asserts, 0 failures.

Pack Four Docs Without the Block Mask and Doc 4 Spends 83.78% of Its Attention on Documents It Never Saw

Devanshu Biswas — Thu, 17 Sep 2026 12:33:46 +0000

Sequence packing concatenates short documents into one row so you stop paying for padding. On the length distribution here that lifts utilisation from 27.98% to 99.81%.

But attention inside a row is global. Without a block-diagonal mask, the fourth document in the row spends 83.78% of its layer-1 attention on documents it has never seen — worst single token 94.90%.

👉 Live, runs in your browser: https://dev48v.infy.uk/dl/day80-sequence-packing.html

The ground truth is declared; everything else is measured

Every document is first pushed through the stack on its own, ordinary causal mask, positions 0…L−1. That unpacked run is the ground truth, and every packed configuration is scored against it token by token, coordinate by coordinate — not against a loss curve, which is exactly the instrument that cannot see either bug.

The only thing that differs between a correct packed row and a broken one:

const allow = cfg.block
  ? (i, s) => s <= i && p.docIds[s] === p.docIds[i]   // block-diagonal ∩ causal
  : (i, s) => s <= i;                                 // causal only: the mask bug
const posIds = cfg.reset ? p.local : p.tokens.map((_, i) => i);

The tolerance is a prediction, not a fudge

With the mask and per-document position reset, the packed row reproduces the unpacked run to 2.22e-16 — across five independently seeded models, every token, every coordinate.

It is not exactly zero for a stated reason. A batched attention kernel subtracts a row maximum before exponentiating, and takes that maximum over the whole causal prefix of the row, not over the current document. Subtracting a constant leaves a softmax mathematically unchanged, so this is a reassociation that moves the last bit and nothing else. The engine reproduces that kernel behaviour rather than hiding it.

The verifier then tests the explanation instead of trusting it: an independently written attention stack that subtracts no maximum at all gives the identity at exactly 0. So the residual is the shift, and nothing else. The asserted tolerance is 1e-12 — four orders above the residual, twelve below the smallest real bug.

Two independent bugs, and the arithmetic proves they are two

configuration	max Δ vs unpacked	worst leaked mass
block mask + position reset	2.22e-16	0
no mask, positions reset	0.3607	94.90%
mask, no position reset	0.7607	0
neither (naive packing)	0.7773	94.90%

0.7773 is neither the sum (1.1214) nor the max (0.7607). They interact, so you cannot subtract one to estimate the other.

Two controls separate them. Zero the position table and the position bug has to vanish exactly — there is nothing left for a wrong id to index — and it drops to 1.11e-16 while the mask bug is untouched at 0.3498. And across a growing-neighbour sweep the two respond completely differently to the same change: the mask error correlates with leaked mass at r = 0.9814, the position error at r = 0.1384, because a position offset is already wrong once the neighbour has a single token.

Nothing throws, and here is why

Every attention row still sums to 1 to 4.44e-16 in all five configurations. A masked softmax renormalises over whatever it is left with, so the shape is right, the loss falls, and the model just learns something slightly wrong.

Both bugs also leave the first document in the row bit-identical — exactly 0, not nearly zero. Causal attention cannot look forward, and its positions start at 0 whether or not anyone resets them. Spot-check a packed batch on the first sequence and both bugs are invisible.

Three honest subtractions

Damage is not monotone in neighbour length. The foreign mass is strictly increasing by construction, from 0.2261 to 0.8772 mean. The error is L×gap and the second factor moves with the content of whatever tokens get appended — there is a visible dip at n = 4. "More neighbour, more damage" is a trend here, not a law.

Length-sorted batching already gets most of the win. Bucketed batches reach 99.06% utilisation against packing's 99.81%. The 3.57× is measured against arrival-order batching, which is the baseline packing is sold against but not the only alternative.

Packing does not make attention cheaper unless the mask does. Attention cost is quadratic in the row, so a packed row computed densely costs 1.027× the padded baseline — slightly more. It is the block-diagonal structure, exploited by a kernel that skips the off-diagonal blocks, that brings it to 0.1625×. The mask is not only what makes packing correct; it is what makes it fast.

The models are not trained, positions are a learned absolute table rather than RoPE, one head, two layers, float64 throughout. So which bug is larger is a property of this toy's scales and is not a general claim. The claim is that both are non-zero, independent, and invisible to the loss.

38 in-page checks, 126 verifier asserts, 0 failures.

Model A Beats B on Every Slice and Loses 57.60% to 79.06%: 9,510 Reversals, 0 on a Shared Mixture

Devanshu Biswas — Thu, 17 Sep 2026 12:33:03 +0000

Two models, one benchmark, three slices. Model A is ahead on all three — 96.0 / 60.0 / 50.0 against 88.0 / 50.0 / 40.0.

Pooled, A scores 57.60% and B scores 79.06%. A 21.5-point defeat assembled entirely out of victories.

👉 Live, runs in your browser: https://dev48v.infy.uk/ml/day80-simpsons-paradox.html

Declared, not sampled — and no winner decided by a float

Every count on the page is a declared integer. Every accuracy is an exact rational, and every comparison is settled by BigInt cross-multiplication, so a reversal is proved rather than observed through rounding:

// every winner is decided by cross-multiplication, never by a float
const rcmp = (a, b) => { const l = a.n * b.d, r = b.n * a.d;
                         return l < r ? -1 : l > r ? 1 : 0; };

There is no p-value anywhere, because nothing is estimated. What is measured is an exhaustive sweep.

The identity the whole thing rests on

Pooled (micro) accuracy is not a different kind of quantity from the per-slice rates. It is their mean, weighted by slice size:

micro(M) = Σ c_g / Σ n_g = Σ (n_g/N) · (c_g/n_g)
macro(M) = (1/G)  ·  Σ (c_g/n_g)

The two metrics differ in exactly one place — the weights — and the entire paradox lives there. A weighted mean with fixed non-negative weights is monotone in its inputs, so per-slice dominance would be inherited by the pooled number. The reversal requires the two models to carry different weights.

Which is why macro and micro disagree in sign in 9,510 of 9,510 reversals found. Not usually. All of them.

The smallest reversal in existence is nine observations

Sweeping every two-group table with per-cell n from 1 to 8 — 3,748,096 tables, of which 774,400 put A strictly ahead in both groups — gives 9,510 reversals. The minimum total sample across all of them is 9, reached by exactly 4 tables:

group    model A        model B
g1       1/1 = 100%     2/3 = 66.7%
g2       1/4 =  25%     0/1 =  0%
pooled   2/5 =  40%     2/4 = 50%

Four cells and nine data points are enough to rank two models backwards. There is nothing subtle or large-sample about it.

At the other end: the largest pooled reversal found is 5/9 against 2/9 — 55.6 points — and the largest per-group lead that still loses is 3/8, so A is ahead by at least 37.5 points in every group and loses anyway. There is no margin of per-group superiority large enough to make a pooled scalar safe to read.

The negative result is the one worth keeping

Re-run the same sweep with the two models forced onto the same slice mixture:

constraint	comparisons	reversals
the full sweep	774,400	9,510
same n in every group	14,400	0
same slice proportions, totals free	32,544	0
same total n only, mixtures free	72,816	502 (smallest at 10 obs)

Zero is not "rare". With the weights shared, pooled accuracy is a weighted mean using the same weights for both models, and monotonicity does the rest. Matching only the proportions is already enough. Matching only the sample size is not.

Every zero is printed next to the number of comparisons behind it, so an impossibility is distinguishable from an empty search — and the degenerate control (one group, where a reversal would be a contradiction) duly reports 0 over 1,936 comparisons.

What it does not say

A third slice does not make it cheaper: each extra group adds two more cells that must each be strictly won, and the smallest three-group reversal costs 13 observations against nine. The sweep covers two groups at per-cell n ≤ 8, so the minimum is exact for two groups and a claim about nothing else. And the page never says which number is right — macro and micro answer different questions, and choosing needs a deployment mixture a benchmark does not carry.

The narrow claim is the defensible one: a pooled scalar published without the slice sizes that produced it is not a ranking anybody can check.

27 in-page checks, 134 verifier asserts, 0 failures.

HTTP Range Has Three Outcomes, Not Two: 32.03% Ignored, 54.95% Partial, 13.02% 416 Over 10,116 Calls

Devanshu Biswas — Thu, 17 Sep 2026 12:32:22 +0000

The Range header is behind every resumed download, every seek in a <video> tag and every byte-serving PDF viewer. It has three outcomes, and servers routinely collapse them to two.

Over an enumerated corpus of 1,124 headers × 9 representation lengths = 10,116 decisions: 3,240 ignored → 200 (32.03%), 5,559 → 206 (54.95%), 1,317 → 416 (13.02%). A malformed field is 2.46× more common than an unsatisfiable one — and those are exactly the pair you conflate by treating "invalid" as "unsatisfiable".

👉 Live, runs in your browser: https://dev48v.infy.uk/solve/day80-http-range-parser.html

Invalid is not unsatisfiable

the field does not parse   -> 200, whole body      (it is ignored, not rejected)
it parses, nothing fits    -> 416 + bytes */len    (nothing else ever gives 416)
it parses, something fits  -> 206, the rest dropped

RFC 7233 is explicit that a Range a recipient cannot parse is dropped — which means an ordinary 200 with the whole body. 416 exists for exactly one situation: the field parsed cleanly and every member starts past the end.

So bytes=9999- against a 1,000-byte file is a 416, and bytes=abc is a 200. Getting those backwards is how a byte-serving endpoint answers 416 to clients that should have received the whole file.

A single backwards member poisons the whole header

This is the one almost everybody gets wrong. bytes=0-99,7-3 is not "one good range and one bad one":

// the rule everyone gets wrong: this poisons the FIELD, not the member
if (last !== null && first > last)
  return bad("first-byte-pos " + first + " is greater than last-byte-pos " + last);

first > last makes the field syntactically invalid, so the whole thing is ignored and the server sends 200 with the entire body. The most obviously "bad" range in the list lands in the 200 column, not the 416 one.

Only first-byte-pos can push you off the end

// only first-byte-pos can make a range unsatisfiable - last merely clamps
if (spec.first > len - 1) return null;
const last = spec.last === null ? len - 1 : Math.min(spec.last, len - 1);

A suffix longer than the representation clamps to the whole representation. A last past the end clamps to len-1. Which is why, against the same 1,000-byte file, bytes=900-9999 is a 206 and bytes=9999- is a 416.

Three spellings of "give me everything" — bytes=0-, bytes=-len and bytes=0-(len-1) — resolve to one identical range at every non-zero length. At length 0 the identity is scoped out rather than fudged: there is no whole representation to name, and 0-(len-1) spells 0--1, which is not a legal spec at all.

The middle case is the majority, and it shrinks twice

When some members fit and some do not, the unsatisfiable ones are dropped and what remains is served as a 206. Not a 416, not an error. 2,152 decisions land in that case.

Coalescing then shrinks the answer again: 5,044 of the 5,559 206s emit fewer parts than the client wrote. Adjacency is the subtle half —

// adjacent, not merely overlapping: 0-99 and 100-199 are ONE part
if (prev && s[i].first <= prev.last + 1) prev.last = Math.max(prev.last, s[i].last);

0-99 and 100-199 share no byte, but they are one contiguous run, and a server that ships them as two multipart parts is paying boundary overhead for nothing.

Six invariants over every one of the 10,116 decisions, 0 violations — including the one that actually bites: the emitted byte set is rebuilt as a plain set union of the satisfiable members and compared in both directions, rather than against itself. And 416 is asserted as an if-and-only-if, so neither direction can rot alone.

What this covers, and what it does not

It is a parser, not a complete server. Out of scope, each changing real behaviour: If-Range and the conditional dance that decides whether a range is honoured at all, Accept-Ranges, the actual multipart body bytes with their boundaries and CRLFs, Content-Encoding interacting with byte offsets, and ranges against a representation whose length is not yet known.

Three strictness choices are documented rather than hidden, because a different implementer could defensibly go the other way: an empty list element such as bytes=0-1,,4-5 is treated as invalid where RFC 7230's list rule would let a recipient skip it; whitespace around the = is rejected because the grammar has none there; and bytes=-0 is treated as unsatisfiable, since a zero-length suffix selects no bytes at all.

37 in-page checks, 202 verifier asserts, 0 failures. Vanilla JS, one file, no build step.

The Same 8,514-Token Prompt Bills 0.1113x or 1.2474x — an 11.21x Gap From One 14-Token Line Moving

Devanshu Biswas — Thu, 17 Sep 2026 12:31:40 +0000

Providers cache your prompt by exact token-prefix match. What you get back is the longest common prefix between this request and the cached one, and nothing after the first difference — however stable the rest of it is.

So one volatile line decides the bill. Take a 14-token Current date: …, which is 0.16% of an 8,514-token system prompt, and walk it through all seven insertion points. Placed last, the steady bill is 0.1113× the uncached price. Placed first, it is 1.2474× — above 1.000×, so caching there costs more than not caching at all. That is 11.21× between two prompts with identical tokens in a different order.

👉 Live, runs in your browser: https://dev48v.infy.uk/prompt/day80-prefix-caching.html

What is measured and what is declared

Declared, and stated above the first number: the three price multipliers — a cache write costs 1.25× a fresh input token, a cache read 0.10×, fresh input 1.00× — plus a $3.00/Mtok input rate for turning multiples into money. Change two constants and every figure moves with them.

Measured: the prefix matcher, the cumulative arithmetic, the break-even algebra and its agreement with a request-by-request simulation, and every count. There is no sampling anywhere, and nothing here simulates language — the engine is an ordered list of segments with declared token lengths.

The prompt is invented. The page says so.

The hit is a cumulative sum, and that is the whole mechanism

volatile field at index	sits before	cache hit	steady bill	break-even N*
0	tool definitions	0	1.2474×	never
1	operating rules	3,120	0.8304×	3
2	output schema	4,960	0.5844×	2
3	house style guide	5,520	0.5096×	2
4	worked examples	5,930	0.4548×	2
5	escalation policy	8,170	0.1554×	2
6	— (it is last)	8,500	0.1113×	2

Put the field at index p and the hit is exactly the cumulative token count of segments 0…p−1. That column is measured by running the matcher; the cumulative sum is computed separately; they agree at every position.

The assertion that carries the page is monotonicity: moving the field one segment earlier can never increase the hit. Checked at every insertion point of this fixture and of a 48-segment one — 56 positions, not a few sampled slots.

Break-even is closed-form, and its denominator is the interesting part

cold = Ta*w + Tb*b                 first request writes everything
warm = H*r + (Ta-H)*w + Tb*b       every request after it

cold + (N-1)*warm  <=  N*(Ta+Tb)*b
  cold - warm      =  H*(w - r)                  the tail cancels
  (Ta+Tb)*b - warm =  H*(w - r) - Ta*(w - b)     and again

  N* = ceil( H(w-r) / ( H(w-r) - Ta(w-b) ) )     denominator <= 0  =>  never

The user's actual turn cancels out of the ratio entirely, which is why it does not appear. And a negative denominator means no number of requests ever repays the write — placed first, the page checks that out to 1,000,000 requests and it is still dearer.

Placed last, the loan is repaid on request 2. The derived N* matches the request-by-request simulation at all 56 positions, and the check is two-sided: at N* the cached run is no dearer, and at N*−1 it really is — so N* is the smallest such N, not merely some N.

The threshold that does not depend on your prompt

Set that denominator to zero and the fixture drops out. What is left is a property of the price model alone: a cached prefix under (w−b)/(w−r) = 21.74% of the cacheable prompt never repays its own write, at any N. The self-check pins the number printed in the prose to the one the formula derives.

Two fixes that look equivalent; one does nothing

The instinct on a cache that never hits is to make the volatile thing smaller — shorter date format, an id instead of a name, drop the seconds. Hit length does not depend on the volatile field's token count at all. At position 0 the hit is 0 whether the field is 1 token or 800, because the matcher stops at the first differing token and the field's own length is entirely on the far side of that stop.

Same corollary for several volatile fields: the hit is decided by the earliest one. Everything after it is already free to change, and tidying those is worth exactly nothing.

What this is not about

Not breakpoints, TTL or minimum-prefix tiers. All of those are held fixed here — they are a different page's subject. This one varies exactly one thing, where the volatile field sits, and prices the consequence. Also not modelled: cache lifetime and eviction, concurrency, and which prefix a provider actually keeps when several are live. And nothing here is about which instruction wins when two conflict — segment order changes the bill; it is not an argument about precedence.

19 in-page checks, 284 verifier asserts, 0 failures — plus 4 deliberate mutations of the engine, 4 caught.

A Roving Tabindex Widget Costs 1 Tab Stop at Every Size 1..50; the Default Markup Costs 1,275

Devanshu Biswas — Thu, 17 Sep 2026 12:30:59 +0000

Tab is for moving between widgets. Arrows are for moving inside one. A toolbar, listbox or menu should add exactly one stop to the page's tab order, not one per item.

Swept over item counts 1…50, the roving widget contributes 1 tab stop at every single size — 50 summed. The default markup, which is what you get by writing no tabindex at all, contributes 1…50 — 1,275 summed.

👉 Live, runs in your browser: https://dev48v.infy.uk/design/day80-roving-tabindex.html

The whole pattern, and the whole correctness story

// the whole pattern, in two lines
const tabIndexOf = (state, i) => (i === state.activeIndex ? 0 : -1);
const next = (items, from, dir, wrap) => /* first ENABLED index that way */;

// and the whole correctness story, in one
items.filter((_, i) => tabIndexOf(state, i) === 0).length === 1

Not "usually one". If it is ever zero, Tab skips the widget and a keyboard user can never get in. If it is ever two, the widget has quietly grown a second tab stop and the bug is invisible to everyone using a mouse.

Enumerated, not sampled — and the reason is one design decision

Over 240 enumerated configurations (item counts × disabled patterns × wrap), 8,892 distinct reachable states and 79,968 key events:

assertion	checks	held
exactly one item at tabindex 0, after every key	80,208	80,208
the active item is never a disabled item	80,208	80,208
the active index is in range, or exactly −1	80,208	80,208

That walk is exhaustive rather than sampled, and it is exhaustive only because of one decision: buffer expiry arrives as a Timeout key, not a setTimeout. A state machine that reads the clock cannot be walked — you can only poke at it. Making time an input is what turns "we tried a few" into a count.

The edges are in by construction, not by luck: the first item disabled, the last item disabled, everything disabled but one. 54 of the 240 configurations have nothing focusable at all, and those are scored against a different rule — zero tab stops, not one — because "exactly one" is the wrong assertion for a widget with nothing to focus.

The qualifier that usually gets deleted instead of the bug

Three identities, 702 checks, 0 broken:

wrap=true: one ArrowRight per enabled item returns you home — the moves are a cyclic group (234 starting points).
wrap=false: both ends absorb; arrowing past the last enabled item is a no-op (186 checks).
ArrowRight then ArrowLeft returns to the same index whenever no end was crossed (282 pairs).

That last qualifier is load-bearing, and 186 pairs are deliberately excluded because an end really was crossed. Under wrap=false the pair is genuinely not reversible at the ends: the first press was absorbed and the second one moves. Asserting reversibility without the qualifier fails on a correct implementation — which is how the qualifier gets deleted instead of the bug.

Typeahead is a search, and it has its own edges

Type b and you land on the next item starting with b; type it again and you cycle; type b then o and you get "Bo…", not the next o. So the buffer decides the needle. Two consequences the page counts rather than hopes for: across 1,302 typeahead searches, the number that landed on a disabled item is 0 — a search that puts you somewhere you cannot act is worse than one that finds nothing — and typeahead wraps even when the arrows do not, because clamping is a statement about a direction and a search does not have one.

aria-activedescendant is the other answer, not the wrong one

It ties roving at 1 tab stop, at every size. The difference is where focus is. With roving the item is really focused, so :focus and :focus-visible style it for free and document.activeElement is the thing the user is on. With activedescendant the item is never focused: you draw the ring yourself, and the element the browser thinks is focused and the element the user is on are two different elements, permanently. What that buys is items that never need to be focusable — which is why comboboxes reach for it, since one cannot move DOM focus into its list without taking it out of the input.

What this models and what it does not

The engine is the keyboard state machine and nothing else. Nothing on this page was measured in a browser — the keystroke figures are counts of engine transitions, not timings. Out of scope, each of which is real work in a real widget: actually applying the tabindex and calling .focus(); items added or removed while the widget is focused, where the active index goes stale; RTL, which changes what ArrowLeft means; 2-D grids, where Home/End have both a row meaning and a grid meaning; and Page Up / Page Down.

23 in-page checks, 112 verifier asserts, 0 failures.

The Mover Loses on Exactly 11 of 299 Piles, and Four Sensible Openings Win 0 of the Other 288

Devanshu Biswas — Thu, 17 Sep 2026 12:30:16 +0000

Fibonacci nim is one pile of counters. You open by taking anything from 1 to n−1 — never the whole pile — and after that nobody may take more than twice what the opponent just took. Whoever lifts the last counter wins.

The player to move loses exactly when the pile is a Fibonacci number. Over the 299 piles from 2 to 300 that is 11 of them, and they are the obvious 11: 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233.

👉 Live, runs in your browser: https://dev48v.infy.uk/game/day80-fibonacci-nim.html

Two halves that are never allowed to talk to each other

The solver knows the rules and nothing else. There is no Fibonacci number anywhere in it.

// the definition, memoised - no theory in here at all
function solveState(rem, cap) {
  cap = Math.min(cap, rem);
  for (let k = 1; k <= cap; k++) {
    if (k === rem) return true;              // took the last counter
    if (!solveState(rem - k, 2 * k)) return true;
  }
  return false;
}
firstPlayerWins(n) = solveState(n, n - 1)    // never the whole pile

The other half is zeckendorf(n) — every positive integer as a sum of non-consecutive Fibonacci numbers, by greedy subtraction. It knows arithmetic and has never heard of the game.

0 disagreements across all 299 piles. And the same rule holds one level down, over the whole state space rather than just the openings: for all 45,150 reachable (remaining, cap) pairs, the mover wins if and only if the smallest Zeckendorf term of what remains fits under the cap — 0 mismatches against the solver.

The strategy is replayed, not asserted

A theorem that says "a winning move exists" is cheap. This one is constructive, so the page plays it back through the solver instead of claiming it:

for (const n of winnablePiles) {
  const z = smallestZeckTerm(n);
  assert(z <= n - 1);                        // legal opening
  assert(solveState(n - z, 2 * z) === false); // opponent is now lost
}

288 winnable piles replayed, 0 broken.

Every opening that feels right wins nothing

This is the part worth the controls. Judged by the solver, over the same 288 winnable piles:

opening rule	piles it wins
smallest Zeckendorf term	288
largest Zeckendorf term	0
take the maximum allowed	0
take half the pile	0
largest Fibonacci that fits	0
always take 1	114

Four plausible rules — including one built from the same arithmetic read from the wrong end — win zero piles between them. "Always take 1" is the interesting near-miss at 114, and 114 is not a coincidence: it is exactly the count of piles whose Zeckendorf representation contains a 1. The page counts those two numbers by separate routes and they agree.

The correct take is small — mean 1075/288 ≈ 3.73 counters, never more than 2/7 of the pile (worst at n=7). It is still not bounded by a constant: at n=199 the right opening is 55.

"The" winning move is a lie the textbook statement tells

winning openings	piles
none (the pile is lost)	11
exactly one	70
two	143
three	75

218 of 288 winnable piles have more than one winning opening. At n=72 they are 1, 4 and 17; at n=300 they are 1, 12 and 67 — and the extras are usually not Fibonacci numbers at all. The Zeckendorf term is always the smallest of them, which is a real checkable property; the others have no tidy description.

What this does not claim

299 piles agreeing is strong evidence for the theorem. It is not the theorem, and the page says which range it verified. The memo is keyed on rem * 4096 + c, which is sound to n=300 and would silently collide long before it became a general solver. And nothing here makes the strategy playable — finding the smallest Zeckendorf term of 300 counters at a table is exactly as awkward as it sounds.

29 in-page checks, 1,016 verifier asserts, 0 failures. Vanilla JS, one file, no build step.

A Search Scope Is a Read Scope: 751 Yes/No Questions Recover Every "Private" Field in My Personal API

Devanshu Biswas — Tue, 08 Sep 2026 08:31:24 +0000

You build one local API over your notes, contacts and calendar. Five endpoints, each behind its own scope: read, list, search, count, aggregate. You hand an agent a token with notes:search and nothing else. The body is withheld on every endpoint that returns a document.

That token recovers all 14 private bodies, character for character, in a mean of 751 yes/no questions each.

👉 PUBLIC, MIT, dependencies = []: https://github.com/dev48v/personal-api
👉 Live measurement in your browser: https://dev48v.infy.uk/agentlab/vol4-04-personal-api.html

python -m personal_api            # what each scope is actually worth
python -m personal_api --oracles  # the five oracles, side by side
python -m personal_api --demo     # watch one note come back out

What each scope promises, and what it delivers

token scope	can it read a body?	bodies recovered anyway
read everything	yes	14/14
titles only	no	0/14
titles + search	no	14/14 exact
titles + count	no	3/14 exact, 14 whole bodies
titles + aggregate (k floor)	no	0/14
search only	no	14/14 exact

The second column is what the scope model promises. The third is the measurement.

The only preset that holds is titles only — and it holds because it has no query endpoint at all. A personal API you cannot ask questions of is not the thing anybody was trying to build.

The attack is not clever

known = ""
while len(known) < MAX:
    for ch in ALPHABET:            # 37 symbols
        if oracle(known + ch, "prefix"):
            known += ch
            break

Ask whether the body starts with a, then b, then c. When one says yes, keep it and ask for the next character. At most 37 questions per character, no backtracking. It is what an agent with a token and a while loop does on its own, and it is linear in the length.

The corpus is invented and written out in one short file before any number is computed, so you can check the counts rather than take them. test_alphabet_covers_corpus holds the alphabet to the corpus — a body containing a symbol outside it would stall the attack and understate every figure here. (That bug was real: the first version omitted digits 1–9 and reported 1/14 instead of 14/14.)

The endpoints ranked by what they are worth to an attacker

oracle	exact	any whole body	fragment recovered	mean calls
search, prefix (autocomplete)	14/14	14/14	100.0%	751
search, contains (a search box)	1/14	1/14	53.5%	401
count, prefix	3/14	14/14	23.1%	706
aggregate, k=2	0/14	0/14	2.8%	68
aggregate, k=3	0/14	0/14	0.7%	43

An autocomplete is worth far more than a search box. A prefix oracle has an anchor — the start of the string — so extension never backtracks and recovery is exact. A contains oracle has to find an anchor first and can only extend rightward, so it returns a fragment: 1 exact body in 14, but 53.5% of each body on average, contiguous. Half of a note about a bank account is not half a secret.

Counting is a read too, just aimed less well. count > 0 is the same boolean, so the same loop runs — but the count is over the whole collection, so greedy extension follows whichever record branches first. It recovers the targeted body 3 times in 14 and a complete body of some record all 14 times. The scope did not stop the leak. It randomised the victim.

The one defence that works, and why that is a caveat

With a k-anonymity floor the channel closes: 0 exact, 0 whole bodies, and 2.8% of a body at k=2 — about one character. That is a real defence and it is the recommendation this build actually supports.

But it works because every body in this corpus is unique, so any prefix long enough to be informative matches fewer than k records and gets suppressed. A floor does not protect a value that k people share. The suite says so out loud in test_the_k_floor_works_because_the_secrets_are_unique, so the caveat cannot quietly come unstuck from the claim.

Rate limiting is arithmetic

limit	one body	all 14
60 req/min	12.5 min	2 h 55 m
600 req/min	1.3 min	17.5 min
6000 req/min	7.5 s	1.8 min

At the tightest limit anyone actually ships, the whole corpus comes out inside one night. Rate limiting converts "instant" into "overnight", which is not a defence against something that runs overnight.

The control that would help is the one I did not model: 968 near-identical prefix queries against one field is a shape, and an audit log can alarm on a shape. That is the honest recommendation out of this build, and it is not a scope.

What this is not

Small invented corpus; the query counts scale with its strings. In-process API — no HTTP layer, no token expiry, no audit log. Nothing models partial field exposure, redaction, differential privacy with real noise, or an LLM in the loop deciding what to answer. Both attacks are the naive ones on purpose: they establish a floor on what is recoverable, not a ceiling.

25 pytest, most of them controls — that read is genuinely denied, that list genuinely withholds the body, that titles only genuinely recovers nothing, and that the attack never once calls a denied endpoint (the denied-call counter is asserted to stay at zero, so it is a reconstruction and not a read wearing a hat).

The claim is narrow. An endpoint that answers questions about a field is an endpoint that returns the field, and a scope model that separates them is describing an intent rather than a boundary.

Your Vector Store Rejects the Embedder Change That Barely Matters and Cannot See the One That Destroys Retrieval

Devanshu Biswas — Tue, 08 Sep 2026 08:30:43 +0000

The provider ships v2, or somebody changes the tokenizer. The index is still there, still returning five results, still with plausible scores.

Change the width and the vector is the wrong shape, so a typed index refuses it outright — and even with that check removed, precision@5 only falls from 0.9417 to 0.7750.

Change the hash seed or the tokenizer and the width is identical, nothing anywhere objects, and precision@5 falls to 0.0333, 0.0917 and 0.1000 — against a chance baseline of 0.0833.

👉 Live, the whole grid computed in your browser: https://dev48v.infy.uk/ai/days/day79-embedding-drift.html

Nothing here calls an embedding API

The embedder is a hashed bag-of-tokens implemented in full — FNV-1a with a stated seed, the signed hashing trick, optional idf, L2 normalisation, cosine. A "version" is a concrete configuration of that embedder, so a version change here is one people actually make.

The corpus is synthetic and declared: 12 topics × 8 characteristic terms, 10 documents each. Because every document's topic is known by construction, retrieval is scored for correctness, not just for agreement with an earlier run.

The detectable failures are the mild ones

version change	does a typed store catch it?	precision@5 after a full re-index	precision@5 if only the query moves
same version (control)	—	0.9417	0.9417
width 128 → 256	yes — wrong width	0.9583	0.7750
width 128 → 64	yes — wrong width	0.8583	0.6333
hash seed 1 → 2	no — same width	0.9250	0.0917
4-grams → 3-grams	no — same width	0.9083	0.0333
4-grams → words	no — same width	0.8583	0.1000
idf weighting on	no — same width	0.9333	0.9250

Sort by damage and you get almost exactly the reverse of sorting by detectability. The 3-gram row lands at 0.0333 — below the 0.0833 you would get by returning five documents at random.

The vectors carry no version tag. Cosine is happy to compare any two of the same length. There is no point in that path where a mismatch can be detected.

The half-migrated index is the shape this actually takes

The migration script runs on new ingests and nobody backfills. Half the index is in one space, half in another:

	half-migrated	single-version	difference
new documents' share of top-5	21.67%	52.50%	−30.83 points
precision@5	0.4667	0.9417	−50.4%
mean top-1 cosine	0.3992	0.5689	−29.8%
queries returning five documents	24 of 24	24 of 24	none

Those documents are not ranked badly. They are unreachable — and the system returns five results every time, drawn from the half that still lines up. No error, never an empty result set.

The monitor you would reach for is the wrong shape too

Without labels, the one thing you can watch is the similarity score. On a total mismatch it does move: mean top-1 cosine falls from 0.5689 to about 0.23, a ~60% drop. A threshold catches that.

On the half-migrated index it falls 29.8% while precision falls 50.4%. Every query still has a well-matched document somewhere in the half that lines up, so the top score stays respectable while half the corpus has quietly left the building.

A threshold tuned on the loud failure will not fire on the quiet one, and the quiet one is the one that happens.

Even the correct migration moves your answers

Re-embedding everything is the right fix and it works — precision comes back to 0.858–0.958, in one case slightly above the original. But top-5 overlap with the old index runs from 0.933 down to 0.725: between 6.7% and 27.5% of retrieved documents are different afterwards.

Nothing is broken. It is just that any prompt, cache, eval set or human sign-off pinned to the old results is now pinned to results that no longer come back — on the day you did everything right.

The fix, and the honest scope

Write the embedder version into the index and refuse to serve a query whose version does not match. Every consequence above follows from the vectors not carrying one.

Scope, because it changes how you read the table: the embedder is a hashed bag-of-tokens, not a neural model. A hash-seed change produces two unrelated spaces — that is the extreme case and an upper bound, not a prediction for what v1→v2 of a trained model does. The tokenizer rows are the better analogue, and they are nearly as bad.

Nothing here models semantic drift: there is no meaning in this corpus beyond term overlap, so this says nothing about a v2 that is simply better at understanding a question. 120 documents, 24 queries, top-5, plain cosine, no chunking, no reranker, no hybrid keyword leg.

28 in-page checks, 95 verifier assertions, 0 failures.

MC Dropout With 10 Samples Is Further From the Truth Than Just Turning Dropout Off. Here Is the Break-Even.

Devanshu Biswas — Tue, 08 Sep 2026 08:30:01 +0000

Dropout is a training trick. At test time you turn it off and the whole network answers. MC dropout says leave it on, run k forward passes, average them, and read the spread as uncertainty.

Both of those are approximations to the same object — the mean of the dropout ensemble. So I computed that object exactly instead of approximating it: 12 dropout units means 2¹² = 4096 masks, every one enumerated.

👉 Live, the full enumeration in your browser: https://dev48v.infy.uk/dl/day79-dropout-at-inference.html

Turning dropout off does not give you the ensemble mean

With inverted dropout, a kept unit is divided by the keep probability, so the expected post-dropout activation equals the plain activation. That is exactly why "turn dropout off" is supposed to give you the ensemble mean.

It does not, and the control says why:

network	largest gap	typical output	CONTROL: gap with a linear head
seed 7	0.5397	0.5537	2.75×10⁻¹⁴
seed 101	0.4106	0.8880	2.75×10⁻¹⁴
seed 2027	0.2956	0.4613	2.71×10⁻¹⁴
seed 31337	0.8336	1.2534	1.19×10⁻¹³
seed 90210	0.2457	0.3972	2.51×10⁻¹⁴

Make the layer after dropout linear and the expectation passes straight through: E[head(drop)] = head(E[drop]), exactly, and the gap collapses to floating-point zero in all five networks. Put the ReLU back and the gap is a third of the signal.

A bug in the enumeration would not switch itself off when the ReLU does. That is what makes this a property of the architecture rather than a mistake in my code.

The break-even is 13.9 passes

The k-sample mean is unbiased, so its RMS error is σ/√k. The deterministic pass has no variance but a fixed bias. Set them equal:

k* = σ² / bias²   =   13.9

And the measured curve crosses exactly where it should:

stochastic passes k	measured RMS error	vs dropout-off (0.2109)
1	0.7859	273% further
2	0.5671	169% further
5	0.3557	69% further
10	0.2492	18% further
30	0.1391	34% closer
100	0.0792	62% closer
1000	0.0240	89% closer

Across five networks the break-even lands between 5.8 and 29.8.

The point is not that 30 passes is expensive. It is that 5 and 10 — the numbers that actually appear in code — sit on the wrong side of the line. Averaging a handful of stochastic passes is a more elaborate way of being further from the answer.

The uncertainty moves. The prediction does not.

The keep probability is a training hyperparameter. Sweeping it:

keep	RMS reported spread	deterministic output
0.50	0.7865	identical
0.60	0.6568	identical
0.70	0.5391	identical
0.80	0.4222	identical
0.90	0.2893	identical
0.95	0.2021	identical

A 3.89× change in the reported uncertainty, and the deterministic output is bit-identical at all 61 inputs — because that pass never sees a mask at all.

Two teams who picked 0.5 and 0.9 will report different uncertainties for the same prediction from the same network, and neither is more right. The answer and the confidence attached to it come from different places: one from the weights, the other from a number chosen during training.

(This one is exact too: at keep ≠ 0.5 the masks are not equally likely, so they are weighted by binomial mass rather than sampled. Still all 4096.)

What I expected to find, and did not

Going in, my hypothesis was that this spread is just a restatement of how large the activations are — that it would track output magnitude and carry nothing else.

network	r( spread , \|output\| )	r( spread , \|gap\| )
seed 7	0.7304	−0.0750
seed 101	0.9550	0.5912
seed 2027	0.9573	−0.3257
seed 31337	0.7092	0.8325
seed 90210	−0.1885	0.5506

In four of five it holds, at 0.71 to 0.96. In the fifth it is −0.19. One counterexample in five is not a rounding error, so that claim is not established and I am not making it.

The second column is the cleaner negative. The spread does not tell you where the deterministic approximation is worst: −0.33 to +0.83 across five networks — not merely weak, but with an unstable sign. Whatever that number is measuring, it is not the size of the error it is standing next to.

What this does not cover

These networks are not trained — the weights come deterministically from a stated seed, so nothing here says what dropout does to a fitted model. One dropout layer, scalar in and out. No label noise, so the spread is purely the ensemble's own disagreement with no aleatoric term to separate it from. And none of this evaluates MC dropout as a calibration method: no labels, no reliability diagram.

25 in-page checks, 66 verifier assertions, 0 failures.

Your 90%-Precision Threshold Delivers 63.5% After a Prevalence Shift — With TPR and FPR Bit-Identical

Devanshu Biswas — Tue, 08 Sep 2026 08:29:20 +0000

You tuned a decision threshold on validation to hit "precision at least 90%". You shipped it. The population's prevalence drifted from 10% to 2%.

The model did not change. The scores did not change. TPR and FPR at that threshold are bit-identical — the same reduced fractions, 22663/29409 and 21043/499953.

Precision is now 63.46%.

👉 Live, all 201 thresholds enumerated exactly in your browser: https://dev48v.infy.uk/ml/day79-threshold-transfer.html

Nothing is sampled and no rate is a float

The score axis is 201 bins of declared integer weights, all 201 thresholds are enumerated, and every rate is an exact rational compared by BigInt cross-multiplication — so an argmax is never decided by rounding, and there are no confidence intervals anywhere because there is nothing to be uncertain about.

The identity the whole thing rests on

Over a virtual population of q·WP·WN, every confusion cell is an exact integer:

TP = p * Sp * WN         FP = (q - p) * Sn * WP
FN = p * (WP - Sp) * WN  TN = (q - p) * (WN - Sn) * WP

So:

TPR = TP / (TP + FN)
    = (p·Sp·WN) / (p·Sp·WN + p·(WP−Sp)·WN)
    = Sp / WP

p and q cancel. TPR does not know what the prevalence is. Neither does FPR. Both are conditioned on the class.

Precision is not. Over six prevalences from 0.1% to 90% at one fixed threshold:

prevalence	TPR (reduced)	FPR (reduced)	precision
0.1%	`22663/29409`	`21043/499953`	0.0180
2%	`22663/29409`	`21043/499953`	0.2720
10%	`22663/29409`	`21043/499953`	0.6704
30%	`22663/29409`	`21043/499953`	0.8870
50%	`22663/29409`	`21043/499953`	0.9482
90%	`22663/29409`	`21043/499953`	0.9940

Two identical columns and a fifty-fold swing. That is an algebraic identity on one side and the entire problem on the other.

Two of five tuning rules pick a portable threshold

Re-running each rule on the shifted population and reading off where it lands:

tuning rule	source	after 10%→2%	portable?
score ≥ 0.50	bin 100	bin 100	n/a — it cannot move
maximise F1	bin 127	bin 146	no
maximise TPR − FPR (Youden)	bin 100	bin 100	yes
precision ≥ 0.90	bin 148	bin 169	no
recall ≥ 0.90	bin 98	bin 98	yes

The two that survive are exactly the two whose definitions mention only class-conditional rates. That is not an empirical coincidence to be spot-checked; it follows from the identity above. Youden picks the same bin at five different prevalences; max-F1 picks a different bin at nearly every one.

Scope: recall≥0.90 is also unmoved by the score shift I used here, but only because that shift moves the negatives. Recall does not look at negatives. Move the positives and it moves like everything else.

Re-tuning meets the target and hollows out the model

Re-running precision≥0.90 on each world does restore the number:

world	re-tuned threshold	precision	recall
source	bin 148	0.9044	0.4955
prevalence shift	bin 169	0.9088	0.2412
score shift	bin 166	0.9041	0.2773
both	bin 193	0.9022	0.0190

In the last world the threshold that satisfies the contract fires on 1.9% of the positives. A precision target is a promise about the answers you give, and it can always be kept by giving fewer. A monitor watching only precision reports a healthy system right up to the point where the system is not answering.

The part with no good answer

In production you usually have no labels. The one thing you can watch is how often the model says yes.

I constructed two worlds with the exactly equal positive rate — not close, the identical rational 780643/49995300 ≈ 1.5614%. The prevalence for the second was solved from rate = π·TPR + (1−π)·FPR, which has an exact rational answer.

world	positive rate	precision	recall
prevalence shift, 10% → 2%	`780643/49995300`	0.6346	0.4955
score shift, prevalence 0.0702%	`780643/49995300`	0.0223	0.4955

Same firing rate. Same recall. A 28.5× difference in precision. Every dashboard that watches the positive rate shows a flat line across both.

What this does not cover

One score axis in 201 bins, one shift direction (negatives up 12 bins, positives never move), one pair of class-conditional shapes, five tuning rules. There is no cost matrix, so "which threshold is right" is never answered here — only which one is stable. No calibration model and no recalibration method: the point is what breaks, not how to fix it.

21 in-page checks, 72 verifier assertions, 0 failures.