DEV Community: Devanshu Biswas

I Built a Triage Tool Whose Only Promise Is That Every Item Lands on Exactly One Shelf

Devanshu Biswas — Mon, 12 Oct 2026 11:22:21 +0000

You have a folder. Screenshots, half-written notes, saved links, a photo of a whiteboard. Somewhere in there are the things that belong to projects you are working on, and the things somebody is waiting on you for.

shelf reads the folder, pulls text out of whatever it can, scores that text against a project list you wrote, and writes a board. It runs on your machine, needs no key, and installs nothing.

📦 Repo (MIT): https://github.com/dev48v/shelf
👉 Walkthrough, runs in your browser: https://dev48.in/agentlab/vol5-01-shelf.html

$ python shelf.py build --pile demo/pile --projects demo/projects.md
6 items -> board.md
note: tesseract not found; images used sidecars or filenames

## Replies you owe  (1)
| item | read via | why |
|---|---|---|
| `email-accountant.txt` | file contents | the text reads as someone waiting on a reply |

## Kitchen rebuild  (2)
| `note-quote.txt` | file contents | matched 83% of the terms for Kitchen rebuild |
| `whiteboard.png` | sidecar .txt  | matched 50% of the terms for Kitchen rebuild |

## The partition
1 Replies you owe + 2 Kitchen rebuild + 1 Cycling trip + 2 Unfiled = 6 of 6

The one rule

Every item lands on exactly one shelf, and the board says why.

Not mostly one. A triage tool that can silently drop an item, or quietly file it twice, is worse than no triage tool — because you stop checking it, and then it is just another pile.

def check_partition(items, projects):
    valid = {p.name for p in projects} | {UNFILED, OWED}
    for item in items:
        if item.shelf not in valid:
            raise AssertionError(f"{item.name} landed on unknown shelf {item.shelf!r}")
        if not item.reason:
            raise AssertionError(f"{item.name} was filed with no reason given")

    shelved = {}
    for item in items:
        shelved.setdefault(item.shelf, []).append(item.name)
    if sum(len(v) for v in shelved.values()) != len(items):
        raise AssertionError("the partition does not account for every item")

It runs on every build and the run fails if it does not hold. The board prints the sum at the bottom so you can see that it did.

🔴 The bug that invariant exists to catch

A .txt sitting next to an image of the same stem is that image's caption — evidence about the image, not an item of its own. Count it as an item and the board claims 7 where the folder holds 6.

It is a one-line mistake, it produces a board that looks entirely reasonable, and the only thing standing between you and it is a count that has to add up.

Term overlap, not similarity

OCR output is garbled. Half a word is missing, 0 is O, a line is nonsense. Counting how many of your terms survive is robust to exactly that damage. An edit distance or an embedding needs clean text to behave, and clean text is the one thing a pile of screenshots does not have.

w1dget !!! spr0cket ... widget ~~~ sprocket   →  still scores 100%

Inject noise every eleventh character of a real note and it still files correctly.

The categories are yours

# Kitchen rebuild
quote, builder, worktop, plumber, tiles, splashback

Headings are shelves, the lines under them are terms. Nothing invents a category or infers one from the pile. A classifier that writes its own labels is answering a question you did not ask. Multi-word terms must appear in order — self assessment matches "my self assessment form" and not "assessment of self".

Provenance, on every row

A score built on a filename is worth far less than one built on contents, and the board says which: file contents, sidecar .txt, OCR, or filename only.

🔴 And a filename alone never counts as somebody waiting on you. can-you-reply-to-this.png is a note to self, not a person.

🔴 How the walkthrough page is checked

The page ports the rules to JavaScript so you can run them live. A port that drifts from its original is the specific failure no JavaScript-only test can see — so the verifier runs the Python and demands the two agree item for item, shelf for shelf, reason for reason and provenance for provenance.

That check found its own bug: execFileSync returns stdout only, unittest writes its summary to stderr, and "the suite ran" was passing on the CLI's print output. spawnSync reading both streams fixed it.

Python 3.10+, standard library only. 51 tests, 124 cross-language verifier asserts, 29 in-page checks, 0 failures.

Cost Per Correct Answer Is Worse Than Useless, and It Will Tell You to Use the Cheap Model

Devanshu Biswas — Mon, 12 Oct 2026 11:21:40 +0000

A small model costs a fraction of a large one and gets most easy requests right. A router that could tell easy from hard would cut the bill dramatically. Every production LLM system reaches this point, and the argument that follows is always accuracy versus cost — which is the wrong frame, because you can buy any accuracy you like by spending more.

👉 Live, runs in your browser: https://dev48.in/ai/days/day81-cost-routing.html

The metric everyone proposes

"Use cost per correct answer." Here it is, on 2,000 requests, small model at 1 unit and big at 20:

strategy	accuracy	cost / request	cost / correct
always small	74.5%	1.00	1.34 ← the winner
route on difficulty	95.8%	6.42	6.70
oracle	95.8%	5.85	6.11
always big	95.8%	20.00	20.88

🔴 It picks the small model outright. A 74%-accurate model at 1 unit beats a 96%-accurate one at 20 on that metric, every time, and it always will. My first draft of this page asserted the opposite and the in-page check disproved it.

Cost per correct answer, on its own, hands the prize to the cheapest model that is vaguely adequate.

The question that does have an answer

Fix a quality floor first. Given the accuracy you have promised, what is the cheapest way to reach it?

the promise	cheapest that keeps it	spend vs always-big
70%	always small	5%
90%	cascade with a verifier	28%
95%	route on difficulty	32%

At a 90% promise the small model is not on the menu — it is not cheap, it is ineligible. And routing then matches always-big's accuracy exactly for under a third of the spend. That saving is the entire value of routing, and it only becomes visible once the quality bar is written down.

🔴 Routing on the small model's confidence

The obvious design: ask the small model, escalate when it says it is unsure. Cheap, needs no training, and it fails exactly where it matters.

the small model says	requests	actually right
95% confident	843	100%
90% confident	406	0%
60% confident	303	61%
55% confident	448	0%

There is a band where it is 90% confident and 0% correct. Those are precisely the requests escalation exists for, and confidence is blind to them by construction. Confidence measures fluency, not correctness.

The router is not free

Most comparisons quietly price the router at zero. It is a model call too.

for (const req of reqs) {
  const r = strategy(req, opts);
  cost += r.cost;
  routerCost += routerPrice;     // on EVERY request, whichever way it decides
}

🔴 In my first version that price was read from the options and never billed — no strategy returned a router field, so the column stayed at zero and the overhead table was meaningless. Difference two runs and a charge that never lands cannot hide.

The ceiling

An oracle router knows in advance which requests the small model will get right and sends only the rest. No real router beats it, and the gap between yours and the oracle is the only honest measure of how much routing skill you have left to gain.

Here the oracle matches always-big's accuracy for 29% of the spend. Routing on difficulty reaches 32%. There is not much left on the table.

Everything is reproducible

No API, no sampling. Difficulty is u² for u uniform, so P(difficulty < d) = √d and every number is re-derivable in closed form — which is how the verifier checks the simulation rather than trusting it.

184 verifier asserts, 34 in-page checks, 0 failures.

The RoPE Decay Curve Everyone Plots Is Noise, and Taking the Absolute Value Does Not Fix It

Devanshu Biswas — Mon, 12 Oct 2026 11:20:59 +0000

Rotary position embeddings do something that sounds impossible. They inject each token's absolute position into the query and key — and the attention score between any two tokens depends only on the distance between them. Move a whole sentence ten tokens later and every score inside it is unchanged.

👉 Live, runs in your browser: https://dev48.in/dl/day81-rotary-embeddings.html

The identity, in one line

Write a 2-D pair of the query at position m as a complex number q, and the key at n as k. RoPE multiplies them by e^(imθ) and e^(inθ):

Re[ (q·e^(imθ)) · conj(k·e^(inθ)) ] = Re[ q·conj(k)·e^(i(m−n)θ) ]

The absolute positions appear only as their difference. Verified over 42 combinations of offset and shift — including a shift of 4000 — worst deviation 1e−15. Additive sinusoidal embeddings, by contrast, give a different score for the same offset at different absolute positions.

function rope(vec, pos, base = 10000) {
  const th = thetas(vec.length, base);
  const out = new Array(vec.length);
  for (let i = 0; i < vec.length / 2; i++) {
    const a = vec[2*i], b = vec[2*i+1], ang = pos * th[i];
    out[2*i]   = a * Math.cos(ang) - b * Math.sin(ang);
    out[2*i+1] = a * Math.sin(ang) + b * Math.cos(ang);
  }
  return out;
}

🔴 Now the part people get wrong

"Attention under RoPE decays with distance." Plot it carelessly and you get a flat line of random scatter, because two independent random vectors have an expected dot product of zero at every distance. The curve you drew was sampling noise.

The usual next move is to average the magnitude instead. That does not rescue it either:

offset	q against itself	mean raw score, 600 pairs	mean \|score\|, same pairs
0	10.60	−0.07	3.17
16	8.05	−0.37	3.12
256	4.95	−0.28	3.04
1024	0.69	−0.38	3.02

Three columns, same vectors, same offsets. The mean raw score never leaves ±0.38 against a typical magnitude of 3.2. The mean |score| moves 4% across the whole range. Only the self column collapses — and that is the decay people are describing: the pairs start perfectly in phase and dephase at different rates until their cosines cancel.

It is dephasing, not attenuation, and it is a statement about a fixed vector.

🔴 The bug that produced the first, wrong measurement

Each "random" vector got its own Park-Miller generator seeded by a nearby integer. Nearby LCG seeds give correlated streams. The supposedly independent mean wandered to ±2.5 — an order of magnitude past its standard error — and looked like structure.

One generator for the whole study, and it drops to ±0.38. If your "independent samples" come from rng(seed + i), they are not independent.

🔴 NTK scaling cannot touch the fastest pair

Train to 512, ask about 4096. It is tempting to blame the fast pairs, but they were already wrapping constantly inside the training window — another twenty turns is nothing new.

The pair that leaves familiar ground is the slowest one. Its period is longer than the whole context, so across training it swept only 2.6% of a turn; push the position eightfold and it swings into angles that never once appeared.

strategy	slowest pair's angle
trained, at 512	0.162 rad	what the model saw
untouched, at 4096	1.295 rad	8× past anything in training
position interpolation	0.162 rad	back exactly where it trained
NTK-aware base	0.162 rad	back exactly where it trained

Both repairs land it in the same place. They differ at the fast end: θ₀ = base⁰ = 1 whatever you do to the base, so NTK cannot slow the fastest pair at all, while interpolation slows every pair equally and costs resolution between adjacent tokens.

How it is checked

The verifier multiplies complex numbers rather than using the real 2×2 form, extracts the rotation as a matrix and tests RᵀR = I with det = +1, and re-measures the decay with a Marsaglia polar sampler so a bug in the page's generator cannot reproduce itself in the check.

It caught a sign error in my closed form — + where the derivation gives − on the sin term.

226 verifier asserts, 28 in-page checks, 0 failures.

Leave-One-Out Attributed 20 for a Prediction of 10, Then Gave Two Real Features Exactly Zero

Devanshu Biswas — Mon, 12 Oct 2026 11:20:16 +0000

You have a model and one prediction. The question everyone asks is: how much of this number is down to each input? Shapley values answer it by asking a more careful question — how much does this feature add, on average, over every possible order in which the features could have arrived?

👉 Live, runs in your browser: https://dev48.in/ml/day81-shapley-values.html

The obvious alternative, and both ways it breaks

Leave-one-out: drop a feature, see how much the prediction moves, call that its importance. One line of code.

Case 1 — interaction. A model that fires only when both features are present, worth 10.

feature	Shapley	leave-one-out
has_card	5	10
is_verified	5	10
total	10 ✓	20 ✗

Drop either one and the prediction falls to the floor, so leave-one-out scores them both as fully responsible. It attributes twice the prediction.

Case 2 — redundancy. Two features carrying the same information.

feature	Shapley	leave-one-out
signal	4	0
signal_copy	4	0

Drop either copy and nothing moves, so it scores both at exactly zero. A feature-importance plot built this way shows real signal as irrelevant, and somebody deletes it.

🔴 It errs in both directions — over-attributing interaction, under-attributing redundancy — and it agrees with Shapley exactly on additive models. Which is precisely why an additive example proves nothing about an attribution method.

Three derivations that agree to the last bit

phi_i = sum over S not containing i of
          |S|! (n-|S|-1)! / n!  *  ( v(S + i) - v(S) )

That factorial weight is not decoration. It is exactly the fraction of the n! orderings in which S arrives before i and everything else after — a probability distribution over the position at which feature i arrives, summing to 1.

The page computes the same values a second way, as an average over every permutation. The verifier computes them a third way that touches neither factorials nor permutations: Möbius inversion of the set function into Harsanyi dividends, then split each dividend equally among the coalition that earned it.

m(S)   = sum over T subset of S of (-1)^(|S|-|T|) v(T)
phi_i  = sum over S containing i of m(S) / |S|

For the AND model the whole dividend sits on the pair — m({a,b}) = 10, m({a}) = m({b}) = 0 — which is the statement that neither feature does anything alone, in one line.

All four axioms, checked

Shapley values are not a heuristic that happens to work. They are the unique attribution satisfying four requirements at once:

Efficiency — the values sum to the prediction minus the baseline. This is the one leave-one-out cannot keep.
Dummy — a feature that never changes anything scores exactly 0.
Symmetry — interchangeable features get identical values.
Linearity — attributing a sum of models is the sum of the attributions.

Checked on every model, with symmetry re-derived by swapping columns of v rather than trusting the page's detector.

The sampled estimator keeps one property for free

Exact Shapley costs 2ⁿ model evaluations. Past about twenty features that is hopeless, so practice samples permutations.

Efficiency stays exact at every sample count — even at 1. Each permutation's marginal contributions telescope to v(full) − v(empty) whatever order you drew. What sampling costs you is per-feature accuracy, and that is the number you should report.

And attribution has a sign

A feature that lowers the prediction gets a negative value: +6 for the helpful one, −4 for the harmful one. A method that reports magnitudes only cannot say that.

470 verifier asserts, 23 in-page checks, 0 failures.

Myers Diff Does 1.6% of the Work on Real Files and 105% on Random Ones

Devanshu Biswas — Mon, 12 Oct 2026 11:19:36 +0000

Every diff you have read is the answer to one question: what is the shortest sequence of deletions and insertions that turns the old file into the new one? Myers' 1986 algorithm answers it by treating the two files as the axes of a grid and finding the cheapest path across — where moving diagonally, on a line both files share, is free.

👉 Live, runs in your browser: https://dev48.in/solve/day81-myers-diff.html

The search

Rather than exploring the grid cell by cell, Myers asks: how far can I get using exactly D edits? For each D it keeps only the furthest-reaching point on each diagonal — one number per diagonal, not a whole row — and stops the first time one reaches the corner. Because D only increases, the first arrival is minimal by construction.

for (let D = 0; D <= MAX; D++) {
  trace.push(V.slice());                   // snapshot, for backtracking later
  for (let k = -D; k <= D; k += 2) {
    let x = (k === -D || (k !== D && V[k-1+MAX] < V[k+1+MAX]))
      ? V[k+1+MAX]                         // moved down (insertion)
      : V[k-1+MAX] + 1;                    // moved right (deletion)
    let y = x - k;
    while (x < N && y < M && a[x] === b[y]) { x++; y++; }   // free diagonal
    V[k+MAX] = x;
    if (x >= N && y >= M) return { distance: D, script: backtrack(...) };
  }
}

🔴 The speed claim is half a truth

Everyone quotes O((N+M)D) against O(NM) and stops. Measured over 800 generated pairs:

corpus	worst D	Myers work vs an O(NM) table
a 40-line file with 4 edits	8	1.6%
random strings, 4-symbol alphabet	14	105%

On random strings D grows with the input, O((N+M)D) degenerates, and Myers does more work than the table it replaced. My first draft of this page quoted only the flattering corpus; the in-page check caught it.

The speed is not a property of the algorithm. It is a property of the algorithm meeting the inputs it was designed for — and files people actually diff are nearly identical, which is why it wins in practice.

The only honest test of a diff

Not "does it look right". Does the script rebuild the target?

function apply(a, script) {
  const out = []; let i = 0;
  for (const e of script) {
    if (e.op === "keep") { if (a[i] !== e.line) throw new Error("keep does not match"); out.push(a[i++]); }
    else if (e.op === "del") { if (a[i] !== e.line) throw new Error("delete does not match"); i++; }
    else out.push(e.line);
  }
  if (i !== a.length) throw new Error("script consumed " + i + " of " + a.length);
  return out;
}

800 of 800 generated pairs rebuilt exactly, with D = N + M − 2L satisfied every time against an independently computed longest common subsequence.

🔴 The bug in the V array

const V = new Array(2 * MAX + 1).fill(0);   // wrong
const V = new Array(2 * MAX + 3).fill(0);   // right

The search reads V[k-1] and V[k+1], so both ends need a slot of headroom. With two empty inputs MAX is 0, the array is one slot, the read runs off the end, returns undefined, and the loop never terminates. Every non-degenerate case worked perfectly.

Shortest is not readable

Several edit scripts can be equally short and they are not equally good to read:

change	D	hunks
two lines changed, adjacent	4	1
two lines changed, far apart	4	2

Same minimal distance, one tidy block versus two separate places to look. Minimising D says nothing whatever about this — which is why real diff tools bolt heuristics on top rather than trusting minimality alone.

How it is checked

188 asserts by three routes the page does not use: breadth-first search over the edit graph by layers, a full-table LCS with the subsequence reconstructed and verified as a subsequence of both inputs, and an exhaustive proof that no script shorter than D exists for small inputs.

188 verifier asserts, 27 in-page checks, 0 failures.

Five Stages of Self-Critique Took 5/17 to 15/17. Removing One Gate Threw All of It Away.

Devanshu Biswas — Mon, 12 Oct 2026 11:18:55 +0000

Metacognitive Prompting borrows the shape of human self-monitoring. Instead of one shot at an answer, the model walks five stages: understand the question, form a preliminary answer, critique it, decide, and report confidence.

The claim is that the critique catches what the first pass got wrong. To measure that rather than assert it, everything here runs against a deterministic model with known, deliberate failure modes against tasks whose answers are known. No API, no sampling. Re-run it and the numbers repeat.

👉 Live, runs in your browser: https://dev48.in/prompt/day81-metacognitive-prompting.html

The gate is the whole design

pipeline	correct	rescued a wrong answer	ruined a right one
single pass (baseline)	5 / 17	—	—
metacognitive, gated	15 / 17	10	0
metacognitive, ungated	5 / 17	5	5

Stage 3 must be allowed to find nothing, and stage 4 must act only on a named defect:

function decide(prelim, crit, { gated = true } = {}) {
  if (gated) {
    if (!crit.defect) return { answer: prelim.answer, revised: false };
    return { answer: crit.suggests, revised: true };
  }
  // the control: always "improve" something, even with nothing named
  return { answer: crit.suggests ?? perturb(prelim.answer), revised: true };
}

🔴 Ungated revision throws the entire gain away. It lands back at 5/17 — level with running no stages at all — and it undoes the five misreads stage 1 had just fixed. Ask a model to find fault and it will always find fault.

Which stage earns which fix — by ablation, not by claim

Run the pipeline with stage 1 removed, with stage 3 removed, and with both, then attribute each recovered task to the stage whose removal loses it.

Remove comprehension → only the misreads are lost.
Remove critique → only the rule slips are.
No task needs both. The two gains sum to the total with nothing unattributed.

🔴 That attribution did not hold at first. My critique stage had a branch that caught misreads, so stage 1 was recovering nothing and the ablation measured 0. The branch was wrong and had to go: an answer carries no evidence that a different question was answered. Only the restatement from stage 1 holds that evidence. Giving the critic a misread branch makes stage 1 decorative — and is a lie about what a critic can see.

Stage 5 is worse than useless

reported confidence	tasks	actually correct
95%	2	0%
90%	10	100%
70%	5	100%

The model is surest exactly where it is least entitled to be. The 95% bucket is the underdetermined questions — the ones with no answer — because there is nothing there to hesitate about. The most confident bucket is the least accurate one.

Adding a confidence stage is cheap and plausible. It is also, on this task set, actively misleading. If you ship one, measure its calibration or do not report it.

🔴 And the mistake that nearly hid all of this

My first task set was 100% traps. The single-pass baseline scored 0/12, so nothing could possibly be worse than it, and the harm of ungated revision was invisible — it rescued 5 and ruined 5 of nothing.

Clean tasks had to be added before the damage was measurable at all. A benchmark made only of failures cannot measure harm, because there is nothing left to break.

The honest limit

Overreach — confidently answering a question that has no answer — survives all five stages. Nothing in the pipeline looks for an underdetermined question, so nothing catches it.

120 verifier asserts, 34 in-page checks, 0 failures.

The Dynamic Island Is a Priority Queue, and Its Corner Radius Is a Constraint

Devanshu Biswas — Mon, 12 Oct 2026 11:18:14 +0000

Apple's Dynamic Island is one black rounded rectangle that changes size. What makes it feel alive is not the animation. It is that two things can want it at once, and the rules for who gets it are fixed, total and deterministic.

👉 Live, runs in your browser: https://dev48.in/design/day81-dynamic-island.html

Not a stack

The tempting model is a stack: whatever started last is on top. It is wrong in a way users notice immediately — start a timer during a phone call and the call disappears.

The real model is a priority queue keyed by the kind of activity, with start time used only to break ties between two activities of equal rank.

function compareActivities(a, b) {
  const ra = KINDS[a.kind].rank, rb = KINDS[b.kind].rank;
  if (ra !== rb) return ra - rb;                  // call beats timer beats music
  if (a.startedAt !== b.startedAt) return a.startedAt - b.startedAt;
  return a.id < b.id ? -1 : a.id > b.id ? 1 : 0;  // total, never 0 for distinct ids
}

Checked over all 31 non-empty subsets of a five-activity pool, in every permutation of each: one order-independent owner every time. Both models agree while only one activity runs, which is exactly why the stack version ships and then breaks in the field.

The corner radius is not a style

Pick a fixed radius and animate the box from the small pill to the tall expanded card, and the corners visibly change character. The shape stops reading as the same object and starts reading as a box that grew.

The radius has to be a function of the height:

function radiusFor(h) {
  if (!(h > 0)) throw new Error("height must be positive");
  return Math.min(SCREEN_R, Math.max(MIN_R, h / 2));
}

A single fixed 22 is wrong by up to 10 points across the four presentations — enough that the small pill looks over-rounded and the sheet looks square.

🔴 Two bugs the verifier found

A stateless resolver teleports. Given the current state it happily names a presentation several hops away. Ending the last activity while expanded resolves straight to minimal — and expanded → minimal is not a transition the machine allows. An exhaustive replay of every 4-event sequence found 36 such jumps.

The fix is not to widen the transition table. It is to make the machine route:

transitionPath("expanded", "minimal")  // => ["expanded", "compact", "minimal"]

An interrupted expansion came back by itself. Expand the music, take a call, end the call — and the music silently reopened in its expanded state. The user expanded it, something took the island, and that expansion is gone for good:

const owner = ownerOf(s.activities);
if (s.expandedBy && (!owner || owner.id !== s.expandedBy)) s.expandedBy = null;

The spring has to land exactly

A spring that merely approaches its target is a bug. The island has to arrive at a whole number of points and stop, or the next transition starts from a position that is off by a fraction and the error compounds. Critically damped, then snap once distance and velocity are both under a threshold.

The verifier checks this against the closed form of a critically damped oscillator —

x(t) = target + (A + Bt)·e^(−ωt), ω = √k

— because "it never overshoots" is a property of the solution, and no amount of stepping establishes it.

One scalar, not two

Animate width and height as separate springs and the island passes through shapes that never existed as a presentation. One scalar drives both, so every frame lies on the straight line between two real presentations. Worst deviation: 0.

🔴 Two independently tuned springs wander up to 15 points off that line — and still arrive at the right size. Which is precisely why a screenshot test of the end state misses this bug entirely.

🔴 And one claim of mine that was wrong

I asserted in the verifier that at least one transition pair must be one-way. It failed. The size ladder is symmetric by design: every legal step is legal in reverse. That is a design choice worth pinning down, not an oversight.

210 verifier asserts, 35 in-page checks, 0 failures.

34 Assignments Satisfy Every Clue and Every Corner. Only 22 Are Solutions.

Devanshu Biswas — Mon, 12 Oct 2026 11:17:33 +0000

Slitherlink gives you a grid. Some cells carry a number from 0 to 3. You draw segments along the grid lines, and when you are done they must form one single closed loop — no branches, no loose ends, and no second loop sitting somewhere else on the board. A cell marked 3 has exactly three of its four sides on that loop.

👉 Live, runs in your browser: https://dev48.in/game/day81-slitherlink.html

The two rules everyone implements

They are both easy and both local.

Clues. Count the drawn sides of each numbered cell. It must equal the number.
Degrees. Every lattice vertex must have 0 segments touching it, or exactly 2 — one in, one out. Anything else is a loose end or a junction.

Write a solver that enforces exactly those and it looks finished. It is not.

What the count says

Here is every edge assignment on one 3×3 board that satisfies every clue and leaves every vertex at degree 0 or 2, grouped by how many loops it actually drew:

what the segments drew	assignments
one loop — a real solution	22
2 separate loops	7
3 separate loops	4
4 separate loops	1

34 pass both local tests. 22 are solutions. A solver that stops there accepts twelve boards it must refuse, on this grid alone.

The reason is structural, not a corner case. A graph in which every vertex has degree 0 or 2 is a disjoint union of cycles. It may be one cycle, or two, or five. Slitherlink demands exactly one, and that condition is global — no amount of looking at one cell or one corner can see it.

// necessary, and nowhere near sufficient
function degreesLegal(clues, on) {
  for (const v of vertices) {
    const d = incidentEdges(v).filter(e => on[e]).length;
    if (d !== 0 && d !== 2) return false;
  }
  return true;
}

// the part that is not local: walk the on-edges and count components
loopCount(clues, on) === 1

Making the search finish

A 3×3 board has 24 candidate segments, so brute force is 2²⁴ — about 16.7 million. Two cheap prunes collapse it.

Assign edges in index order. The moment the last edge of a clued cell is decided, that clue is final — check it now rather than at the leaf. Same for a vertex: once every edge touching it has a value, its degree is final and must be 0 or 2. Both tests fire deep in the tree.

The pruned search touches 0.1752% of the brute-force tree. Without the prunes the 3×3 boards do not finish.

Every 2×2 board there is

Five possibilities per cell — blank, 0, 1, 2, 3 — over four cells. 625 boards:

	boards
no solution at all	523
exactly one — a real puzzle	61
more than one — ambiguous	41

Most boards are not puzzles. That is worth knowing before you write a generator.

🔴 And a 4 is legal

I asserted in the self-check that a 4 is impossible, and the check failed. It is satisfiable: the loop is that cell's own perimeter, four segments, every corner at degree 2. Standard puzzles omit 4 because it solves itself, not because it is illegal.

A lone 3 on a 1×1 board genuinely is impossible — three sides is an open path.

How it is checked

The page runs 24 checks in your browser. A separate verifier runs 210 asserts by a route the page does not use: it enumerates every simple cycle in the lattice graph by depth-first search, so "it is one closed loop" is true by construction and never tested for. A solution is then just a cycle whose edges happen to satisfy the clues.

That verifier found a real gap in my own reasoning. I first checked single-loop-ness as connected, with edges equal to vertices. That also admits a lollipop — one cycle with a tree hanging off it. The degrees have to be checked too.

210 verifier asserts, 24 in-page checks, 0 failures. One file, no build step, no dependencies.

Your Withdrawal Rate Moves Time-to-FI by 31 Months. Your Balance, the Number in the Biggest Font, Moves It 3.

Devanshu Biswas — Thu, 17 Sep 2026 12:35:10 +0000

A freedom calculator answers three questions — how long the money lasts, what number ends the job, how far away that is — and they are one recurrence asked from two sides. Build it exactly, measure it honestly, and the finding is not that the arithmetic is wrong. It is that the answer is mostly a statement about assumptions, and the ones that move it most are the ones the genre does not show you.

Weekend Builds Vol 4 · #05 · the finale — Vol 4 is now complete at 5 of 5.

👉 Live, runs in your browser: https://dev48v.infy.uk/agentlab/vol4-05-freedom-calculator.html
👉 PUBLIC, MIT, dependencies = []: https://github.com/dev48v/freedom-calculator

python -m freedom_calculator                # runway, the FI number, time to FI
python -m freedom_calculator --sensitivity  # which input the answer rests on
python -m freedom_calculator --assumptions  # the model, and what it costs
python -m freedom_calculator --float        # why money is not a float here

Declared inputs, derived signs, measured spans

You declare five numbers. The scenario on the page: 42,000 balance, 3,200 a month out, 5,400 a month in, 5% real return, 4% withdrawal rate. From those: an FI number of 960,000 (25× annual spend) and 231 months to reach it.

Every direction below is derived on paper from the recurrence, then measured by bumping the input and watching the answer. The two are kept apart so they can be checked against each other, and the suite asserts they agree for every input at six bump sizes — plus a non-vacuity test, because a calculator that ignored its inputs entirely would pass a sign check perfectly.

±10% on…	months, down / base / up	span
what you spend a month	198 / 231 / 269	71
what you earn a month	271 / 231 / 202	69
withdrawal rate (usually hidden)	248 / 231 / 217	31
real return you assume	241 / 231 / 222	19
balance you already have	233 / 231 / 230	3

The row to sit with is the last one. The genre is arranged around a number worth about a tenth of what the hidden constant is worth.

There is also a row that must be exactly zero: runway does not depend on the withdrawal rate at all. It is a drawdown question. The derived sign is 0 and the measured direction must be 0 too — not small, not usually, exactly, at every bump size on every scenario. That identity is what stops the two halves of the calculator leaking into each other.

The argument people actually have, priced

The 4% rule is a rule about a number under argument — 3% for the cautious, 5% for the cheerful:

withdrawal rate	FI number	months
3%	1,280,000	278
4%	960,000	231
5%	768,000	198

512,000 of spread — 53.3% of the headline number — and 80 months of your life, from one constant most calculators will not let you type.

Forgetting to subtract inflation is worth 43 months: a nominal 8% instead of a real 5% brings the date from 231 months forward to 188, on which date you actually hold ~717,562 against a target of 960,000.

Money is an exact rational because the output is an integer

Not fastidiousness. The output is a month count that turns on a comparison, and a comparison decided by the last bit of a binary approximation is a month count decided by the last bit of a binary approximation.

Over 120,000 balances that are exactly n months of spend, floor(balance / spend) in doubles answers n−1 in 12,736 of them — 10.61%. Exact rationals: 0. Whole months, not cents.

The zero-net-burn case is worse. Summed as floats, the same basket of line items lands on a net burn of 2.27e-13 instead of zero, and a calculator that divides reports 43,980,465,111,040,000 months with a straight face. This package reaches the unbounded answer with a comparison and no division at all:

// runway and time-to-FI are the SAME recurrence, asked from two sides
//     b[m+1] = b[m] * (1 + r) + flow          flow = income - spend
// the zero-net-burn identity, decided BEFORE any division ever happens
if (cmp(add(mul(balance, r), flow), ZERO) >= 0) return { status: "unbounded" };

A test walks the syntax tree of runway() and asserts there is no division operator in it, with docstrings stripped first so it cannot pass on the paragraph explaining that the division is not there.

Same returns, reversed, a 17× different answer

The model has no sequence-of-returns risk in it. That is a limitation, and here is what it costs — with no RNG anywhere. One declared list of 30 yearly real returns, mean exactly the same 5% the smooth model uses, applied forward, then reversed. Start at the FI number, draw 4%:

order	ending balance
forward (crash at the front)	~127,381
reversed	~2,163,974
the smooth model's answer	~1,625,807

17.0× from order alone. Same numbers, same mean, same multiset, same product — and the control proves the product is identical: withdraw nothing and the two orders end on the identical cent, 4,176,967.52, because multiplication commutes. The entire gap is order, and the smooth model's answer sits between the two giving no hint which one you got.

The sequence was built to put the crash at the front. Stated, not hidden — the size of the crash is not the point. The point is that reversing a list changes the answer at all, which the smooth model says is impossible.

What this is not

Deterministic arithmetic on declared inputs. No RNG, no Monte Carlo, no historical data. Constant real returns, constant burn in today's money, no taxes, no lumpy years, no property, no state pension. The monthly rate is the annual rate over twelve rather than its twelfth root — the usual convention, and slightly generous. Anything past a 100-year horizon is reported as beyond the horizon rather than as a number, because a model that answers "3,847 months" is not being more precise, it is being less honest.

The JavaScript on the page is exact rationals over BigInt, checked against the Python's reference.json numerator for denominator, and against a third implementation in the verifier that simulates month by month instead of using the closed form.

50 pytest · 239 verifier asserts · 44 in-page self-checks · 0 failures. MIT, Python standard library only.

180 RAG Citation Defects Only a Free id Lookup Catches, 60 Only the Grader Does: Neither Subsumes the Other

Devanshu Biswas — Thu, 17 Sep 2026 12:34:28 +0000

A RAG answer says [c091] and everybody relaxes. There are two checks you can run on that bracket, and they catch different things.

Citation resolution is pure code: is c091 actually one of the chunks we assembled for this question? Zero model calls. Faithfulness is model-scored: does the assembled context support the claim?

👉 Live, runs in your browser: https://dev48v.infy.uk/ai/days/day80-citation-verification.html

The 2×2, and why both off-diagonals matter

420 seeded answers, 7 defect classes, 60 each:

	faithfulness says supported	says unsupported
resolution passes	120	60
resolution fails	180	60

If either off-diagonal cell were empty, one check would subsume the other and you could drop it. Both are populated by measurement: 180 answers where the identifier is wrong and the prose is fine, 60 where the identifier is fine and the prose is invented.

A pipeline running one check is not running a cheaper version of the other. It is running a different check whose blind spot is exactly what the other one is for.

The class people get wrong

stale_id: a real store id that was not in this assembled context. It looks real. It resolves perfectly against your vector store.

check	catches
resolution against the assembled context	60 / 60
resolution against the chunk store	0 / 60

Same check, one wrong set. That is the entire difference between a working guard and a decorative one:

// check 1 — resolution. No model. This is the entire thing.
const inContext = new Set(answer.assembled);          // ids WE assembled
const resolves  = answer.cites.every(id => inContext.has(id));
const pass      = answer.cites.length > 0 && resolves;

// the bug: checking the STORE instead of the assembled context
const wrong = answer.cites.every(id => store.has(id)); // c091 is real...
                                                       // ...just not here

Both blind spots, stated as numbers

Resolution never reads the claim. An unsupported sentence carrying an id that genuinely is in the context is a clean pass — 0 of 60 caught.

A context-scoped faithfulness grader is id-blind. Claim in, assembled chunks in, supported-or-not out. An invented id, a stale one, or no id at all does not change its input by one token — 0 of 180 citation defects caught. That is not inferred: the self-check replaces every citation with nonsense and measures the verdict move on 0 of 420 answers.

Neither number moves by tuning a threshold. Both are structural.

The residual neither one catches

60 answers cite a chunk that is in the assembled context and does not support the claim, while some other assembled chunk does. Resolution passes — the id is in the context. Context-scoped faithfulness passes — the context supports the claim. Both answer their own question correctly; neither question is "does this chunk support this sentence".

A citation-level grader does catch all 60, and costs one call per citation rather than per answer — 180 here against 420, and far worse on answers citing five sources each. It also cannot run on a citation that did not resolve: you cannot hand a grader a chunk you could not find. Third layer, never a replacement.

What the free check is worth

	defects caught (of 360)
faithfulness alone	120
resolution alone	240
union	300
plus citation-level	360

Zero false alarms from either check on the clean class. Resolution costs 360 comparisons and 0 model calls over the whole corpus; grading everything costs 420 calls; running the free check first and grading only what survives costs 180 — 240 model calls removed with the union catch rate unchanged.

One cost that is not money: everything short-circuiting skips sits in the cell where the answer text was fine. For those the repair is re-citing, not re-answering, and an ordering that dumps them into one undifferentiated "failed" bucket sends correct answers back to be rewritten.

The limitation to read first

The entailment check here is a stand-in, not a model — a stated token rule over 12 disjoint topic vocabularies, which separates supported from unsupported with no overlap at all. Real graders are wrong a great deal, and that error is part of why the free check is worth keeping. So read the faithfulness column as a ceiling and the resolution column as exact, because that one is code. The corpus is synthetic, one claim per answer, one citation per claim, no multi-hop, no reranker, no abstention. And nothing here asks whether the source is any good.

38 in-page checks, 138 verifier asserts, 0 failures.

Pack Four Docs Without the Block Mask and Doc 4 Spends 83.78% of Its Attention on Documents It Never Saw

Devanshu Biswas — Thu, 17 Sep 2026 12:33:46 +0000

Sequence packing concatenates short documents into one row so you stop paying for padding. On the length distribution here that lifts utilisation from 27.98% to 99.81%.

But attention inside a row is global. Without a block-diagonal mask, the fourth document in the row spends 83.78% of its layer-1 attention on documents it has never seen — worst single token 94.90%.

👉 Live, runs in your browser: https://dev48v.infy.uk/dl/day80-sequence-packing.html

The ground truth is declared; everything else is measured

Every document is first pushed through the stack on its own, ordinary causal mask, positions 0…L−1. That unpacked run is the ground truth, and every packed configuration is scored against it token by token, coordinate by coordinate — not against a loss curve, which is exactly the instrument that cannot see either bug.

The only thing that differs between a correct packed row and a broken one:

const allow = cfg.block
  ? (i, s) => s <= i && p.docIds[s] === p.docIds[i]   // block-diagonal ∩ causal
  : (i, s) => s <= i;                                 // causal only: the mask bug
const posIds = cfg.reset ? p.local : p.tokens.map((_, i) => i);

The tolerance is a prediction, not a fudge

With the mask and per-document position reset, the packed row reproduces the unpacked run to 2.22e-16 — across five independently seeded models, every token, every coordinate.

It is not exactly zero for a stated reason. A batched attention kernel subtracts a row maximum before exponentiating, and takes that maximum over the whole causal prefix of the row, not over the current document. Subtracting a constant leaves a softmax mathematically unchanged, so this is a reassociation that moves the last bit and nothing else. The engine reproduces that kernel behaviour rather than hiding it.

The verifier then tests the explanation instead of trusting it: an independently written attention stack that subtracts no maximum at all gives the identity at exactly 0. So the residual is the shift, and nothing else. The asserted tolerance is 1e-12 — four orders above the residual, twelve below the smallest real bug.

Two independent bugs, and the arithmetic proves they are two

configuration	max Δ vs unpacked	worst leaked mass
block mask + position reset	2.22e-16	0
no mask, positions reset	0.3607	94.90%
mask, no position reset	0.7607	0
neither (naive packing)	0.7773	94.90%

0.7773 is neither the sum (1.1214) nor the max (0.7607). They interact, so you cannot subtract one to estimate the other.

Two controls separate them. Zero the position table and the position bug has to vanish exactly — there is nothing left for a wrong id to index — and it drops to 1.11e-16 while the mask bug is untouched at 0.3498. And across a growing-neighbour sweep the two respond completely differently to the same change: the mask error correlates with leaked mass at r = 0.9814, the position error at r = 0.1384, because a position offset is already wrong once the neighbour has a single token.

Nothing throws, and here is why

Every attention row still sums to 1 to 4.44e-16 in all five configurations. A masked softmax renormalises over whatever it is left with, so the shape is right, the loss falls, and the model just learns something slightly wrong.

Both bugs also leave the first document in the row bit-identical — exactly 0, not nearly zero. Causal attention cannot look forward, and its positions start at 0 whether or not anyone resets them. Spot-check a packed batch on the first sequence and both bugs are invisible.

Three honest subtractions

Damage is not monotone in neighbour length. The foreign mass is strictly increasing by construction, from 0.2261 to 0.8772 mean. The error is L×gap and the second factor moves with the content of whatever tokens get appended — there is a visible dip at n = 4. "More neighbour, more damage" is a trend here, not a law.

Length-sorted batching already gets most of the win. Bucketed batches reach 99.06% utilisation against packing's 99.81%. The 3.57× is measured against arrival-order batching, which is the baseline packing is sold against but not the only alternative.

Packing does not make attention cheaper unless the mask does. Attention cost is quadratic in the row, so a packed row computed densely costs 1.027× the padded baseline — slightly more. It is the block-diagonal structure, exploited by a kernel that skips the off-diagonal blocks, that brings it to 0.1625×. The mask is not only what makes packing correct; it is what makes it fast.

The models are not trained, positions are a learned absolute table rather than RoPE, one head, two layers, float64 throughout. So which bug is larger is a property of this toy's scales and is not a general claim. The claim is that both are non-zero, independent, and invisible to the loss.

38 in-page checks, 126 verifier asserts, 0 failures.

Model A Beats B on Every Slice and Loses 57.60% to 79.06%: 9,510 Reversals, 0 on a Shared Mixture

Devanshu Biswas — Thu, 17 Sep 2026 12:33:03 +0000

Two models, one benchmark, three slices. Model A is ahead on all three — 96.0 / 60.0 / 50.0 against 88.0 / 50.0 / 40.0.

Pooled, A scores 57.60% and B scores 79.06%. A 21.5-point defeat assembled entirely out of victories.

👉 Live, runs in your browser: https://dev48v.infy.uk/ml/day80-simpsons-paradox.html

Declared, not sampled — and no winner decided by a float

Every count on the page is a declared integer. Every accuracy is an exact rational, and every comparison is settled by BigInt cross-multiplication, so a reversal is proved rather than observed through rounding:

// every winner is decided by cross-multiplication, never by a float
const rcmp = (a, b) => { const l = a.n * b.d, r = b.n * a.d;
                         return l < r ? -1 : l > r ? 1 : 0; };

There is no p-value anywhere, because nothing is estimated. What is measured is an exhaustive sweep.

The identity the whole thing rests on

Pooled (micro) accuracy is not a different kind of quantity from the per-slice rates. It is their mean, weighted by slice size:

micro(M) = Σ c_g / Σ n_g = Σ (n_g/N) · (c_g/n_g)
macro(M) = (1/G)  ·  Σ (c_g/n_g)

The two metrics differ in exactly one place — the weights — and the entire paradox lives there. A weighted mean with fixed non-negative weights is monotone in its inputs, so per-slice dominance would be inherited by the pooled number. The reversal requires the two models to carry different weights.

Which is why macro and micro disagree in sign in 9,510 of 9,510 reversals found. Not usually. All of them.

The smallest reversal in existence is nine observations

Sweeping every two-group table with per-cell n from 1 to 8 — 3,748,096 tables, of which 774,400 put A strictly ahead in both groups — gives 9,510 reversals. The minimum total sample across all of them is 9, reached by exactly 4 tables:

group    model A        model B
g1       1/1 = 100%     2/3 = 66.7%
g2       1/4 =  25%     0/1 =  0%
pooled   2/5 =  40%     2/4 = 50%

Four cells and nine data points are enough to rank two models backwards. There is nothing subtle or large-sample about it.

At the other end: the largest pooled reversal found is 5/9 against 2/9 — 55.6 points — and the largest per-group lead that still loses is 3/8, so A is ahead by at least 37.5 points in every group and loses anyway. There is no margin of per-group superiority large enough to make a pooled scalar safe to read.

The negative result is the one worth keeping

Re-run the same sweep with the two models forced onto the same slice mixture:

constraint	comparisons	reversals
the full sweep	774,400	9,510
same n in every group	14,400	0
same slice proportions, totals free	32,544	0
same total n only, mixtures free	72,816	502 (smallest at 10 obs)

Zero is not "rare". With the weights shared, pooled accuracy is a weighted mean using the same weights for both models, and monotonicity does the rest. Matching only the proportions is already enough. Matching only the sample size is not.

Every zero is printed next to the number of comparisons behind it, so an impossibility is distinguishable from an empty search — and the degenerate control (one group, where a reversal would be a contradiction) duly reports 0 over 1,936 comparisons.

What it does not say

A third slice does not make it cheaper: each extra group adds two more cells that must each be strictly won, and the smallest three-group reversal costs 13 observations against nine. The sweep covers two groups at per-cell n ≤ 8, so the minimum is exact for two groups and a claim about nothing else. And the page never says which number is right — macro and micro answer different questions, and choosing needs a deployment mixture a benchmark does not carry.

The narrow claim is the defensible one: a pooled scalar published without the slice sizes that produced it is not a ranking anybody can check.

27 in-page checks, 134 verifier asserts, 0 failures.