<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Devanshu Biswas</title>
    <description>The latest articles on DEV Community by Devanshu Biswas (@dev48v).</description>
    <link>https://dev.to/dev48v</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3929385%2F75a3696c-143d-4252-ba59-6ed4083ca827.jpg</url>
      <title>DEV Community: Devanshu Biswas</title>
      <link>https://dev.to/dev48v</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/dev48v"/>
    <language>en</language>
    <item>
      <title>The Same Link Is Fine on Windows and Broken in CI: 9 of 95 in One Note Folder</title>
      <dc:creator>Devanshu Biswas</dc:creator>
      <pubDate>Mon, 09 Nov 2026 16:58:22 +0000</pubDate>
      <link>https://dev.to/dev48v/the-same-link-is-fine-on-windows-and-broken-in-ci-9-of-95-in-one-note-folder-4g38</link>
      <guid>https://dev.to/dev48v/the-same-link-is-fine-on-windows-and-broken-in-ci-9-of-95-in-one-note-folder-4g38</guid>
      <description>&lt;p&gt;A pile of notes rots from the edges. The note is still there, still readable, still confidently telling you to &lt;em&gt;see the setup doc&lt;/em&gt; — and the setup doc was renamed eight months ago.&lt;/p&gt;

&lt;p&gt;Markdown does not notice. The link renders. The anchor renders. The code block you pasted out of a source file renders exactly as it did the day you pasted it, including the two lines that source file no longer contains.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;stale&lt;/code&gt; reads a folder of notes and reports three kinds of rot — links whose target file is gone, anchors whose heading is gone, and fenced blocks that claim to be copied from a source whose lines have changed — with the evidence that produced every verdict.&lt;/p&gt;

&lt;p&gt;👉 &lt;strong&gt;Live, with every table computed in your browser by a port of the same engine:&lt;/strong&gt; &lt;a href="https://dev48.in/agentlab/vol5-03-stale.html" rel="noopener noreferrer"&gt;https://dev48.in/agentlab/vol5-03-stale.html&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  🔴 The corpus is invented
&lt;/h2&gt;

&lt;p&gt;There is no public archive of one person's notes, and borrowing somebody else's is both a privacy problem and a licensing one. So the &lt;strong&gt;30 notes and 6 source files&lt;/strong&gt; were generated from the fixed seed &lt;code&gt;20260083&lt;/code&gt;, and re-running the generator reproduces the tree byte for byte.&lt;/p&gt;

&lt;p&gt;What that costs: every point estimate below is a property of &lt;em&gt;this&lt;/em&gt; tree and would move on a real one. What transfers is the shape — that the three option flags disagree with each other at all, and roughly by how much, follows from how markdown and filesystems work rather than from the draw. The defects were not placed to hit a target: the generator draws each reference from a distribution, and every number here was measured afterwards by running the tool.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;30 notes, 136 references checked, 45 stale.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The link that passes on your laptop and 404s in CI
&lt;/h2&gt;

&lt;p&gt;NTFS and APFS match filenames without regard to case. ext4 does not.&lt;/p&gt;

&lt;p&gt;A link written as &lt;code&gt;setup.md&lt;/code&gt; against a file called &lt;code&gt;Setup.md&lt;/code&gt; works on the machine it was written on and breaks the moment the notes are published from Linux. A checker built on &lt;code&gt;os.path.exists&lt;/code&gt; reports &lt;strong&gt;PASS&lt;/strong&gt; on the first machine and &lt;strong&gt;FAIL&lt;/strong&gt; on the second, and never mentions that it changed its mind.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;9 of the 95 link references — 9.5% — resolve only case-insensitively&lt;/strong&gt;, across 9 notes and 7 target files.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;notes/cell-formatting.md:8   setup.md#step-1-install
  no notes/setup.md; notes/SETUP.md differs only in case
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;So &lt;code&gt;stale&lt;/code&gt; resolves against an &lt;em&gt;index&lt;/em&gt; of the tree rather than against the filesystem, and the case rule is a flag with a printed value. The answer is a property of the notes.&lt;/p&gt;

&lt;p&gt;It is also not pure subtraction. A link that fails on case never had its &lt;code&gt;#fragment&lt;/code&gt; read, so switching to &lt;code&gt;--case insensitive&lt;/code&gt; &lt;strong&gt;exposes 5 anchor references&lt;/strong&gt; that were hiding behind a link failure. Fixing one check uncovers another.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two anchor conventions, disagreeing about four in ten
&lt;/h2&gt;

&lt;p&gt;GitHub lowercases a heading, &lt;strong&gt;deletes&lt;/strong&gt; punctuation rather than replacing it, hyphenates the spaces, and suffixes a repeat with &lt;code&gt;-1&lt;/code&gt;. The hand-rolled version everybody writes lowercases, hyphenates, and stops.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;heading&lt;/th&gt;
&lt;th&gt;GitHub slug&lt;/th&gt;
&lt;th&gt;naive slug&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;What's next?&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;whats-next&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;what's-next?&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;Step 1: install&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;step-1-install&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;step-1:-install&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;Rollout / rollback&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;rollout--rollback&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;rollout-/-rollback&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;Notes&lt;/code&gt; (second one)&lt;/td&gt;
&lt;td&gt;&lt;code&gt;notes-1&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;notes&lt;/code&gt; — collides&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;rollout--rollback&lt;/code&gt; has &lt;strong&gt;two&lt;/strong&gt; hyphens, because the slash is deleted and both spaces around it survive. Nobody writes that anchor by hand, which is exactly why the naive checker calls the correct one broken.&lt;/p&gt;

&lt;p&gt;Over the corpus's &lt;strong&gt;27 anchor references the two conventions disagree on 11 — 40.7%&lt;/strong&gt; — and in both directions: 6 that only the GitHub rule accepts, 5 that only the naive rule accepts. Neither is a superset of the other, so neither is the safe default.&lt;/p&gt;

&lt;p&gt;The verdict on four in ten anchors is a property of the slugger you picked, not of the notes.&lt;/p&gt;

&lt;h2&gt;
  
  
  One line of whitespace handling moves a third of the quote verdicts
&lt;/h2&gt;

&lt;p&gt;A fenced block can declare where it came from, and that declaration is the thing being checked:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;&lt;span class="c"&gt;&amp;lt;!-- from: ../src/parse.py#L12-L20 --&amp;gt;&lt;/span&gt;
&lt;span class="p"&gt;```&lt;/span&gt;&lt;span class="nl"&gt;python
&lt;/span&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;retry&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;fn&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;attempts&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;backoff&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;0.5&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="bp"&gt;...&lt;/span&gt;
&lt;span class="p"&gt;```&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Of &lt;strong&gt;14 quoted blocks, 7 differ from their source byte for byte. 2 of those 7 — 28.6% — are character-identical once whitespace is normalised&lt;/strong&gt;: re-indented when they were pasted, or carrying a trailing space. Same notes, same sources, one flag.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;stale&lt;/code&gt; defaults to &lt;code&gt;exact&lt;/code&gt;, because indentation is semantic in the language most of these blocks are written in. But it prints &lt;code&gt;resolved-after-normalising&lt;/code&gt; rather than &lt;code&gt;resolved&lt;/code&gt;, so you can see which of the two answers you are reading.&lt;/p&gt;

&lt;h2&gt;
  
  
  The number that decides whether anyone reads the report
&lt;/h2&gt;

&lt;p&gt;45 findings sounds like a working detector right up until you look at where they land.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;They land in 22 of 30 notes — 73%.&lt;/strong&gt; And across all eight combinations of the three flags, the base rate never drops below &lt;strong&gt;60%&lt;/strong&gt;:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;case / anchors / quote&lt;/th&gt;
&lt;th&gt;stale&lt;/th&gt;
&lt;th&gt;notes flagged&lt;/th&gt;
&lt;th&gt;base rate&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;strict / slug / exact&lt;/td&gt;
&lt;td&gt;45&lt;/td&gt;
&lt;td&gt;22&lt;/td&gt;
&lt;td&gt;73%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;strict / raw / exact&lt;/td&gt;
&lt;td&gt;46&lt;/td&gt;
&lt;td&gt;22&lt;/td&gt;
&lt;td&gt;73%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;insensitive / slug / exact&lt;/td&gt;
&lt;td&gt;36&lt;/td&gt;
&lt;td&gt;20&lt;/td&gt;
&lt;td&gt;67%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;insensitive / raw / normalised&lt;/td&gt;
&lt;td&gt;38&lt;/td&gt;
&lt;td&gt;18&lt;/td&gt;
&lt;td&gt;60%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;insensitive / slug / normalised&lt;/td&gt;
&lt;td&gt;34&lt;/td&gt;
&lt;td&gt;18&lt;/td&gt;
&lt;td&gt;60%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Loosening every flag at once removes 11 findings and still leaves 60% of the pile flagged. A tool that flags three quarters of your notes has not triaged anything — it has handed you the folder back.&lt;/p&gt;

&lt;p&gt;The flags are not the lever. The findings are &lt;strong&gt;not independent&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;One file that got renamed shows up once per note that linked to it. Collapse the findings onto the thing you would actually repair and &lt;strong&gt;45 findings become 25 fixes, with the top four clearing 18 of them&lt;/strong&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;  6  target-missing: notes/setup-guide.md
  5  target-missing: notes/index.md
  4  target-missing: notes/old-glossary.md
  3  case-only-match: notes/2026/Handoff-2025.md
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Nothing about the detector changed. Only what it counted. That is the difference between a list of symptoms and a list of jobs, and it is the only honest answer a sensitive detector has to its own base rate.&lt;/p&gt;

&lt;h2&gt;
  
  
  How it is checked
&lt;/h2&gt;

&lt;p&gt;The page ships a JS port of the Python package. The day's verifier lifts that port &lt;strong&gt;straight out of the HTML&lt;/strong&gt; — the bytes the browser runs — evaluates it in Node, runs the Python package as a subprocess, and compares them on 76 trees: the real corpus under all eight flag settings, 60 generated ones, and the edge cases.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1,933 references compared — verdict, reason, and evidence string, 9,162 assertions, 0 failures.&lt;/strong&gt; Plus 84 pytest cases and a 36-check panel in the page itself.&lt;/p&gt;

&lt;p&gt;A port that is only checked against itself is not checked. The failure worth guarding against is the one that does not look like a failure: both halves run, both produce a report, and the reports differ.&lt;/p&gt;

&lt;h2&gt;
  
  
  Offline, standard library, no model
&lt;/h2&gt;

&lt;p&gt;No key, no network, no LLM. Python 3.8+, &lt;code&gt;dependencies = []&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;MIT. Part of Agent Lab Vol 5 — &lt;em&gt;reading the pile you already have&lt;/em&gt;.&lt;/p&gt;

</description>
      <category>python</category>
      <category>opensource</category>
      <category>ai</category>
      <category>documentation</category>
    </item>
    <item>
      <title>TTFT and TPOT: The Latency Number You Report Hides the One Users Feel</title>
      <dc:creator>Devanshu Biswas</dc:creator>
      <pubDate>Mon, 09 Nov 2026 16:57:42 +0000</pubDate>
      <link>https://dev.to/dev48v/ttft-and-tpot-the-latency-number-you-report-hides-the-one-users-feel-31ff</link>
      <guid>https://dev.to/dev48v/ttft-and-tpot-the-latency-number-you-report-hides-the-one-users-feel-31ff</guid>
      <description>&lt;p&gt;Push a simulated serving queue from 0.2 to 3.0 requests a second and mean end-to-end latency gets &lt;strong&gt;17x&lt;/strong&gt; worse.&lt;/p&gt;

&lt;p&gt;Over the same range, the 99th percentile time to first token gets &lt;strong&gt;762x&lt;/strong&gt; worse — &lt;strong&gt;206 ms&lt;/strong&gt; to &lt;strong&gt;157 s&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Both numbers describe the same 300 requests. Only one of them describes the empty box a user is staring at.&lt;/p&gt;

&lt;p&gt;👉 &lt;strong&gt;Live, with sliders for arrival rate, prompt length and batch size:&lt;/strong&gt; &lt;a href="https://dev48.in/ai/days/day83-ttft-vs-tpot.html" rel="noopener noreferrer"&gt;https://dev48.in/ai/days/day83-ttft-vs-tpot.html&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The workload is simulated and seeded
&lt;/h2&gt;

&lt;p&gt;There is no GPU here and no model. There is a discrete-event simulation with a fixed seed (&lt;code&gt;mulberry32&lt;/code&gt;, seed &lt;code&gt;20261109&lt;/code&gt;): 300 requests, exponential inter-arrival times, log-normal prompt lengths (median 512 tokens) and output lengths (median 180 tokens), a FIFO queue, and a batch.&lt;/p&gt;

&lt;p&gt;The cost model is two lines:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;prefill(group) = 8 ms  + 0.08 ms x (total prompt tokens)
decode step    = 18 ms + 1.2  ms x (sequences in the batch)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;None of these figures are measurements of any real serving stack, and none of them are claims about one. They are the arithmetic of serving, which is the part that survives a change of hardware. Everything below comes out of that loop, and a Node verifier re-derives all of it from an independent token-by-token replay.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two clocks, glued by an identity
&lt;/h2&gt;

&lt;p&gt;Time to first token is the silence: queueing plus prefill, before anything appears. Time per output token is the cadence after it. They compose exactly:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;e2e = TTFT + TPOT x (output tokens - 1)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Not approximately. Across all 300 requests the worst disagreement is &lt;strong&gt;3.64e-12 ms&lt;/strong&gt;. The minus one is there because the first token is already paid for inside TTFT.&lt;/p&gt;

&lt;p&gt;That identity is why a single total is useless for diagnosis. A slow total is a long silence, a slow cadence, or a long answer — three different causes with three different fixes, and the total cannot tell you which. In the default run the request with the worst TTFT and the request with the worst end-to-end are &lt;strong&gt;not the same request&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  The queue lands almost entirely on the silence
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;arrival rate&lt;/th&gt;
&lt;th&gt;TTFT p50&lt;/th&gt;
&lt;th&gt;TTFT p99&lt;/th&gt;
&lt;th&gt;TPOT p50&lt;/th&gt;
&lt;th&gt;tok/s&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;0.2 req/s&lt;/td&gt;
&lt;td&gt;55.5 ms&lt;/td&gt;
&lt;td&gt;206.3 ms&lt;/td&gt;
&lt;td&gt;20.40 ms&lt;/td&gt;
&lt;td&gt;48.2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.8 req/s&lt;/td&gt;
&lt;td&gt;71.1 ms&lt;/td&gt;
&lt;td&gt;5,966 ms&lt;/td&gt;
&lt;td&gt;26.46 ms&lt;/td&gt;
&lt;td&gt;191.9&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1.2 req/s&lt;/td&gt;
&lt;td&gt;13,377 ms&lt;/td&gt;
&lt;td&gt;24,010 ms&lt;/td&gt;
&lt;td&gt;29.14 ms&lt;/td&gt;
&lt;td&gt;260.2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2.0 req/s&lt;/td&gt;
&lt;td&gt;53,606 ms&lt;/td&gt;
&lt;td&gt;110,884 ms&lt;/td&gt;
&lt;td&gt;29.30 ms&lt;/td&gt;
&lt;td&gt;266.8&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3.0 req/s&lt;/td&gt;
&lt;td&gt;76,188 ms&lt;/td&gt;
&lt;td&gt;157,311 ms&lt;/td&gt;
&lt;td&gt;29.30 ms&lt;/td&gt;
&lt;td&gt;267.4&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;TTFT p99 moves &lt;strong&gt;762x&lt;/strong&gt;. TPOT p50 moves &lt;strong&gt;1.44x&lt;/strong&gt;. Mean end-to-end — the column most dashboards actually plot — moves &lt;strong&gt;17x&lt;/strong&gt;, because it is dominated by how many tokens the answers had, which is a property of the questions and not of the server.&lt;/p&gt;

&lt;p&gt;A deploy that moved the first column from 55 ms to 76 seconds would show up on a mean-latency chart as a moderate regression.&lt;/p&gt;

&lt;h2&gt;
  
  
  TPOT is batch size, and it is exactly batch size
&lt;/h2&gt;

&lt;p&gt;A decode step reads the weights out of memory once and uses them for every sequence in the batch. Fill the batch and time it:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;full batch of&lt;/th&gt;
&lt;th&gt;TPOT&lt;/th&gt;
&lt;th&gt;tokens/s to one user&lt;/th&gt;
&lt;th&gt;tokens/s for the server&lt;/th&gt;
&lt;th&gt;ms of step per sequence&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;19.20 ms&lt;/td&gt;
&lt;td&gt;52.1&lt;/td&gt;
&lt;td&gt;50.9&lt;/td&gt;
&lt;td&gt;19.200&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;8&lt;/td&gt;
&lt;td&gt;27.60 ms&lt;/td&gt;
&lt;td&gt;36.2&lt;/td&gt;
&lt;td&gt;246.8&lt;/td&gt;
&lt;td&gt;3.450&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;32&lt;/td&gt;
&lt;td&gt;56.40 ms&lt;/td&gt;
&lt;td&gt;17.7&lt;/td&gt;
&lt;td&gt;420.4&lt;/td&gt;
&lt;td&gt;1.762&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;64&lt;/td&gt;
&lt;td&gt;94.80 ms&lt;/td&gt;
&lt;td&gt;10.5&lt;/td&gt;
&lt;td&gt;476.2&lt;/td&gt;
&lt;td&gt;1.481&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The same number, divided two ways. Per token the fixed cost falls towards 1.2 ms; per &lt;strong&gt;sequence&lt;/strong&gt; it only ever rises. The server gets 9.4x faster and every individual stream gets 4.9x slower, and only one of those is on a capacity dashboard.&lt;/p&gt;

&lt;p&gt;Prompt length does not appear in the step expression at all. Measured in the clean room, TPOT across a 64x range of prompt lengths at four batch sizes has a total spread of &lt;strong&gt;8.38e-13 ms&lt;/strong&gt; — floating-point noise, nothing else.&lt;/p&gt;

&lt;h2&gt;
  
  
  Except that it leaks, and the leak is scheduling
&lt;/h2&gt;

&lt;p&gt;Put the same server under load and prompt length suddenly does move TPOT. Batch 16, 1.4 req/s. The control column prices prefill at zero and changes nothing else — same arrivals, same batch, same decoding:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;median prompt&lt;/th&gt;
&lt;th&gt;TPOT, prefill free&lt;/th&gt;
&lt;th&gt;TPOT, prefill priced&lt;/th&gt;
&lt;th&gt;server time in prefill&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;128&lt;/td&gt;
&lt;td&gt;31.035 ms&lt;/td&gt;
&lt;td&gt;32.730 ms&lt;/td&gt;
&lt;td&gt;2.96%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;512&lt;/td&gt;
&lt;td&gt;31.035 ms&lt;/td&gt;
&lt;td&gt;36.462 ms&lt;/td&gt;
&lt;td&gt;8.44%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2048&lt;/td&gt;
&lt;td&gt;31.035 ms&lt;/td&gt;
&lt;td&gt;49.119 ms&lt;/td&gt;
&lt;td&gt;27.08%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4096&lt;/td&gt;
&lt;td&gt;31.035 ms&lt;/td&gt;
&lt;td&gt;63.025 ms&lt;/td&gt;
&lt;td&gt;43.01%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The control column is identical to the last digit, because with prefill free the prompt does not exist. The priced column doubles. Nothing about decoding changed — somebody else's 4,000-token prefill is holding the accelerator while your sequence waits for its next token. This is the effect chunked prefill exists to spread out, and it is why a TPOT regression can be caused by a change nobody made to decoding.&lt;/p&gt;

&lt;h2&gt;
  
  
  The assertion that failed
&lt;/h2&gt;

&lt;p&gt;I expected the classic trade: make the scheduler wait to accumulate a full batch before prefilling, fuse the prefills, buy throughput, pay in TTFT. Half of it showed up.&lt;/p&gt;

&lt;p&gt;2,048-token prompts, 0.8 req/s, batch 16:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;batch wait window&lt;/th&gt;
&lt;th&gt;TTFT p50&lt;/th&gt;
&lt;th&gt;tok/s&lt;/th&gt;
&lt;th&gt;prefill share&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;0 ms&lt;/td&gt;
&lt;td&gt;207.5 ms&lt;/td&gt;
&lt;td&gt;191.7&lt;/td&gt;
&lt;td&gt;17.726%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;150 ms&lt;/td&gt;
&lt;td&gt;369.2 ms&lt;/td&gt;
&lt;td&gt;191.6&lt;/td&gt;
&lt;td&gt;17.668%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;400 ms&lt;/td&gt;
&lt;td&gt;640.0 ms&lt;/td&gt;
&lt;td&gt;191.5&lt;/td&gt;
&lt;td&gt;17.578%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;800 ms&lt;/td&gt;
&lt;td&gt;1,052.6 ms&lt;/td&gt;
&lt;td&gt;191.3&lt;/td&gt;
&lt;td&gt;17.494%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The fusing worked exactly as advertised: prefill's share of server time falls monotonically. Throughput moves &lt;strong&gt;−0.22%&lt;/strong&gt;. This server is decode-bound, so the time the optimisation freed was never the bottleneck.&lt;/p&gt;

&lt;p&gt;TTFT p50 went up &lt;strong&gt;5.1x&lt;/strong&gt; anyway. The optimisation succeeded and only the bill arrived — charged to users, in a column the throughput dashboard does not have.&lt;/p&gt;

&lt;h2&gt;
  
  
  A percentile over requests is not a percentile over tokens
&lt;/h2&gt;

&lt;p&gt;p99 TPOT usually means: average each request's cadence, take the percentile over requests. That gives a two-token reply exactly as many votes as a two-thousand-token one. Weight by tokens emitted instead and the number moves — at 1.4 req/s, &lt;strong&gt;32.12 ms&lt;/strong&gt; per request against &lt;strong&gt;31.27 ms&lt;/strong&gt; per token.&lt;/p&gt;

&lt;p&gt;Neither is wrong. They answer different questions, and a dashboard that does not say which one it is answering is not reporting anything.&lt;/p&gt;

&lt;h2&gt;
  
  
  Budget the two separately, because one SLO cannot hold both
&lt;/h2&gt;

&lt;p&gt;Take a budget of &lt;strong&gt;TTFT p90 ≤ 1 s&lt;/strong&gt; and &lt;strong&gt;TPOT p90 ≤ 60 ms&lt;/strong&gt;, and sweep batch size at 2.0 req/s:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;batch&lt;/th&gt;
&lt;th&gt;TTFT p90&lt;/th&gt;
&lt;th&gt;TPOT p90&lt;/th&gt;
&lt;th&gt;verdict&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;1,102,077 ms&lt;/td&gt;
&lt;td&gt;19.2 ms&lt;/td&gt;
&lt;td&gt;TTFT over&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;td&gt;31,947 ms&lt;/td&gt;
&lt;td&gt;42.6 ms&lt;/td&gt;
&lt;td&gt;TTFT over&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;32&lt;/td&gt;
&lt;td&gt;3,796 ms&lt;/td&gt;
&lt;td&gt;64.8 ms&lt;/td&gt;
&lt;td&gt;both over&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;64&lt;/td&gt;
&lt;td&gt;154 ms&lt;/td&gt;
&lt;td&gt;73.0 ms&lt;/td&gt;
&lt;td&gt;TPOT over&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Small batches blow the TTFT ceiling on queueing. Large ones blow the TPOT ceiling on step time. &lt;strong&gt;No batch size satisfies both.&lt;/strong&gt; That is not a tuning problem, it is a capacity verdict — and the split metrics are the only thing that can state it, because the same budget at 0.8 req/s and batch 16 is met comfortably (TTFT p90 &lt;strong&gt;129 ms&lt;/strong&gt;, TPOT p90 &lt;strong&gt;30.9 ms&lt;/strong&gt;).&lt;/p&gt;

&lt;p&gt;An end-to-end budget would have been satisfied or violated by a change in answer length that nobody deployed.&lt;/p&gt;

&lt;h2&gt;
  
  
  One bug worth keeping
&lt;/h2&gt;

&lt;p&gt;The batch-wait scheduler advances the clock to &lt;code&gt;head.arrival + batchWindow&lt;/code&gt;, then tests whether the window has expired. That sum rounds a hair low in a double — &lt;code&gt;299.9999999999999&lt;/code&gt; for a 300 ms window — so the test says no, the clock is advanced to the same instant again, and the loop spins forever on one timestamp. Fourteen window values hung before the comparison got an epsilon. The page now asserts that all 28 window-and-batch combinations terminate and serve all 300 requests.&lt;/p&gt;

&lt;h2&gt;
  
  
  What is checked
&lt;/h2&gt;

&lt;p&gt;The page runs &lt;strong&gt;47 assertions&lt;/strong&gt; in your browser, including the nearest-rank percentile against a hand-checked array, a fixed seed reproducing a run string-identically, the end-to-end identity to 1e-9 for every request, TTFT monotone in arrival rate, and TPOT invariant to prompt length in isolation and demonstrably not invariant under load.&lt;/p&gt;

&lt;p&gt;A Node verifier lifts the same engine out of the HTML and adds &lt;strong&gt;177&lt;/strong&gt; more, including an independent replay that records a timestamp for every one of ~70,000 tokens and re-derives TTFT, TPOT and end-to-end from that log rather than from the three scalars the page keeps. Both at zero failures.&lt;/p&gt;

&lt;p&gt;Report TTFT and TPOT separately, at p90 or p99, and say which output length you assumed when you turned them into a total. Then when the total moves, the identity tells you which of the three terms moved.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>performance</category>
      <category>python</category>
    </item>
    <item>
      <title>Prefix tuning changes no weights: 128 numbers steering a frozen model</title>
      <dc:creator>Devanshu Biswas</dc:creator>
      <pubDate>Mon, 09 Nov 2026 16:57:01 +0000</pubDate>
      <link>https://dev.to/dev48v/prefix-tuning-changes-no-weights-128-numbers-steering-a-frozen-model-3fci</link>
      <guid>https://dev.to/dev48v/prefix-tuning-changes-no-weights-128-numbers-steering-a-frozen-model-3fci</guid>
      <description>&lt;p&gt;LoRA computes &lt;code&gt;W_eff = W + BA&lt;/code&gt;. It changes a weight matrix by a low-rank delta, and at serve time you can add the delta in and forget it was ever separate.&lt;/p&gt;

&lt;p&gt;Prefix tuning never forms a delta. It leaves every weight bit-for-bit where it was and prepends &lt;code&gt;P&lt;/code&gt; learned key rows and &lt;code&gt;P&lt;/code&gt; learned value rows to the attention of &lt;strong&gt;every&lt;/strong&gt; layer. Two &lt;code&gt;concat&lt;/code&gt; lines, and the softmax does the rest.&lt;/p&gt;

&lt;p&gt;That difference is the whole subject, and it is easy to lose in the phrase "parameter-efficient fine-tuning", which gets applied to both.&lt;/p&gt;

&lt;p&gt;👉 &lt;strong&gt;Live, with the training run executing in your browser:&lt;/strong&gt; &lt;a href="https://dev48.in/dl/day83-prefix-tuning.html" rel="noopener noreferrer"&gt;https://dev48.in/dl/day83-prefix-tuning.html&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  🔴 The model is a toy and the numbers are a property of it
&lt;/h2&gt;

&lt;p&gt;The page builds a miniature decoder from a fixed seed: &lt;code&gt;d = 8&lt;/code&gt;, 2 attention layers, 6 tokens per sequence, a 12-token vocabulary, 2 classes. &lt;strong&gt;624 numbers in total.&lt;/strong&gt; Re-running reproduces it exactly.&lt;/p&gt;

&lt;p&gt;What that costs: every point estimate below is a property of &lt;em&gt;this&lt;/em&gt; model, and would move on a real one. What transfers is the shape — where the gradient stops, what the parameter count is a function of, and what happens when the prefix is left to grow unbounded.&lt;/p&gt;

&lt;h2&gt;
  
  
  The whole method
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;K&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;Pk&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;concat&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;k&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;   &lt;span class="c1"&gt;// P learned key rows, then the real tokens&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;V&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;Pv&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;concat&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;v&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;   &lt;span class="c1"&gt;// P learned value rows, then the real tokens&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;No &lt;code&gt;W&lt;/code&gt; is read differently. No &lt;code&gt;W&lt;/code&gt; is written. Every query now scores against &lt;code&gt;P + T&lt;/code&gt; candidates instead of &lt;code&gt;T&lt;/code&gt;, and a softmax that must sum to one has no choice but to spend some of its mass on the new rows.&lt;/p&gt;

&lt;p&gt;That is the leverage. The prefix does not add a term to the output — it &lt;strong&gt;takes mass away&lt;/strong&gt; from the real tokens and redirects it to vectors you control.&lt;/p&gt;

&lt;p&gt;You cannot prepend to &lt;code&gt;q&lt;/code&gt;. A query belongs to a token, and you are not adding tokens. That asymmetry is why this is key/value prefixing and not "prompt tuning".&lt;/p&gt;

&lt;h2&gt;
  
  
  Where the gradient stops
&lt;/h2&gt;

&lt;p&gt;After the softmax derivative you hold a gradient for all &lt;code&gt;P + T&lt;/code&gt; key rows and all &lt;code&gt;P + T&lt;/code&gt; value rows.&lt;/p&gt;

&lt;p&gt;The first &lt;code&gt;P&lt;/code&gt; of each were not computed from anything. They &lt;em&gt;are&lt;/em&gt; the parameters, so their gradient is the answer and the chain ends there. The remaining &lt;code&gt;T&lt;/code&gt; came out of &lt;code&gt;W_k&lt;/code&gt; and &lt;code&gt;W_v&lt;/code&gt;, so their gradient keeps travelling — back through those matrices, into &lt;code&gt;x&lt;/code&gt;, and down to the layer below, because layer 0's prefix is only reachable through layer 1's attention.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;dWq&lt;/code&gt;, &lt;code&gt;dWk&lt;/code&gt;, &lt;code&gt;dWv&lt;/code&gt;, &lt;code&gt;dWo&lt;/code&gt; are never formed. Not computed and then zeroed — the lines that would build them are absent, and the page checks its own &lt;code&gt;backward&lt;/code&gt; source for that.&lt;/p&gt;

&lt;p&gt;Backprop is written out by hand, and every coordinate is checked against a central finite difference of the loss, which knows nothing about the derivation. &lt;strong&gt;All 64 coordinates of a P = 2 prefix agree to better than 1e-5 relative.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  It trains, and the base comes back untouched
&lt;/h2&gt;

&lt;p&gt;The task is built so the frozen model cannot already do it: one signal token hidden among filler, label = which signal. The base sits at &lt;strong&gt;0.6941&lt;/strong&gt; against a chance value of &lt;code&gt;ln(2) = 0.6931&lt;/code&gt;, and gets &lt;strong&gt;62.5%&lt;/strong&gt; right.&lt;/p&gt;

&lt;p&gt;200 steps of SGD with momentum, moving &lt;strong&gt;128 numbers&lt;/strong&gt;:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;value&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;loss, before&lt;/td&gt;
&lt;td&gt;0.6941&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;loss, after 200 steps&lt;/td&gt;
&lt;td&gt;0.00337&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;training accuracy&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;held-out accuracy, 24 unseen examples&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;frozen numbers changed&lt;/td&gt;
&lt;td&gt;0 of 624&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The last row is the claim of the method stated as a hash. All 624 frozen numbers are fingerprinted before the loop and again afterwards, compared element by element, and the base object is handed to &lt;code&gt;Object.freeze&lt;/code&gt; recursively first — so an accidental write could not land even if a line tried.&lt;/p&gt;

&lt;p&gt;And the control that makes the rest mean something: with &lt;code&gt;P = 0&lt;/code&gt; the prefix-aware forward pass returns &lt;strong&gt;bit-identical&lt;/strong&gt; logits to a forward pass written without any prefix code at all, compared with &lt;code&gt;Object.is&lt;/code&gt;. With &lt;code&gt;P = 0&lt;/code&gt;, 80 steps at a learning rate of 5 leave the loss exactly where it started.&lt;/p&gt;

&lt;h2&gt;
  
  
  Longer is not better
&lt;/h2&gt;

&lt;p&gt;Every extra slot is more capacity &lt;em&gt;and&lt;/em&gt; more attention mass stolen from the input. The same loop, the same seed, at each &lt;code&gt;P&lt;/code&gt;:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;P&lt;/th&gt;
&lt;th&gt;trainable&lt;/th&gt;
&lt;th&gt;final loss&lt;/th&gt;
&lt;th&gt;first step under 0.05&lt;/th&gt;
&lt;th&gt;attention mass on prefix&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0.69412&lt;/td&gt;
&lt;td&gt;never&lt;/td&gt;
&lt;td&gt;0.000&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;32&lt;/td&gt;
&lt;td&gt;0.01390&lt;/td&gt;
&lt;td&gt;127&lt;/td&gt;
&lt;td&gt;0.144&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;64&lt;/td&gt;
&lt;td&gt;0.00342&lt;/td&gt;
&lt;td&gt;51&lt;/td&gt;
&lt;td&gt;0.560&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;128&lt;/td&gt;
&lt;td&gt;0.00337&lt;/td&gt;
&lt;td&gt;58&lt;/td&gt;
&lt;td&gt;0.656&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;8&lt;/td&gt;
&lt;td&gt;256&lt;/td&gt;
&lt;td&gt;0.00492&lt;/td&gt;
&lt;td&gt;83&lt;/td&gt;
&lt;td&gt;0.738&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;td&gt;512&lt;/td&gt;
&lt;td&gt;0.00775&lt;/td&gt;
&lt;td&gt;134&lt;/td&gt;
&lt;td&gt;0.839&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A &lt;strong&gt;single slot&lt;/strong&gt; — 32 numbers — already solves it, just slower. The curve bottoms out at &lt;code&gt;P = 4&lt;/code&gt; and then turns around: by &lt;code&gt;P = 16&lt;/code&gt; the loss is worse than at &lt;code&gt;P = 4&lt;/code&gt;, while &lt;strong&gt;0.839&lt;/strong&gt; of the attention mass is going to the prefix. The model is spending most of its softmax on vectors instead of on its own input.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the reparameterisation trick is actually for
&lt;/h2&gt;

&lt;p&gt;Li &amp;amp; Liang reported that optimising the table directly was unstable and generated it from a small embedding through an MLP instead, discarding the MLP afterwards. On this page the generator's output layer is bounded, so &lt;code&gt;|prefix| ≤ 45.255&lt;/code&gt; is arithmetic rather than a hope.&lt;/p&gt;

&lt;p&gt;Running both parameterisations at the same seed across a learning-rate sweep:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;learning rate&lt;/th&gt;
&lt;th&gt;direct loss&lt;/th&gt;
&lt;th&gt;direct |prefix|&lt;/th&gt;
&lt;th&gt;direct mass on prefix&lt;/th&gt;
&lt;th&gt;reparam loss&lt;/th&gt;
&lt;th&gt;reparam |prefix|&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;0.6&lt;/td&gt;
&lt;td&gt;0.0034&lt;/td&gt;
&lt;td&gt;15&lt;/td&gt;
&lt;td&gt;0.656&lt;/td&gt;
&lt;td&gt;0.0002&lt;/td&gt;
&lt;td&gt;32.7&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;0.0003&lt;/td&gt;
&lt;td&gt;23&lt;/td&gt;
&lt;td&gt;0.710&lt;/td&gt;
&lt;td&gt;0.7514&lt;/td&gt;
&lt;td&gt;45.1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12&lt;/td&gt;
&lt;td&gt;0.0000&lt;/td&gt;
&lt;td&gt;40&lt;/td&gt;
&lt;td&gt;0.832&lt;/td&gt;
&lt;td&gt;0.7991&lt;/td&gt;
&lt;td&gt;45.3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;50&lt;/td&gt;
&lt;td&gt;0.0000&lt;/td&gt;
&lt;td&gt;79&lt;/td&gt;
&lt;td&gt;0.957&lt;/td&gt;
&lt;td&gt;0.5086&lt;/td&gt;
&lt;td&gt;45.3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;100&lt;/td&gt;
&lt;td&gt;2.5643&lt;/td&gt;
&lt;td&gt;12,450&lt;/td&gt;
&lt;td&gt;1.000&lt;/td&gt;
&lt;td&gt;1.3659&lt;/td&gt;
&lt;td&gt;45.3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;0.0000&lt;/td&gt;
&lt;td&gt;6,880&lt;/td&gt;
&lt;td&gt;1.000&lt;/td&gt;
&lt;td&gt;1.1548&lt;/td&gt;
&lt;td&gt;45.3&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The column that matters is not the loss, it is the norm. The direct prefix is free coordinates and nothing bounds them: at &lt;code&gt;lr = 100&lt;/code&gt; the run ends at &lt;strong&gt;2.5643&lt;/strong&gt;, above the chance value of 0.6931, with &lt;strong&gt;1.000&lt;/strong&gt; of the attention mass on the prefix. That last number is the mechanism of the failure — one prefix key grows until it wins the softmax outright, the real tokens get nothing, and the model is no longer reading its input.&lt;/p&gt;

&lt;p&gt;Be honest about the other half. Bounding does not make it better everywhere: at &lt;code&gt;lr = 0.6&lt;/code&gt; the reparameterised run reaches 0.00020 against the direct run's 0.00337, but high up the sweep it stops learning rather than exploding. It trades a blow-up for a ceiling.&lt;/p&gt;

&lt;p&gt;There is also a parameter count that gets quoted two ways. With the generator you &lt;strong&gt;train 640&lt;/strong&gt; numbers and &lt;strong&gt;ship 128&lt;/strong&gt;, because the MLP is run once at the end and thrown away. Without it you train and ship the same 128.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the ratio gets better as the model gets bigger
&lt;/h2&gt;

&lt;p&gt;A prefix costs &lt;code&gt;2 × L × P × d&lt;/code&gt;. The vocabulary is not in that formula, and neither is the feed-forward width — which is where most of a transformer's parameters actually live.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;model shape&lt;/th&gt;
&lt;th&gt;prefix&lt;/th&gt;
&lt;th&gt;full fine-tune&lt;/th&gt;
&lt;th&gt;prefix params&lt;/th&gt;
&lt;th&gt;share&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;this page's toy&lt;/td&gt;
&lt;td&gt;P = 4&lt;/td&gt;
&lt;td&gt;624&lt;/td&gt;
&lt;td&gt;128&lt;/td&gt;
&lt;td&gt;20.5128%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;d=768, L=12, V=50k&lt;/td&gt;
&lt;td&gt;P = 10&lt;/td&gt;
&lt;td&gt;123,532,032&lt;/td&gt;
&lt;td&gt;184,320&lt;/td&gt;
&lt;td&gt;0.1492%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;d=4096, L=32, V=32k&lt;/td&gt;
&lt;td&gt;P = 16&lt;/td&gt;
&lt;td&gt;6,573,522,944&lt;/td&gt;
&lt;td&gt;4,194,304&lt;/td&gt;
&lt;td&gt;0.0638%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;d=8192, L=80, V=32k&lt;/td&gt;
&lt;td&gt;P = 16&lt;/td&gt;
&lt;td&gt;64,686,653,440&lt;/td&gt;
&lt;td&gt;20,971,520&lt;/td&gt;
&lt;td&gt;0.0324%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The toy row is the honest one to be suspicious of — 20.5% of a 624-number model is not a headline. The share falls to &lt;strong&gt;0.0324%&lt;/strong&gt; by the bottom row because the prefix grows with &lt;code&gt;L × d&lt;/code&gt; while the model grows with &lt;code&gt;L × d²&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  The trade, stated plainly
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;LoRA&lt;/th&gt;
&lt;th&gt;prefix tuning&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;the edit&lt;/td&gt;
&lt;td&gt;&lt;code&gt;W_eff = W + BA&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;K = [Pk ; xWk]&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;trainable here&lt;/td&gt;
&lt;td&gt;256 (r=4, on Wq and Wv)&lt;/td&gt;
&lt;td&gt;128 (P = 4)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;where the gradient stops&lt;/td&gt;
&lt;td&gt;in A and B, inside the product&lt;/td&gt;
&lt;td&gt;in the first P rows of K and V&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;can it be folded into W?&lt;/td&gt;
&lt;td&gt;yes, add BA once&lt;/td&gt;
&lt;td&gt;no — it is extra KV rows&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;cost at inference&lt;/td&gt;
&lt;td&gt;none, once folded&lt;/td&gt;
&lt;td&gt;P extra keys/values, every layer, forever&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;two tasks in one batch&lt;/td&gt;
&lt;td&gt;one adapter at a time&lt;/td&gt;
&lt;td&gt;a different prefix per row&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Read the last three rows together, because they are the whole trade. LoRA's delta disappears into the weights at serve time and costs nothing afterwards — but you can only have one folded in. A prefix never disappears: it occupies &lt;code&gt;P&lt;/code&gt; slots of every attention, in every layer, for every token, forever. What you buy with that is the last row — two requests in the same batch can carry two different prefixes, because the prefix is data, not weights.&lt;/p&gt;

&lt;h2&gt;
  
  
  And it is not a prompt
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;a discrete prompt of 4 tokens&lt;/th&gt;
&lt;th&gt;a soft prefix, P = 4&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;lives in&lt;/td&gt;
&lt;td&gt;the 12 embedding points the vocab has&lt;/td&gt;
&lt;td&gt;all of R⁸&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;distinct settings reachable&lt;/td&gt;
&lt;td&gt;20,736&lt;/td&gt;
&lt;td&gt;continuous&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;enters the network at&lt;/td&gt;
&lt;td&gt;layer 0 only&lt;/td&gt;
&lt;td&gt;every one of the 2 layers&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;deeper layers see it&lt;/td&gt;
&lt;td&gt;second-hand, via the layers below&lt;/td&gt;
&lt;td&gt;directly, with its own learned vectors&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;numbers you train&lt;/td&gt;
&lt;td&gt;0 — you pick token ids&lt;/td&gt;
&lt;td&gt;128&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A four-token prompt over this vocabulary has &lt;strong&gt;20,736&lt;/strong&gt; possible settings and you would have to try them. The soft version is continuous, differentiable, and — decisively — injected into every layer rather than only the first.&lt;/p&gt;

&lt;p&gt;The page carries 38 in-browser assertions covering all of this: the frozen weights, the gradient check, the &lt;code&gt;P = 0&lt;/code&gt; identity, the bound, and the divergence. All 38 pass, with the engine on the page being the same one every table above was computed by.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://dev48.in/dl/day83-prefix-tuning.html" rel="noopener noreferrer"&gt;https://dev48.in/dl/day83-prefix-tuning.html&lt;/a&gt;&lt;/p&gt;

</description>
      <category>machinelearning</category>
      <category>deeplearning</category>
      <category>ai</category>
      <category>python</category>
    </item>
    <item>
      <title>Your Top-Rated Shop Has One Rating: Partial Pooling Cuts Total Error 5.30x</title>
      <dc:creator>Devanshu Biswas</dc:creator>
      <pubDate>Mon, 09 Nov 2026 16:56:21 +0000</pubDate>
      <link>https://dev.to/dev48v/your-top-rated-shop-has-one-rating-partial-pooling-cuts-total-error-530x-40i0</link>
      <guid>https://dev.to/dev48v/your-top-rated-shop-has-one-rating-partial-pooling-cuts-total-error-530x-40i0</guid>
      <description>&lt;p&gt;Forty shops. One has 860 ratings. Five have exactly one.&lt;/p&gt;

&lt;p&gt;Shop #0 has a single rating, and that rating says &lt;strong&gt;80.43&lt;/strong&gt;. It sits at the top of the leaderboard. Its true quality is &lt;strong&gt;57.61&lt;/strong&gt; — above average, but nowhere near the top of anything.&lt;/p&gt;

&lt;p&gt;Partial pooling estimates it at &lt;strong&gt;53.12&lt;/strong&gt;, and across all forty shops it cuts total squared error from &lt;strong&gt;1763.35&lt;/strong&gt; to &lt;strong&gt;332.54&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;👉 &lt;strong&gt;Live, with the shrinkage weight recomputed as you drag the sample size:&lt;/strong&gt; &lt;a href="https://dev48.in/ml/day83-partial-pooling.html" rel="noopener noreferrer"&gt;https://dev48.in/ml/day83-partial-pooling.html&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The data is synthetic and seeded
&lt;/h2&gt;

&lt;p&gt;Every number here comes from a seeded generator: 40 groups, sizes &lt;code&gt;1, 1, 1, 1, 1, 2, … 688, 860&lt;/code&gt; for &lt;strong&gt;4,345 observations&lt;/strong&gt;, true group means drawn from &lt;code&gt;Normal(50, 4²)&lt;/code&gt; and observations from &lt;code&gt;Normal(θ_g, 12²)&lt;/code&gt;. That is the entire point of synthesising it — the &lt;strong&gt;true&lt;/strong&gt; group means are known exactly, so every estimator is scored against the truth rather than against another estimate. Nothing on the page is fitted to real shop data and none of these figures are claims about real shops.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three estimators, scored against truth
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;estimator&lt;/th&gt;
&lt;th&gt;what it does&lt;/th&gt;
&lt;th&gt;total squared error&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;no pooling&lt;/td&gt;
&lt;td&gt;each group's own mean&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;1763.35&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;complete pooling&lt;/td&gt;
&lt;td&gt;one global mean for all 40&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;794.38&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;partial pooling&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;shrink each toward the mean&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;332.54&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;5.30x better than trusting every group, 2.39x better than trusting none of them. Nothing told it which groups were small.&lt;/p&gt;

&lt;h2&gt;
  
  
  The weight is a ratio of two variances, not a tuning knob
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;lambda_g = tau^2 / (tau^2 + sigma^2 / n_g)
estimate = lambda_g * ybar_g + (1 - lambda_g) * mu
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;sigma²/n&lt;/code&gt; is how much of a group's average is sampling noise. &lt;code&gt;tau²&lt;/code&gt; is how much real spread there is between groups. &lt;code&gt;lambda&lt;/code&gt; is the share of what the group said that is signal. It is bounded in [0,1] by construction, because it is a ratio whose denominator contains its numerator.&lt;/p&gt;

&lt;p&gt;At &lt;code&gt;sigma = 12&lt;/code&gt;:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;spread&lt;/th&gt;
&lt;th&gt;n = 1&lt;/th&gt;
&lt;th&gt;n = 3&lt;/th&gt;
&lt;th&gt;n = 10&lt;/th&gt;
&lt;th&gt;n = 30&lt;/th&gt;
&lt;th&gt;n = 100&lt;/th&gt;
&lt;th&gt;n = 860&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;τ = 1&lt;/td&gt;
&lt;td&gt;0.0069&lt;/td&gt;
&lt;td&gt;0.0204&lt;/td&gt;
&lt;td&gt;0.0649&lt;/td&gt;
&lt;td&gt;0.1724&lt;/td&gt;
&lt;td&gt;0.4098&lt;/td&gt;
&lt;td&gt;0.8566&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;τ = 4&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.1000&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;0.2500&lt;/td&gt;
&lt;td&gt;0.5263&lt;/td&gt;
&lt;td&gt;0.7692&lt;/td&gt;
&lt;td&gt;0.9174&lt;/td&gt;
&lt;td&gt;0.9896&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;τ = 16&lt;/td&gt;
&lt;td&gt;0.6400&lt;/td&gt;
&lt;td&gt;0.8421&lt;/td&gt;
&lt;td&gt;0.9467&lt;/td&gt;
&lt;td&gt;0.9816&lt;/td&gt;
&lt;td&gt;0.9944&lt;/td&gt;
&lt;td&gt;0.9993&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Complete pooling is this method at &lt;code&gt;lambda = 0&lt;/code&gt;. No pooling is this method at &lt;code&gt;lambda = 1&lt;/code&gt;. They are not rivals of it, they are its two endpoints — and notice there is no threshold anywhere. No group is ever dropped for being "too small"; it is continuously discounted in proportion to how much of its number is noise.&lt;/p&gt;

&lt;h2&gt;
  
  
  Both variances have to be estimated, and one of them is hard
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;sigma²&lt;/code&gt; is easy: pool the within-group deviations over &lt;code&gt;N − G = 4345 − 40 = 4305&lt;/code&gt; degrees of freedom. The one-rating shops contribute nothing to it, which is correct — a single point has no spread.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;tau²&lt;/code&gt; is the hard one, because you never see the true means. The naive answer, the spread of the 40 observed group averages, is &lt;strong&gt;83.658&lt;/strong&gt; against a truth of &lt;strong&gt;16&lt;/strong&gt; — a &lt;strong&gt;5.23x&lt;/strong&gt; overstatement, because the one-rating shops are scattered by ±12 points of pure noise. Use that and &lt;code&gt;lambda&lt;/code&gt; goes to nearly 1 and you shrink nothing.&lt;/p&gt;

&lt;p&gt;The moment identity fixes it. The precision-weighted scatter &lt;code&gt;Q&lt;/code&gt; of the group averages has expectation &lt;code&gt;G − 1 = 39&lt;/code&gt; if the groups are truly identical, so the excess is real spread:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;quantity&lt;/th&gt;
&lt;th&gt;value&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Q observed&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;421.61&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;E[Q] if all groups identical&lt;/td&gt;
&lt;td&gt;39&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;τ̂² recovered&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;14.697&lt;/strong&gt; (true 16.000)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;σ̂² recovered&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;148.689&lt;/strong&gt; (true 144.000)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Over 300 seeded worlds the estimates average &lt;strong&gt;144.069&lt;/strong&gt; and &lt;strong&gt;16.124&lt;/strong&gt; against truths of 144 and 16, and &lt;code&gt;τ̂²&lt;/code&gt; was clamped to zero in 0 of the 300. The clamp is not decoration — a sample can genuinely look &lt;em&gt;tighter&lt;/em&gt; than chance predicts, and the honest answer there is &lt;code&gt;τ̂² = 0&lt;/code&gt;, which collapses the whole method to complete pooling.&lt;/p&gt;

&lt;h2&gt;
  
  
  One group, sample size varied, truth held fixed
&lt;/h2&gt;

&lt;p&gt;The true score of shop #0 is 57.61 in every row — the true means come from their own seeded stream, so changing the sample size cannot disturb them. Only the evidence changes.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;ratings&lt;/th&gt;
&lt;th&gt;λ&lt;/th&gt;
&lt;th&gt;raw average&lt;/th&gt;
&lt;th&gt;partial pooled&lt;/th&gt;
&lt;th&gt;pulled by&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;0.0900&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;80.43&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;53.12&lt;/td&gt;
&lt;td&gt;27.31&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;0.2855&lt;/td&gt;
&lt;td&gt;63.67&lt;/td&gt;
&lt;td&gt;54.24&lt;/td&gt;
&lt;td&gt;9.44&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;td&gt;0.6137&lt;/td&gt;
&lt;td&gt;59.23&lt;/td&gt;
&lt;td&gt;55.87&lt;/td&gt;
&lt;td&gt;3.36&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;128&lt;/td&gt;
&lt;td&gt;0.9309&lt;/td&gt;
&lt;td&gt;59.51&lt;/td&gt;
&lt;td&gt;58.89&lt;/td&gt;
&lt;td&gt;0.62&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2000&lt;/td&gt;
&lt;td&gt;0.9968&lt;/td&gt;
&lt;td&gt;57.78&lt;/td&gt;
&lt;td&gt;57.75&lt;/td&gt;
&lt;td&gt;0.02&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  It is not a hack — it is the estimator with lower expected error
&lt;/h2&gt;

&lt;p&gt;Over 300 independently seeded worlds:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;estimator&lt;/th&gt;
&lt;th&gt;mean total squared error&lt;/th&gt;
&lt;th&gt;best world&lt;/th&gt;
&lt;th&gt;worst world&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;no pooling&lt;/td&gt;
&lt;td&gt;1421.02&lt;/td&gt;
&lt;td&gt;337.75&lt;/td&gt;
&lt;td&gt;2977.61&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;complete pooling&lt;/td&gt;
&lt;td&gt;660.40&lt;/td&gt;
&lt;td&gt;312.53&lt;/td&gt;
&lt;td&gt;1280.33&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;partial pooling&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;290.45&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;96.52&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;507.22&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Partial pooling beat both at once in &lt;strong&gt;300 of 300&lt;/strong&gt;. Its &lt;em&gt;worst&lt;/em&gt; world (507.22) is better than no pooling's &lt;em&gt;best&lt;/em&gt; one (337.75).&lt;/p&gt;

&lt;p&gt;That is the James–Stein result in applied form. For a normal mean in three or more dimensions, using each group's own average — unbiased, maximum likelihood, the obvious thing — is &lt;strong&gt;inadmissible&lt;/strong&gt;: a shrunken estimator has lower expected total squared error for &lt;em&gt;every&lt;/em&gt; possible set of true means. On 30 groups of 5, the classical James–Stein estimator scores &lt;strong&gt;350.76&lt;/strong&gt; against the MLE's &lt;strong&gt;806.59&lt;/strong&gt;, a 2.30x improvement, winning &lt;strong&gt;58 of 60&lt;/strong&gt; seeded worlds. Not 60 of 60 — inadmissibility is a statement about the expectation, and a page claiming otherwise would be lying to you.&lt;/p&gt;

&lt;h2&gt;
  
  
  Error moved out of variance and into bias, deliberately
&lt;/h2&gt;

&lt;p&gt;Hold the forty true means fixed and redraw only the observations, 400 times. Now "bias" means something.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;estimator, group&lt;/th&gt;
&lt;th&gt;n&lt;/th&gt;
&lt;th&gt;bias&lt;/th&gt;
&lt;th&gt;bias²&lt;/th&gt;
&lt;th&gt;variance&lt;/th&gt;
&lt;th&gt;MSE&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;no pooling, shop #0&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;−0.107&lt;/td&gt;
&lt;td&gt;0.011&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;143.386&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;143.398&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;partial, shop #0&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;−7.029&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;49.403&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;1.071&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;50.473&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;no pooling, shop #39&lt;/td&gt;
&lt;td&gt;860&lt;/td&gt;
&lt;td&gt;−0.022&lt;/td&gt;
&lt;td&gt;0.000&lt;/td&gt;
&lt;td&gt;0.166&lt;/td&gt;
&lt;td&gt;0.167&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;partial, shop #39&lt;/td&gt;
&lt;td&gt;860&lt;/td&gt;
&lt;td&gt;0.021&lt;/td&gt;
&lt;td&gt;0.000&lt;/td&gt;
&lt;td&gt;0.164&lt;/td&gt;
&lt;td&gt;0.164&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The one-rating group's no-pooling estimate is almost perfectly &lt;strong&gt;unbiased&lt;/strong&gt; and almost perfectly &lt;strong&gt;useless&lt;/strong&gt; — its error is essentially all variance. Partial pooling takes on a real, systematic, never-averaging-away pull of −7.03 toward the middle, and the variance collapses by 134x. Total error falls 2.84x. The 860-rating group is left alone.&lt;/p&gt;

&lt;p&gt;Being unbiased was never the objective. Being close was.&lt;/p&gt;

&lt;h2&gt;
  
  
  The leaderboard lesson, measured
&lt;/h2&gt;

&lt;p&gt;Pick the single top shop, 200 seeded worlds:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;ranking&lt;/th&gt;
&lt;th&gt;true score of the winner&lt;/th&gt;
&lt;th&gt;names the genuinely best shop&lt;/th&gt;
&lt;th&gt;groups with ≤3 ratings in its top 5&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;by raw average&lt;/td&gt;
&lt;td&gt;53.76&lt;/td&gt;
&lt;td&gt;32 / 200&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;2.87 of 5&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;by partial pooling&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;57.12&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;78 / 200&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;0.29 of 5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;best possible&lt;/td&gt;
&lt;td&gt;58.77&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Twelve of the forty groups have three ratings or fewer. The raw top five holds on average 2.87 of them — not because small groups are good, but because small groups have the widest distribution of outcomes and a maximum picks from the tails.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this costs
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;14 of 40 individual groups come out worse.&lt;/strong&gt; Shop #11 has 3 ratings and a raw error of 0.157; partial pooling moves it to 4.052. The method optimises the sum, and individual groups pay for it.&lt;/p&gt;

&lt;p&gt;So if your decision is about one named group — a safety alert, a fraud case, a single hospital's mortality rate — the shrunken number is the wrong one to argue from. And the pooling assumption has to hold: it says the groups are exchangeable draws from one population, and shrinking toward a mean that no group belongs to makes things worse, not better.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this does not cover
&lt;/h2&gt;

&lt;p&gt;One normal-normal model with a scalar outcome, no predictors, no crossed or nested levels, and no full Bayes — &lt;code&gt;tau²&lt;/code&gt; is a point estimate here and the uncertainty in it is never propagated. Group sizes vary but the within-group variance is assumed common across groups. Outcomes are Gaussian, so nothing here is a binomial or count hierarchy.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;37 in-page assertions, 56 verifier assertions, 0 failures.&lt;/strong&gt; The verifier shares none of the page's machinery: a different generator, and the shrinkage weight recovered three other ways — trapezoid quadrature over the Bayes posterior, a golden-section minimiser on the penalised least-squares objective, and a grid search over λ against empirical mean squared error on 300,000 draws.&lt;/p&gt;

</description>
      <category>machinelearning</category>
      <category>datascience</category>
      <category>statistics</category>
      <category>python</category>
    </item>
    <item>
      <title>RFC 6570 URI Templates: the `?` That Must Not Appear, and the Name That Must Repeat</title>
      <dc:creator>Devanshu Biswas</dc:creator>
      <pubDate>Mon, 09 Nov 2026 16:55:40 +0000</pubDate>
      <link>https://dev.to/dev48v/rfc-6570-uri-templates-the-that-must-not-appear-and-the-name-that-must-repeat-19im</link>
      <guid>https://dev.to/dev48v/rfc-6570-uri-templates-the-that-must-not-appear-and-the-name-that-must-repeat-19im</guid>
      <description>&lt;p&gt;&lt;code&gt;/users/{id}/repos{?page,per_page}&lt;/code&gt;. It looks like string formatting with a nicer syntax. It is a 40-page RFC because every one of the decisions you make by hand when you concatenate a URL has a wrong answer that ships.&lt;/p&gt;

&lt;p&gt;I implemented all four levels &lt;strong&gt;from the grammar and the §3.2.1 expansion algorithm&lt;/strong&gt;, not from a library: eight operators, both modifiers, strings, lists and associative arrays, with a per-character trace of which rule produced each run of output.&lt;/p&gt;

&lt;p&gt;👉 &lt;strong&gt;Live, expanding in your browser:&lt;/strong&gt; &lt;a href="https://dev48.in/solve/day83-uri-template.html" rel="noopener noreferrer"&gt;https://dev48.in/solve/day83-uri-template.html&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Every figure below is a count from the verifier or the in-page checks.&lt;/p&gt;

&lt;h2&gt;
  
  
  The whole specification is five columns
&lt;/h2&gt;

&lt;p&gt;There is no branch anywhere in the engine on the operator character. There is one table.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;operator&lt;/th&gt;
&lt;th&gt;prefix&lt;/th&gt;
&lt;th&gt;separator&lt;/th&gt;
&lt;th&gt;named&lt;/th&gt;
&lt;th&gt;ifemp&lt;/th&gt;
&lt;th&gt;kept unencoded&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;{var}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;&lt;code&gt;,&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;unreserved only&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;{+var}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;&lt;code&gt;,&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;unreserved &lt;strong&gt;+ reserved&lt;/strong&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;{#var}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;#&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;,&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;unreserved &lt;strong&gt;+ reserved&lt;/strong&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;{.var}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;.&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;.&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;unreserved only&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;{/var}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;/&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;/&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;unreserved only&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;{;var}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;;&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;;&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;yes&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;``&lt;/td&gt;
&lt;td&gt;unreserved only&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;{?var}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;?&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;&amp;amp;&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;yes&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;=&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;unreserved only&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;{&amp;amp;var}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;&amp;amp;&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;&amp;amp;&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;yes&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;=&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;unreserved only&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Three of the eight are named. Two of the eight allow reserved characters. One column separates &lt;code&gt;{var}&lt;/code&gt; from &lt;code&gt;{+var}&lt;/code&gt;, and that column is the one people get wrong.&lt;/p&gt;

&lt;h2&gt;
  
  
  The prefix is emitted once, and only if something survived
&lt;/h2&gt;

&lt;p&gt;This is the rule the spec exists for. The &lt;code&gt;?&lt;/code&gt; is not written when the expression is parsed — it is written when the first &lt;em&gt;defined&lt;/em&gt; variable is about to be.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;variables defined&lt;/th&gt;
&lt;th&gt;&lt;code&gt;{?a,b,c}&lt;/code&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;none&lt;/td&gt;
&lt;td&gt;`&lt;code&gt; (empty string, not &lt;/code&gt;?`)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;c&lt;/code&gt; only&lt;/td&gt;
&lt;td&gt;&lt;code&gt;?c=3&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;a&lt;/code&gt; and &lt;code&gt;c&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;&lt;code&gt;?a=1&amp;amp;c=3&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;all three&lt;/td&gt;
&lt;td&gt;&lt;code&gt;?a=1&amp;amp;b=2&amp;amp;c=3&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;No dangling &lt;code&gt;?&lt;/code&gt;, no &lt;code&gt;&amp;amp;&amp;amp;&lt;/code&gt;, no leading &lt;code&gt;&amp;amp;&lt;/code&gt;. There is nowhere to get it wrong because the flag lives in the loop.&lt;/p&gt;

&lt;p&gt;And undefined is not the same as empty. An absent key and &lt;code&gt;null&lt;/code&gt; are skipped entirely; an empty &lt;strong&gt;list&lt;/strong&gt; and an empty &lt;strong&gt;map&lt;/strong&gt; are also skipped, because there is nothing to join; an empty &lt;strong&gt;string&lt;/strong&gt; is a value and keeps its name. A list holding one empty string has length 1, so it is defined:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;expression&lt;/th&gt;
&lt;th&gt;&lt;code&gt;""&lt;/code&gt;&lt;/th&gt;
&lt;th&gt;&lt;code&gt;[]&lt;/code&gt;&lt;/th&gt;
&lt;th&gt;&lt;code&gt;[""]&lt;/code&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;{?v}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;?v=&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;``&lt;/td&gt;
&lt;td&gt;&lt;code&gt;?v=&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;{;v}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;;v&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;``&lt;/td&gt;
&lt;td&gt;&lt;code&gt;;v=&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;X{.v}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;X.&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;X&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;X.&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Three operators, three different right answers for the same value, all from the &lt;code&gt;ifemp&lt;/code&gt; column.&lt;/p&gt;

&lt;h2&gt;
  
  
  &lt;code&gt;{+var}&lt;/code&gt; is not "the same but shorter"
&lt;/h2&gt;

&lt;p&gt;Of the 128 ASCII code points, the default allow-set percent-encodes &lt;strong&gt;62&lt;/strong&gt;. The reserved allow-set encodes &lt;strong&gt;44&lt;/strong&gt;. The 18 characters in the gap are the URI structural punctuation &lt;code&gt;: / ? # [ ] @ ! $ &amp;amp; ' ( ) * + , ; =&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;{hello}   -&amp;gt;  Hello%20World%21
{+hello}  -&amp;gt;  Hello%20World!
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;!&lt;/code&gt; is a sub-delim. It is data under &lt;code&gt;{var}&lt;/code&gt; and punctuation under &lt;code&gt;{+var}&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The consequence that actually bites is idempotence. &lt;code&gt;+&lt;/code&gt; copies an existing &lt;code&gt;%XX&lt;/code&gt; triplet through untouched; the default re-encodes the &lt;code&gt;%&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;value "/a b/" expanded twice
  {+p}  -&amp;gt;  /a%20b/            stable
  {p}   -&amp;gt;  %252Fa%2520b%252F  destroyed
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That is the right way round. The default's job is to make a value opaque, and running an opaque value through again should not be a no-op.&lt;/p&gt;

&lt;h2&gt;
  
  
  The star is not "spread". It is "use my separator."
&lt;/h2&gt;

&lt;p&gt;Without &lt;code&gt;*&lt;/code&gt;, a composite value is joined with &lt;strong&gt;commas&lt;/strong&gt;, whatever operator you used. With &lt;code&gt;*&lt;/code&gt;, the operator's own separator takes over.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;op&lt;/th&gt;
&lt;th&gt;&lt;code&gt;{op list}&lt;/code&gt;&lt;/th&gt;
&lt;th&gt;&lt;code&gt;{op list*}&lt;/code&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;(none)&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;red,green,blue&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;red,green,blue&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;+&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;red,green,blue&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;red,green,blue&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;#&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;#red,green,blue&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;#red,green,blue&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;.&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;.red,green,blue&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;.red.green.blue&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;/red,green,blue&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;/red/green/blue&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;;&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;;list=red,green,blue&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;;list=red;list=green;list=blue&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;?&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;?list=red,green,blue&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;?list=red&amp;amp;list=green&amp;amp;list=blue&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;&amp;amp;&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;&amp;amp;list=red,green,blue&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;&amp;amp;list=red&amp;amp;list=green&amp;amp;list=blue&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Read the bottom three rows twice. &lt;strong&gt;A named operator with an exploded list repeats the variable name.&lt;/strong&gt; The tempting reading of &lt;code&gt;*&lt;/code&gt; is "drop the name and spread the values", which gives &lt;code&gt;?red&amp;amp;green&amp;amp;blue&lt;/code&gt; — three parameters named &lt;code&gt;red&lt;/code&gt;, &lt;code&gt;green&lt;/code&gt; and &lt;code&gt;blue&lt;/code&gt;, with no values. It is not a list at all.&lt;/p&gt;

&lt;p&gt;The five unnamed operators expand identically whether you get this right or wrong. A test suite built on &lt;code&gt;{/list*}&lt;/code&gt; will never catch it.&lt;/p&gt;

&lt;h2&gt;
  
  
  An unexploded map loses the pairing entirely
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;keys = { semi: ";", dot: ".", comma: "," }

{keys}    -&amp;gt;  semi,%3B,dot,.,comma,%2C
{keys*}   -&amp;gt;  semi=%3B,dot=.,comma=%2C
{+keys}   -&amp;gt;  semi,;,dot,.,comma,,
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;{keys}&lt;/code&gt; is six comma-separated items, not three pairs. Nothing in the output says which is which. And &lt;code&gt;{+keys}&lt;/code&gt; is the same map under the reserved allow-set, where the comma value is a literal comma — the output has a run of three commas and is ambiguous by design.&lt;/p&gt;

&lt;h2&gt;
  
  
  &lt;code&gt;:3&lt;/code&gt; counts characters of the decoded value
&lt;/h2&gt;

&lt;p&gt;Truncation happens &lt;strong&gt;before&lt;/strong&gt; encoding, and it counts code points, not bytes and not output characters.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;path = "/foo/bar"
{path:4}   -&amp;gt;  %2Ffoo    4 characters in, 6 out
{+path:4}  -&amp;gt;  /foo      4 characters in, 4 out
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Counting the modifier against the output would cut a &lt;code&gt;%2F&lt;/code&gt; triplet in half. Counting bytes would cut a 4-byte emoji in half — &lt;code&gt;{e:2}&lt;/code&gt; over three &lt;code&gt;😀&lt;/code&gt; gives exactly &lt;code&gt;%F0%9F%98%80%F0%9F%98%80&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;A prefix modifier on a list or a map is &lt;strong&gt;illegal&lt;/strong&gt;, not "truncate each member". The engine throws on all four forms tested.&lt;/p&gt;

&lt;h2&gt;
  
  
  How it is checked
&lt;/h2&gt;

&lt;p&gt;The verifier never reads the page's test data. It carries its own copy of every expansion printed in RFC 6570, and a &lt;strong&gt;second expander&lt;/strong&gt; written from the §3.2.1 pseudocode in a deliberately different shape — regex scan, no trace, encoding via Node's &lt;code&gt;Buffer&lt;/code&gt; instead of the page's hand-written UTF-8.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;120&lt;/strong&gt; RFC example expansions, byte for byte: 120/120&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;232&lt;/strong&gt; combinations in an operator × modifier × value-shape sweep, both implementations: 232/232 identical&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;128&lt;/strong&gt; ASCII code points, both allow-sets&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;695&lt;/strong&gt; trace segments across the 120 vectors, every one reassembling into its expansion and carrying a rule&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;63&lt;/strong&gt; verifier asserts, &lt;strong&gt;0&lt;/strong&gt; failures&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;55&lt;/strong&gt; in-page checks, &lt;strong&gt;0&lt;/strong&gt; failures&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The page itself holds &lt;strong&gt;115&lt;/strong&gt; section vectors plus &lt;strong&gt;66&lt;/strong&gt; Appendix A level vectors and runs them on load, so the badge is measuring the same engine the tool uses.&lt;/p&gt;

&lt;h2&gt;
  
  
  The one thing worth taking away
&lt;/h2&gt;

&lt;p&gt;If you are building a query string by concatenation, the bug is not the escaping. The bug is that you decided what &lt;code&gt;?&lt;/code&gt; means before you knew whether any parameter was defined.&lt;/p&gt;

&lt;p&gt;Day 83 of SolveFromZero. Tomorrow: the reverse direction, and why matching a URI back to a template is not the same problem.&lt;/p&gt;

</description>
      <category>api</category>
      <category>javascript</category>
      <category>programming</category>
      <category>webdev</category>
    </item>
    <item>
      <title>Five Prompts Beat Five Samples: 21/24 vs 15/24 on the Identical Five-Call Bill</title>
      <dc:creator>Devanshu Biswas</dc:creator>
      <pubDate>Mon, 09 Nov 2026 16:54:59 +0000</pubDate>
      <link>https://dev.to/dev48v/five-prompts-beat-five-samples-2124-vs-1524-on-the-identical-five-call-bill-12pl</link>
      <guid>https://dev.to/dev48v/five-prompts-beat-five-samples-2124-vs-1524-on-the-identical-five-call-bill-12pl</guid>
      <description>&lt;p&gt;Two methods, same model, same 24 questions, same budget of five calls per question.&lt;/p&gt;

&lt;p&gt;Self-consistency — one prompt, five samples at temperature 0.3, majority vote — scores &lt;strong&gt;15/24&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;An ensemble of five &lt;em&gt;different&lt;/em&gt; prompts, plurality vote, scores &lt;strong&gt;21/24&lt;/strong&gt;. The best single prompt on its own scores &lt;strong&gt;17/24&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;👉 &lt;strong&gt;Live, with the correlation dial and every aggregation rule on sliders:&lt;/strong&gt; &lt;a href="https://dev48.in/prompt/day83-prompt-ensembling.html" rel="noopener noreferrer"&gt;https://dev48.in/prompt/day83-prompt-ensembling.html&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  These are not the same technique
&lt;/h2&gt;

&lt;p&gt;Self-consistency varies &lt;strong&gt;the sampler's dice&lt;/strong&gt;. Every one of the k samples read identical words, so the vote cancels sampling noise and nothing else.&lt;/p&gt;

&lt;p&gt;Prompt ensembling varies &lt;strong&gt;the prompt&lt;/strong&gt;: a different phrasing, a different output format, the options listed in a different order. The members can disagree for a reason, and a vote only buys you something when the voters can disagree for a reason.&lt;/p&gt;

&lt;p&gt;The ceiling makes the difference concrete. Push self-consistency to k = 301 and it lands on &lt;strong&gt;17/24&lt;/strong&gt; — and not approximately. At k = 301 it reproduces that one prompt's own greedy pick on &lt;strong&gt;24 of 24&lt;/strong&gt; questions. The plurality over samples converges to the prompt's most likely answer, so the prompt &lt;em&gt;is&lt;/em&gt; the ceiling. k buys stability, never a second point of view.&lt;/p&gt;

&lt;h2&gt;
  
  
  The engine simulates nothing about language
&lt;/h2&gt;

&lt;p&gt;24 arithmetic questions are DATA. Each has four options: the true value, an off-by-one, a digit swap, and a rounded figure. The in-page self-check re-evaluates &lt;code&gt;17 × 24&lt;/code&gt; and the other 23 rather than trusting the stored answer.&lt;/p&gt;

&lt;p&gt;A "prompt variant" is a scoring profile over those four options — how strongly it is drawn to the truth, to each of the three traps, and to whatever is listed first. Each question is tempting for one particular trap. A member goes wrong where its weakness meets a matching question, which is what makes two variants fail on &lt;em&gt;different&lt;/em&gt; questions rather than the same ones.&lt;/p&gt;

&lt;p&gt;Noise is a &lt;strong&gt;hash&lt;/strong&gt; of (seed, member, question, option), not a stream. Reordering the member array therefore cannot change what any member answers, and the verifier proves it across all &lt;strong&gt;120&lt;/strong&gt; permutations of five members on all 24 questions.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;prompt variant&lt;/th&gt;
&lt;th&gt;accuracy&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;plain ask&lt;/td&gt;
&lt;td&gt;17/24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;options as a table&lt;/td&gt;
&lt;td&gt;17/24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;work it out step by step&lt;/td&gt;
&lt;td&gt;16/24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;answer with the value only&lt;/td&gt;
&lt;td&gt;11/24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;same ask, options reversed&lt;/td&gt;
&lt;td&gt;15/24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;plurality over all five&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;21/24&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;confidence-weighted&lt;/td&gt;
&lt;td&gt;22/24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Borda (rank aggregation)&lt;/td&gt;
&lt;td&gt;23/24&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Mean pairwise error correlation across those five: &lt;strong&gt;−0.144&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  The finding: an ensemble only pays when the errors are uncorrelated
&lt;/h2&gt;

&lt;p&gt;Same machinery, same five members, same 24 questions, same seed, same 5× cost. One dial: blend each member's profile toward the first member's, so the set slides from five genuinely different prompts to five rewordings of one.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;blend&lt;/th&gt;
&lt;th&gt;error correlation&lt;/th&gt;
&lt;th&gt;best single&lt;/th&gt;
&lt;th&gt;ensemble&lt;/th&gt;
&lt;th&gt;gain&lt;/th&gt;
&lt;th&gt;agreement when wrong&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;0.0 — five different prompts&lt;/td&gt;
&lt;td&gt;−0.144&lt;/td&gt;
&lt;td&gt;17/24&lt;/td&gt;
&lt;td&gt;21/24&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;+4&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;53.3%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.2&lt;/td&gt;
&lt;td&gt;−0.079&lt;/td&gt;
&lt;td&gt;18/24&lt;/td&gt;
&lt;td&gt;22/24&lt;/td&gt;
&lt;td&gt;+4&lt;/td&gt;
&lt;td&gt;60.0%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.4&lt;/td&gt;
&lt;td&gt;0.000&lt;/td&gt;
&lt;td&gt;18/24&lt;/td&gt;
&lt;td&gt;19/24&lt;/td&gt;
&lt;td&gt;+1&lt;/td&gt;
&lt;td&gt;64.0%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.6&lt;/td&gt;
&lt;td&gt;0.172&lt;/td&gt;
&lt;td&gt;17/24&lt;/td&gt;
&lt;td&gt;18/24&lt;/td&gt;
&lt;td&gt;+1&lt;/td&gt;
&lt;td&gt;73.3%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.8&lt;/td&gt;
&lt;td&gt;0.346&lt;/td&gt;
&lt;td&gt;17/24&lt;/td&gt;
&lt;td&gt;17/24&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;80.0%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1.0 — five rewordings of one&lt;/td&gt;
&lt;td&gt;0.508&lt;/td&gt;
&lt;td&gt;17/24&lt;/td&gt;
&lt;td&gt;16/24&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;−1&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;87.5%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The gain does not merely shrink. It crosses zero and goes negative. Cost does not move.&lt;/p&gt;

&lt;p&gt;The question to measure before shipping an ensemble is not "are my prompts different?" It is "do my prompts fail on different questions?"&lt;/p&gt;

&lt;h2&gt;
  
  
  A majority of biased members is confidently wrong
&lt;/h2&gt;

&lt;p&gt;That last column is the part that should worry you.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;member set&lt;/th&gt;
&lt;th&gt;error correlation&lt;/th&gt;
&lt;th&gt;best single&lt;/th&gt;
&lt;th&gt;ensemble&lt;/th&gt;
&lt;th&gt;agreement when wrong&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;5 different prompts&lt;/td&gt;
&lt;td&gt;−0.144&lt;/td&gt;
&lt;td&gt;17/24&lt;/td&gt;
&lt;td&gt;21/24&lt;/td&gt;
&lt;td&gt;53.3%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;5 rewordings of one prompt&lt;/td&gt;
&lt;td&gt;0.644&lt;/td&gt;
&lt;td&gt;17/24&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;16/24&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;92.5%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;5 identical calls&lt;/td&gt;
&lt;td&gt;1.000&lt;/td&gt;
&lt;td&gt;17/24&lt;/td&gt;
&lt;td&gt;17/24&lt;/td&gt;
&lt;td&gt;100.0%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A diverse ensemble that is wrong is wrong about 3-to-2 and looks shaky. A correlated ensemble that is wrong is wrong 5-to-0 and looks certain. The correlated set is less accurate &lt;strong&gt;and&lt;/strong&gt; more convincing about it — so a gate that routes high-agreement answers past review fails in exactly the same direction the ensemble does.&lt;/p&gt;

&lt;p&gt;And the bottom row is the limit case. Five identical calls at temperature zero return the single prompt's own sheet under &lt;strong&gt;12 of 12&lt;/strong&gt; rule × tie-break combinations, for 5× the bill. If your "variants" are politeness edits, that row is what you bought.&lt;/p&gt;

&lt;h2&gt;
  
  
  The tie-break nobody writes down is order-dependent
&lt;/h2&gt;

&lt;p&gt;Five members over four options tie more often than people expect. Somebody has to decide, and in most code nobody decided: a loop that keeps a running maximum returns whichever tied option the &lt;em&gt;first&lt;/em&gt; member voted for.&lt;/p&gt;

&lt;p&gt;Over all 120 permutations of the five members, that default returns different answers on &lt;strong&gt;2 of 24&lt;/strong&gt; questions. Listing the same five members in reverse takes the sheet from &lt;strong&gt;21/24 to 20/24&lt;/strong&gt;. Nothing about the model, the prompts or the seed changed — only the order of an array.&lt;/p&gt;

&lt;p&gt;Breaking ties on a property of the &lt;em&gt;options&lt;/em&gt; instead (lowest listed position, summed confidence, Borda score) is invariant across all 120 orderings. It costs nothing.&lt;/p&gt;

&lt;h2&gt;
  
  
  k is not a slider you push right
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;k&lt;/th&gt;
&lt;th&gt;cost&lt;/th&gt;
&lt;th&gt;plurality&lt;/th&gt;
&lt;th&gt;questions decided by tie-break&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;1×&lt;/td&gt;
&lt;td&gt;17/24&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;2×&lt;/td&gt;
&lt;td&gt;18/24&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;12&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;3×&lt;/td&gt;
&lt;td&gt;20/24&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;4×&lt;/td&gt;
&lt;td&gt;18/24&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;9&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;5×&lt;/td&gt;
&lt;td&gt;21/24&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Accuracy goes 17 → 18 → 20 → &lt;strong&gt;18&lt;/strong&gt; → 21. Adding a member is not a safe move; a weak member's votes land somewhere. And an even k sends half your sheet to the tie-break: k = 2 decides 12 of 24 questions by policy rather than by vote.&lt;/p&gt;

&lt;p&gt;If you ensemble, use an odd k.&lt;/p&gt;

&lt;h2&gt;
  
  
  And no aggregation rule wins everywhere
&lt;/h2&gt;

&lt;p&gt;Same engine, seven seeds:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;seed&lt;/th&gt;
&lt;th&gt;best single&lt;/th&gt;
&lt;th&gt;plurality&lt;/th&gt;
&lt;th&gt;confidence-weighted&lt;/th&gt;
&lt;th&gt;Borda&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;20260083&lt;/td&gt;
&lt;td&gt;17/24&lt;/td&gt;
&lt;td&gt;21/24&lt;/td&gt;
&lt;td&gt;22/24&lt;/td&gt;
&lt;td&gt;23/24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;7&lt;/td&gt;
&lt;td&gt;19/24&lt;/td&gt;
&lt;td&gt;20/24&lt;/td&gt;
&lt;td&gt;19/24&lt;/td&gt;
&lt;td&gt;21/24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;101&lt;/td&gt;
&lt;td&gt;18/24&lt;/td&gt;
&lt;td&gt;19/24&lt;/td&gt;
&lt;td&gt;19/24&lt;/td&gt;
&lt;td&gt;22/24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4242&lt;/td&gt;
&lt;td&gt;20/24&lt;/td&gt;
&lt;td&gt;22/24&lt;/td&gt;
&lt;td&gt;20/24&lt;/td&gt;
&lt;td&gt;22/24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;90210&lt;/td&gt;
&lt;td&gt;18/24&lt;/td&gt;
&lt;td&gt;21/24&lt;/td&gt;
&lt;td&gt;20/24&lt;/td&gt;
&lt;td&gt;22/24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;31337&lt;/td&gt;
&lt;td&gt;17/24&lt;/td&gt;
&lt;td&gt;20/24&lt;/td&gt;
&lt;td&gt;21/24&lt;/td&gt;
&lt;td&gt;22/24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;555&lt;/td&gt;
&lt;td&gt;17/24&lt;/td&gt;
&lt;td&gt;19/24&lt;/td&gt;
&lt;td&gt;21/24&lt;/td&gt;
&lt;td&gt;21/24&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Plurality beats confidence-weighted on three seeds and loses on three. One run does not establish an ordering between them.&lt;/p&gt;

&lt;p&gt;Borda is at least as good as both on &lt;strong&gt;7 of 7&lt;/strong&gt;, which is the strongest claim this page will make for any rule — and it is the only rule with a cost that is not in the call count. Borda needs every member to rank all the options, which is more output tokens per call, and a member that will not rank reliably cannot be in a Borda ensemble at all. It is also the only rule that can elect an option &lt;strong&gt;nobody&lt;/strong&gt; put first: five members whose first choices are 0, 0, 1, 1, 3 elect option 2, the universal second choice, with zero first-place votes. A plurality vote cannot see it.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to actually do
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Write variants that &lt;strong&gt;fail differently&lt;/strong&gt;, not variants that &lt;strong&gt;read differently&lt;/strong&gt;. Change the format, the option order, the reasoning demanded. Politeness edits produce the 5-identical-calls row.&lt;/li&gt;
&lt;li&gt;Fan out in parallel, or the latency is k times too.&lt;/li&gt;
&lt;li&gt;Pick an aggregation rule on purpose, and write the tie-break down.&lt;/li&gt;
&lt;li&gt;Measure, on a held-out set, before shipping: every single variant's accuracy, the pairwise error correlation, the agreement on the &lt;strong&gt;wrong&lt;/strong&gt; answers, and accuracy at k = 1, 3, 5.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;If the ensemble is not beating your best single variant, or the correlation is high, ship k = 1 and keep the money. k prompts cost k times as much for a gain you have to measure rather than assume.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;186 verifier asserts over 120 member permutations, 42 in-page checks, 0 failures.&lt;/strong&gt; Vanilla JS, one file, no build step, no network.&lt;/p&gt;

&lt;p&gt;👉 &lt;a href="https://dev48.in/prompt/day83-prompt-ensembling.html" rel="noopener noreferrer"&gt;https://dev48.in/prompt/day83-prompt-ensembling.html&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>prompt</category>
      <category>statistics</category>
    </item>
    <item>
      <title>The Cursor Jumps 28 Pixels or Glides 4: Same Packets, One Delayed Clock</title>
      <dc:creator>Devanshu Biswas</dc:creator>
      <pubDate>Mon, 09 Nov 2026 16:54:19 +0000</pubDate>
      <link>https://dev.to/dev48v/the-cursor-jumps-28-pixels-or-glides-4-same-packets-one-delayed-clock-2g61</link>
      <guid>https://dev.to/dev48v/the-cursor-jumps-28-pixels-or-glides-4-same-packets-one-delayed-clock-2g61</guid>
      <description>&lt;p&gt;A multiplayer cursor looks like a drawing problem. It is a timing problem.&lt;/p&gt;

&lt;p&gt;Everyone else in the file is a stream of sparse positions that arrive late, out of order, and sometimes not at all. The screen redraws about sixty times a second. The positions do not. Almost every frame you paint is therefore something you made up, and the whole craft of the feature is in how you make it up.&lt;/p&gt;

&lt;p&gt;👉 &lt;strong&gt;Live, with every table computed in your browser by the same engine:&lt;/strong&gt; &lt;a href="https://dev48.in/design/day83-presence-cursors.html" rel="noopener noreferrer"&gt;https://dev48.in/design/day83-presence-cursors.html&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  🔴 The wire is simulated
&lt;/h2&gt;

&lt;p&gt;There is no socket here. A seeded transport models a &lt;strong&gt;90 ms latency floor, 100 ms of uniform jitter and a 12% drop rate&lt;/strong&gt;, and the clock is an explicit fixed 16 ms step, so the replay is a pure function of (seed, settings) and reproduces byte for byte. The digest over every rendered position of every frame is &lt;code&gt;31da09e9&lt;/code&gt; on every machine.&lt;/p&gt;

&lt;p&gt;What that costs: the point estimates below are properties of this draw. What transfers is the shape — that jitter wider than the send interval reorders packets at all, that a leading-edge throttle strands the last sample — follows from how clocks and queues work, not from the seed.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;240 frames, 3824 ms, 4 peers, 240 packets sent, 202 delivered, 34 dropped.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Teleporting versus gliding, from identical packets
&lt;/h2&gt;

&lt;p&gt;Assign the newest packet straight to the element and the cursor sits still, then hops. Interpolate between the last two samples and it never stops moving.&lt;/p&gt;

&lt;p&gt;Same replay, same 202 delivered packets. The only difference is whether the frames &lt;em&gt;between&lt;/em&gt; two packets are drawn or skipped:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;drawing strategy&lt;/th&gt;
&lt;th&gt;worst single-frame jump&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;newest packet, assigned to the element&lt;/td&gt;
&lt;td&gt;28.3 px&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;interpolated on a delayed clock&lt;/td&gt;
&lt;td&gt;4.2 px&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;6.8× smaller&lt;/strong&gt;, for free, out of data you already had.&lt;/p&gt;

&lt;p&gt;The eye is very good at spotting things that move in discrete hops, because nothing physical does. A cursor that is smoothly 300 ms behind reads as presence. A cursor that is exactly right in lurches reads as a bug.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the renderer has to run in the past
&lt;/h2&gt;

&lt;p&gt;To interpolate &lt;em&gt;between&lt;/em&gt; two samples you need the second one, and it has not arrived yet. So the renderer draws what the world looked like a moment ago.&lt;/p&gt;

&lt;p&gt;Sweeping that one number over the whole replay:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;render delay&lt;/th&gt;
&lt;th&gt;interpolated&lt;/th&gt;
&lt;th&gt;held (buffer ran dry)&lt;/th&gt;
&lt;th&gt;worst jump&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;0 ms&lt;/td&gt;
&lt;td&gt;0.0%&lt;/td&gt;
&lt;td&gt;100.0%&lt;/td&gt;
&lt;td&gt;28.3 px&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;100 ms&lt;/td&gt;
&lt;td&gt;1.9%&lt;/td&gt;
&lt;td&gt;98.1%&lt;/td&gt;
&lt;td&gt;28.3 px&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;180 ms&lt;/td&gt;
&lt;td&gt;57.0%&lt;/td&gt;
&lt;td&gt;42.1%&lt;/td&gt;
&lt;td&gt;18.3 px&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;220 ms&lt;/td&gt;
&lt;td&gt;80.6%&lt;/td&gt;
&lt;td&gt;17.7%&lt;/td&gt;
&lt;td&gt;14.1 px&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;260 ms&lt;/td&gt;
&lt;td&gt;92.7%&lt;/td&gt;
&lt;td&gt;4.3%&lt;/td&gt;
&lt;td&gt;8.8 px&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;300 ms&lt;/td&gt;
&lt;td&gt;94.7%&lt;/td&gt;
&lt;td&gt;1.3%&lt;/td&gt;
&lt;td&gt;4.2 px&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;At zero delay there is nothing in front of the render time, ever, so every frame holds the last known point — which is the hopping you were trying to remove, dressed up as interpolation. The delay has to clear the latency floor &lt;em&gt;plus&lt;/em&gt; the jitter &lt;em&gt;plus&lt;/em&gt; one send interval before any of this works at all. Below 180 ms, almost none of it does.&lt;/p&gt;

&lt;h2&gt;
  
  
  Throttling is coalescing, not sampling
&lt;/h2&gt;

&lt;p&gt;A pointer fires once a frame. Sixty-plus messages a second per person, through a socket that also carries the edits.&lt;/p&gt;

&lt;p&gt;The slot matters more than the interval: &lt;code&gt;push&lt;/code&gt; never sends, it overwrites a single pending position, so what finally goes out is where the pointer &lt;em&gt;is&lt;/em&gt;, not where it happened to be at a tick boundary.&lt;/p&gt;

&lt;p&gt;63 pointer events on a 16 ms clock, one second:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;interval&lt;/th&gt;
&lt;th&gt;messages sent&lt;/th&gt;
&lt;th&gt;coalesced&lt;/th&gt;
&lt;th&gt;traffic cut&lt;/th&gt;
&lt;th&gt;min gap&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;0 ms&lt;/td&gt;
&lt;td&gt;63&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0%&lt;/td&gt;
&lt;td&gt;16 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;33 ms&lt;/td&gt;
&lt;td&gt;21&lt;/td&gt;
&lt;td&gt;41&lt;/td&gt;
&lt;td&gt;67%&lt;/td&gt;
&lt;td&gt;48 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;50 ms&lt;/td&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;td&gt;46&lt;/td&gt;
&lt;td&gt;75%&lt;/td&gt;
&lt;td&gt;64 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;100 ms&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;53&lt;/td&gt;
&lt;td&gt;86%&lt;/td&gt;
&lt;td&gt;112 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;200 ms&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;57&lt;/td&gt;
&lt;td&gt;92%&lt;/td&gt;
&lt;td&gt;208 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;63 events become exactly 16 messages.&lt;/strong&gt; Nothing visible changes, because the interpolator fills the gaps.&lt;/p&gt;

&lt;h2&gt;
  
  
  The trailing edge is the half everyone forgets
&lt;/h2&gt;

&lt;p&gt;A leading-edge throttle fires at once and then ignores everything until the interval is up. Stop the pointer inside that window — which is what happens every single time somebody finishes a gesture and parks on a word — and the final position is sitting in the pending slot when the events stop.&lt;/p&gt;

&lt;p&gt;Nothing ever sends it.&lt;/p&gt;

&lt;p&gt;A pointer moving &lt;code&gt;0 → 60 → 130 → 190 → 220&lt;/code&gt; and stopping at t=48 ms, with a 50 ms throttle:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;throttle&lt;/th&gt;
&lt;th&gt;last x reported&lt;/th&gt;
&lt;th&gt;truth&lt;/th&gt;
&lt;th&gt;error&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;leading edge only&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;220&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;220 px, forever&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;leading + trailing&lt;/td&gt;
&lt;td&gt;220&lt;/td&gt;
&lt;td&gt;220&lt;/td&gt;
&lt;td&gt;0 px&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The trailing emission landed at t=60 ms, &lt;strong&gt;12 ms after the pointer stopped&lt;/strong&gt;. One timer, and the remote cursor lands where the person actually put it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Jitter reorders, so the receiver distrusts arrival order
&lt;/h2&gt;

&lt;p&gt;With a 50 ms throttle the peers send every 64 ms, and the jitter is 100 ms wide. That is enough for two messages to swap places in flight: &lt;strong&gt;103 packets in this replay were scheduled to land before one sent earlier.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;If you append in arrival order, one of those walks the cursor backwards and then forwards again — a visible stutter from a packet that was otherwise harmless. So every message carries the sender's timestamp and the buffer refuses anything not strictly newer. &lt;strong&gt;14 of 202 delivered samples were rejected; 188 made it into a timeline.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Dropped packets need no handling beyond that. A gap is just a longer segment to interpolate across, which is why a lossy cursor channel still looks fine — until it doesn't:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;throttle @ loss&lt;/th&gt;
&lt;th&gt;delivered&lt;/th&gt;
&lt;th&gt;frames held&lt;/th&gt;
&lt;th&gt;worst jump&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;16 ms @ 12%&lt;/td&gt;
&lt;td&gt;808&lt;/td&gt;
&lt;td&gt;0.0%&lt;/td&gt;
&lt;td&gt;9.6 px&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;50 ms @ 12%&lt;/td&gt;
&lt;td&gt;202&lt;/td&gt;
&lt;td&gt;1.3%&lt;/td&gt;
&lt;td&gt;4.2 px&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;50 ms @ 35%&lt;/td&gt;
&lt;td&gt;143&lt;/td&gt;
&lt;td&gt;9.5%&lt;/td&gt;
&lt;td&gt;20.3 px&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;200 ms @ 12%&lt;/td&gt;
&lt;td&gt;60&lt;/td&gt;
&lt;td&gt;33.0%&lt;/td&gt;
&lt;td&gt;64.8 px&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  🔴 Presence is a timeout, never a leave event
&lt;/h2&gt;

&lt;p&gt;The obvious design: send &lt;code&gt;join&lt;/code&gt; on open, &lt;code&gt;leave&lt;/code&gt; on close, broadcast both.&lt;/p&gt;

&lt;p&gt;Nobody sends &lt;code&gt;leave&lt;/code&gt; when the lid shuts, the tunnel drops, or the tab is killed. The ghost cursor still sitting in the document an hour after someone left the meeting is always this bug.&lt;/p&gt;

&lt;p&gt;So a peer exists while packets arrive, and is removed when none has for a while. One peer here goes silent at &lt;strong&gt;t=1200 ms&lt;/strong&gt; with no goodbye of any kind, and is evicted at &lt;strong&gt;t=2800 ms&lt;/strong&gt; — one 1500 ms timeout after its last packet. The other three never flicker. There is no &lt;code&gt;leave&lt;/code&gt; message in the protocol at all; the store throws on any message that is not a cursor.&lt;/p&gt;

&lt;h2&gt;
  
  
  Colour is identity, so it cannot come from a counter
&lt;/h2&gt;

&lt;p&gt;Palette plus join counter: first to connect is amber, second is rose. It survives until somebody's connection blips — they drop, the timeout removes them, they reconnect two seconds later, and now they are a different colour. In this five-person room, one reconnect recolours &lt;strong&gt;four of the five&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Worse, eight swatches is not enough to be safe anyway. Five people drawing from eight slots collide &lt;strong&gt;four times out of five&lt;/strong&gt;, in the one feature whose entire job is telling people apart.&lt;/p&gt;

&lt;p&gt;So the hue comes off a hash of the stable id — FNV-1a with a murmur3 avalanche, because raw FNV-1a puts &lt;code&gt;u-1&lt;/code&gt; and &lt;code&gt;u-2&lt;/code&gt; in neighbouring buckets. 4000 sequential ids land across all twelve 30° zones of the wheel with no zone above 1.4× its share, and the five peers in the demo sit &lt;strong&gt;at least 49° apart&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Name tags flip per axis
&lt;/h2&gt;

&lt;p&gt;The tag hangs down and to the right of the hotspot, which is wrong at every edge. Clipping is the worst option — a half-visible name is unreadable and still takes the space.&lt;/p&gt;

&lt;p&gt;Compute the preferred box, test each axis against the viewport, flip that axis if it does not fit, clamp only when the tag cannot fit on either side. Checked against a search model that enumerates all four corner candidates, at &lt;strong&gt;71,736 anchors including positions outside the viewport: zero escapes, zero disagreements&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  What is checked
&lt;/h2&gt;

&lt;p&gt;The page's engine is lifted out of the HTML and run in Node against a reference model written from the rules rather than the code — a brute-force interpolator that scans the whole buffer, a throttle expressed as a predicate over the tick times, a label placer that searches candidates instead of flipping, an independent hash.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;164 verifier asserts. 49 in-page checks. 0 failures.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The replay is the point. Every number above moved while the settings were being chosen, and none of them moved by accident.&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>javascript</category>
      <category>ux</category>
      <category>frontend</category>
    </item>
    <item>
      <title>The Nine Men's Morris Capture Rule Almost Everyone Gets Wrong</title>
      <dc:creator>Devanshu Biswas</dc:creator>
      <pubDate>Mon, 09 Nov 2026 16:53:38 +0000</pubDate>
      <link>https://dev.to/dev48v/the-nine-mens-morris-capture-rule-almost-everyone-gets-wrong-1f2g</link>
      <guid>https://dev.to/dev48v/the-nine-mens-morris-capture-rule-almost-everyone-gets-wrong-1f2g</guid>
      <description>&lt;p&gt;A piece standing in a mill cannot be taken. Unless every enemy piece is standing in a mill, in which case any of them can.&lt;/p&gt;

&lt;p&gt;That second sentence is the one people skip, and skipping it does not produce a subtle bug. On a position where White owes a removal and all six Black pieces sit inside mills, the take-list goes from &lt;strong&gt;6&lt;/strong&gt; to &lt;strong&gt;0&lt;/strong&gt;. Zero legal moves means the mover is blocked, and a blocked player loses. So the engine hands the win to the side that was being ground down.&lt;/p&gt;

&lt;p&gt;👉 &lt;strong&gt;Playable, with a selectable-depth alpha-beta opponent:&lt;/strong&gt; &lt;a href="https://dev48.in/game/day83-nine-mens-morris.html" rel="noopener noreferrer"&gt;https://dev48.in/game/day83-nine-mens-morris.html&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The board is not a grid, and storing it like one costs you the rest of the build
&lt;/h2&gt;

&lt;p&gt;Twenty-four points on three nested squares. The corner of a square has &lt;strong&gt;2&lt;/strong&gt; neighbours, the midpoint of the outer or inner square has &lt;strong&gt;3&lt;/strong&gt;, and the midpoint of the middle square has &lt;strong&gt;4&lt;/strong&gt;. Across the board that splits as &lt;strong&gt;12 / 8 / 4&lt;/strong&gt; points, degrees summing to &lt;strong&gt;64&lt;/strong&gt;, which is twice the &lt;strong&gt;32&lt;/strong&gt; edges.&lt;/p&gt;

&lt;p&gt;A 7×7 array can hold that picture, but 25 of its 49 cells are holes and every rule then needs a special case for them.&lt;/p&gt;

&lt;p&gt;So the board is a graph, and the only thing typed by hand is the sixteen lines of three:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;MILLS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
  &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;              &lt;span class="c1"&gt;// outer ring&lt;/span&gt;
  &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;9&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;11&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;12&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;12&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;13&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;14&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;14&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;15&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;     &lt;span class="c1"&gt;// middle ring&lt;/span&gt;
  &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;16&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;17&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;18&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;18&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;19&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;21&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;22&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;22&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;23&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;16&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;  &lt;span class="c1"&gt;// inner ring&lt;/span&gt;
  &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;9&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;17&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;11&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;19&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;13&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;21&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;15&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="mi"&gt;23&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;        &lt;span class="c1"&gt;// the four spokes&lt;/span&gt;
&lt;span class="p"&gt;];&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Adjacency is computed from them: each line contributes exactly two pairs, 16 × 2 = 32, and after de-duplication there are still &lt;strong&gt;32&lt;/strong&gt;. None of the pairs collide, which is itself a check on the list.&lt;/p&gt;

&lt;p&gt;Every point lies on exactly &lt;strong&gt;2&lt;/strong&gt; lines (24 × 2 = 48 = 16 × 3), so a mill test after a move reads two lines instead of sixteen.&lt;/p&gt;

&lt;h2&gt;
  
  
  A mill does not end your turn
&lt;/h2&gt;

&lt;p&gt;This is where a first implementation breaks. Closing a mill obliges you to remove an enemy piece, and only then does the opponent move. Model a turn as "one move, then swap sides" and the removal has nowhere to live, so it usually ends up as a prompt outside the engine — at which point the search cannot see captures at all.&lt;/p&gt;

&lt;p&gt;The fix is a flag, and a move generator that answers it first:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;millAt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;n&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;b&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;mv&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;to&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;p&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="nx"&gt;n&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;pending&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;   &lt;span class="c1"&gt;// same player moves again&lt;/span&gt;
&lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="nx"&gt;n&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;turn&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;opp&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;p&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="c1"&gt;// ...and the generator:&lt;/span&gt;
&lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;s&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;pending&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;removable&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;s&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nf"&gt;opp&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;p&lt;/span&gt;&lt;span class="p"&gt;)).&lt;/span&gt;&lt;span class="nf"&gt;map&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;at&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;t&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;r&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;at&lt;/span&gt; &lt;span class="p"&gt;}));&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The capture exception is then three lines, a filter with a fallback:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;removable&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;s&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;victim&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;all&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[],&lt;/span&gt; &lt;span class="nx"&gt;free&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[];&lt;/span&gt;
  &lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;24&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;s&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;b&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;i&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="nx"&gt;victim&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;continue&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nx"&gt;all&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;i&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nf"&gt;millAt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;s&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;b&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;victim&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="nx"&gt;free&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;i&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;free&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="p"&gt;?&lt;/span&gt; &lt;span class="nx"&gt;free&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;all&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;   &lt;span class="c1"&gt;// all milled -&amp;gt; protection lapses&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Measured on two positions that differ by one Black piece: with a loose piece on the board the list is &lt;strong&gt;2&lt;/strong&gt; long and the milled pieces are protected. With every Black piece milled it is &lt;strong&gt;6&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Flying is where the branching factor explodes
&lt;/h2&gt;

&lt;p&gt;Phase is never stored. It is a question asked at the top of the generator: pieces in hand means placing, exactly three pieces means flying, otherwise slide.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;position&lt;/th&gt;
&lt;th&gt;phase&lt;/th&gt;
&lt;th&gt;White's legal moves&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;opening&lt;/td&gt;
&lt;td&gt;placing&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;24&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;mid-game, six pieces&lt;/td&gt;
&lt;td&gt;moving&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;16&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;White down to three&lt;/td&gt;
&lt;td&gt;flying&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;54&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Three flying pieces times 18 empty points is 54. The search slows down exactly when the position gets sharp.&lt;/p&gt;

&lt;h2&gt;
  
  
  Without draw rules a deterministic engine never stops
&lt;/h2&gt;

&lt;p&gt;Two sides that cannot get through each other shuffle forever. Driving a four-ply shuffle (White 0↔1, Black 16↔17) through the engine with history tracked, &lt;strong&gt;threefold repetition fires at ply 9&lt;/strong&gt;. The same shuffle with history off runs to the fifty-move limit and is drawn at &lt;strong&gt;exactly ply 100&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Both are needed. Repetition catches the shuffle; the fifty-move counter catches a side that keeps finding new positions and no progress. The counter resets on anything irreversible, which here means a capture or a placement.&lt;/p&gt;

&lt;h2&gt;
  
  
  The search has to ask whose turn it is
&lt;/h2&gt;

&lt;p&gt;Plain negamax negates each level, which bakes in the assumption that sides alternate. In this game a mill gives the same player another ply, so that assumption scores captures for the wrong side. Asking &lt;code&gt;s.turn === me&lt;/code&gt; at every node costs nothing and makes pending removals just another node that happens to maximise again.&lt;/p&gt;

&lt;p&gt;Alpha-beta on top of that, same position, measured both ways:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;depth&lt;/th&gt;
&lt;th&gt;plain minimax&lt;/th&gt;
&lt;th&gt;with alpha-beta&lt;/th&gt;
&lt;th&gt;saving&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;22&lt;/td&gt;
&lt;td&gt;22&lt;/td&gt;
&lt;td&gt;1.0x&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;244&lt;/td&gt;
&lt;td&gt;37&lt;/td&gt;
&lt;td&gt;6.6x&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;4,201&lt;/td&gt;
&lt;td&gt;682&lt;/td&gt;
&lt;td&gt;6.2x&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;39,264&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;1,236&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;31.8x&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Same score and the same chosen move at every depth. Pruning changed the cost, never the answer.&lt;/p&gt;

&lt;p&gt;On a position where White can slide 3→2, close 0-1-2 and take Black from three pieces to two, the depth-3 search finds that single move in &lt;strong&gt;25 nodes&lt;/strong&gt; and scores it &lt;strong&gt;999,998&lt;/strong&gt; — a win has to outrank every static evaluation, so it is scored as &lt;code&gt;WIN - ply&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Twenty-four opening moves are really four
&lt;/h2&gt;

&lt;p&gt;The board's symmetries are four rotations, four reflections, and the swap of the inner and outer rings, which is a symmetry of the graph even though it is not a symmetry of the drawing. The page finds them by searching for every relabelling of the points that preserves adjacency, rather than assuming a list, and gets &lt;strong&gt;16&lt;/strong&gt;. They split the points into &lt;strong&gt;4&lt;/strong&gt; orbits:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;orbit&lt;/th&gt;
&lt;th&gt;points&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;corners of the outer and inner rings&lt;/td&gt;
&lt;td&gt;8&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;midpoints of the outer and inner rings&lt;/td&gt;
&lt;td&gt;8&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;corners of the middle ring&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;midpoints of the middle ring&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;So the opening tree is four times smaller than it looks.&lt;/p&gt;

&lt;h2&gt;
  
  
  What was actually checked
&lt;/h2&gt;

&lt;p&gt;The page runs &lt;strong&gt;38&lt;/strong&gt; assertions in the browser against the same engine the game is played with, and the badge reads 38 of 38.&lt;/p&gt;

&lt;p&gt;A separate verifier builds the board a second time from geometry instead of from the list of lines: two points touch when they share a row or column with nothing between them and the segment does not run through the empty centre. That derivation independently produces &lt;strong&gt;32&lt;/strong&gt; edges and &lt;strong&gt;16&lt;/strong&gt; lines, and both match the page exactly.&lt;/p&gt;

&lt;p&gt;It then re-implements the rules over a board of characters and drives both engines side by side through &lt;strong&gt;12&lt;/strong&gt; seeded games, &lt;strong&gt;1,743&lt;/strong&gt; plies, comparing the full legal-move set and the resulting position at every single ply. &lt;strong&gt;65&lt;/strong&gt; asserts, 0 failures. Endings: 8 by reduction to two pieces, 3 by the fifty-move rule, 1 by repetition.&lt;/p&gt;

&lt;p&gt;The engine playing itself at depth 1 finishes in &lt;strong&gt;45&lt;/strong&gt; plies.&lt;/p&gt;

</description>
      <category>algorithms</category>
      <category>javascript</category>
      <category>programming</category>
      <category>gamedev</category>
    </item>
    <item>
      <title>The 4-Characters-Per-Token Rule Under-Counts Real Code by 27%</title>
      <dc:creator>Devanshu Biswas</dc:creator>
      <pubDate>Tue, 03 Nov 2026 08:26:06 +0000</pubDate>
      <link>https://dev.to/dev48v/the-4-characters-per-token-rule-under-counts-real-code-by-27-no6</link>
      <guid>https://dev.to/dev48v/the-4-characters-per-token-rule-under-counts-real-code-by-27-no6</guid>
      <description>&lt;p&gt;Counting tokens is the easy part. Every library does it.&lt;/p&gt;

&lt;p&gt;The hard part is what happens &lt;strong&gt;after&lt;/strong&gt; you are over budget. You have to drop something — and most tools drop silently. They truncate the tail of the prompt, or they keep adding files until they hit the limit and stop. Both leave you with an agent that answered from a prompt you never saw.&lt;/p&gt;

&lt;p&gt;So I built &lt;strong&gt;roomy&lt;/strong&gt;: it packs under an explicit budget and returns, for every candidate, either the reason it is in or the reason it is out.&lt;/p&gt;

&lt;p&gt;👉 &lt;strong&gt;Repo (MIT):&lt;/strong&gt; &lt;a href="https://github.com/dev48v/roomy" rel="noopener noreferrer"&gt;https://github.com/dev48v/roomy&lt;/a&gt;&lt;br&gt;
👉 &lt;strong&gt;The write-up, live:&lt;/strong&gt; &lt;a href="https://dev48.in/agentlab/vol5-02-roomy.html" rel="noopener noreferrer"&gt;https://dev48.in/agentlab/vol5-02-roomy.html&lt;/a&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ python roomy.py ../mcpfromzero --budget 6000 --reserve 1000
budget 6000  reserve 1000  used 4820  headroom 180
------------------------------------------------------------------------
IN  README.md                                   2871  fits (priority 2)
IN  docs/NOTICE.md                               412  fits (priority 1)
IN  python/mcpfromzero/__main__.py                27  fits (priority 0)
...
out conformance/run.py                          3694  no room left: 3694 needed, 180 free
out typescript/src/tools.ts                     4371  no room left: 4371 needed, 180 free
out python/mcpfromzero/tools.py                 4666  no room left: 4666 needed, 180 free
------------------------------------------------------------------------
9 in, 14 out
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  The rule of thumb is wrong in the dangerous direction
&lt;/h2&gt;

&lt;p&gt;Four characters per token is an average over &lt;strong&gt;English prose&lt;/strong&gt;. Agent prompts are made of code, where identifiers split, punctuation is its own token, and every newline costs.&lt;/p&gt;

&lt;p&gt;Measured over one real repository:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;measure&lt;/th&gt;
&lt;th&gt;tokens&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;characters&lt;/td&gt;
&lt;td&gt;206,658&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;the 4-chars-per-token rule&lt;/td&gt;
&lt;td&gt;51,664&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;roomy's estimate&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;65,859&lt;/strong&gt; (+27%)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;An over-count wastes a little budget. An &lt;strong&gt;under-count overflows the window&lt;/strong&gt;, and that failure arrives as a truncated reply rather than as an error. roomy is deliberately biased toward the first.&lt;/p&gt;

&lt;p&gt;It ships &lt;strong&gt;no tokenizer&lt;/strong&gt;, on purpose — vendoring one ties the tool to a single model family and is wrong for every other.&lt;/p&gt;

&lt;h2&gt;
  
  
  The reserve is not optional
&lt;/h2&gt;

&lt;p&gt;The window is shared with the output. A prompt that "fits" in 6,000 tokens does not fit if the model then needs 1,000 to answer:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;reserved for the reply&lt;/th&gt;
&lt;th&gt;usable&lt;/th&gt;
&lt;th&gt;files in&lt;/th&gt;
&lt;th&gt;tokens used&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;6000&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;5917&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1000&lt;/td&gt;
&lt;td&gt;5000&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;4820&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2000&lt;/td&gt;
&lt;td&gt;4000&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;3959&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4000&lt;/td&gt;
&lt;td&gt;2000&lt;/td&gt;
&lt;td&gt;7&lt;/td&gt;
&lt;td&gt;1949&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  A required item that does not fit raises
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;plan&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;pack&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;candidates&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;budget&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;8000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;reserve&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;   &lt;span class="c1"&gt;# raises WontFit
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It does &lt;strong&gt;not&lt;/strong&gt; return a plan with the thing missing. A caller who ignores that exception has built a prompt without something they declared they could not work without — and that failure comes back as a confident, wrong answer rather than a crash.&lt;/p&gt;

&lt;h2&gt;
  
  
  Smallest-first is a heuristic, and the gap is measured
&lt;/h2&gt;

&lt;p&gt;Packing a budget optimally is knapsack. roomy sorts by priority, then size, then name. Rather than claiming that is fine, the page measures it against an exhaustive &lt;code&gt;2^16&lt;/code&gt; search:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;budget&lt;/th&gt;
&lt;th&gt;greedy&lt;/th&gt;
&lt;th&gt;optimal&lt;/th&gt;
&lt;th&gt;greedy / optimal&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1500&lt;/td&gt;
&lt;td&gt;1403&lt;/td&gt;
&lt;td&gt;1498&lt;/td&gt;
&lt;td&gt;93.7%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3000&lt;/td&gt;
&lt;td&gt;2721&lt;/td&gt;
&lt;td&gt;3000&lt;/td&gt;
&lt;td&gt;90.7%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;6000&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;4966&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;6000&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;82.8%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12000&lt;/td&gt;
&lt;td&gt;11809&lt;/td&gt;
&lt;td&gt;12000&lt;/td&gt;
&lt;td&gt;98.4%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;24000&lt;/td&gt;
&lt;td&gt;14381&lt;/td&gt;
&lt;td&gt;14381&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;It gives up real window. That is the trade: a packing you can &lt;strong&gt;explain line by line&lt;/strong&gt; and reproduce exactly is worth more here than a few percent, because the output is read by a human deciding what to include — not by a bin.&lt;/p&gt;

&lt;p&gt;The sort ends on the &lt;strong&gt;name&lt;/strong&gt; so the result never depends on the order the caller happened to build the list in.&lt;/p&gt;

&lt;h2&gt;
  
  
  Tested cross-language
&lt;/h2&gt;

&lt;p&gt;The write-up ships a JavaScript port of the packer. A port checked only against itself is not checked, so the verifier runs the &lt;strong&gt;Python&lt;/strong&gt; package as a subprocess and demands the two agree, decision for decision, on the same inputs — 38 randomised cases plus deliberate edges (empty lists, duplicate names, a reserve that swallows the budget, a zero-token candidate).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;36 unit tests. 909 verifier asserts including the cross-language comparison. 0 failures.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The four invariants worth guarding, and the only ones the tests really care about: the budget is never exceeded, every candidate gets a decision, the result does not depend on input order, and a required item that does not fit raises rather than vanishing.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>python</category>
      <category>opensource</category>
    </item>
    <item>
      <title>Synonym Expansion Cannot Lower Recall. That Is Exactly Why Recall Is the Wrong Thing to Tune It On.</title>
      <dc:creator>Devanshu Biswas</dc:creator>
      <pubDate>Tue, 03 Nov 2026 08:25:24 +0000</pubDate>
      <link>https://dev.to/dev48v/synonym-expansion-cannot-lower-recall-that-is-exactly-why-recall-is-the-wrong-thing-to-tune-it-on-5dba</link>
      <guid>https://dev.to/dev48v/synonym-expansion-cannot-lower-recall-that-is-exactly-why-recall-is-the-wrong-thing-to-tune-it-on-5dba</guid>
      <description>&lt;p&gt;Query rewriting is the first stage of most RAG pipelines and the one that gets tuned by vibes. So I built a real BM25 index over a real corpus, applied real rewrites, and scored them against hand-labelled relevance.&lt;/p&gt;

&lt;p&gt;👉 &lt;strong&gt;Live, runs in your browser:&lt;/strong&gt; &lt;a href="https://dev48.in/ai/days/day82-query-rewriting.html" rel="noopener noreferrer"&gt;https://dev48.in/ai/days/day82-query-rewriting.html&lt;/a&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;rewrite&lt;/th&gt;
&lt;th&gt;recall@3&lt;/th&gt;
&lt;th&gt;precision@3&lt;/th&gt;
&lt;th&gt;MRR&lt;/th&gt;
&lt;th&gt;mean query length&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;verbatim (keeps stopwords)&lt;/td&gt;
&lt;td&gt;0.688&lt;/td&gt;
&lt;td&gt;0.375&lt;/td&gt;
&lt;td&gt;0.938&lt;/td&gt;
&lt;td&gt;3.4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;raw (drops stopwords)&lt;/td&gt;
&lt;td&gt;0.688&lt;/td&gt;
&lt;td&gt;0.375&lt;/td&gt;
&lt;td&gt;0.938&lt;/td&gt;
&lt;td&gt;2.9&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;strong&gt;expand&lt;/strong&gt; (synonyms)&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.938&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.542&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;0.938&lt;/td&gt;
&lt;td&gt;5.8&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;expandNoisy (wrong synonyms)&lt;/td&gt;
&lt;td&gt;0.688&lt;/td&gt;
&lt;td&gt;0.375&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.813&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;5.8&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;strong&gt;hyde&lt;/strong&gt; (pseudo-answer)&lt;/td&gt;
&lt;td&gt;0.563&lt;/td&gt;
&lt;td&gt;0.292&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.323&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;16.9&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;decompose&lt;/td&gt;
&lt;td&gt;0.688&lt;/td&gt;
&lt;td&gt;0.375&lt;/td&gt;
&lt;td&gt;0.938&lt;/td&gt;
&lt;td&gt;2.9&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  The claim I had to withdraw
&lt;/h2&gt;

&lt;p&gt;I wrote this assertion and it &lt;strong&gt;failed&lt;/strong&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nf"&gt;t&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;INVERSION: expand raises recall but LOWERS precision&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;exp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;recall&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="nx"&gt;base&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;recall&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nx"&gt;exp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;precision&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="nx"&gt;base&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;precision&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Expansion raised &lt;strong&gt;both&lt;/strong&gt; — recall 0.688 → 0.938 &lt;em&gt;and&lt;/em&gt; precision 0.375 → 0.542.&lt;/p&gt;

&lt;p&gt;That is not a flaw in the measurement. It is what a genuine vocabulary mismatch looks like: the documents say &lt;em&gt;automobile&lt;/em&gt; and &lt;em&gt;vehicle&lt;/em&gt; where the queries say &lt;em&gt;car&lt;/em&gt;, so the added terms are not noise, they are the &lt;strong&gt;missing words&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The precision/recall trade-off everyone quotes is a property of &lt;strong&gt;synonym quality&lt;/strong&gt;, not of expansion. The control proves it — the same mechanism, same code path, &lt;strong&gt;identical query length&lt;/strong&gt; (5.8 tokens either way), but with plausible-and-wrong synonyms:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;precision 0.375 → 0.375, &lt;strong&gt;MRR 0.938 → 0.813&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Interesting detail: precision@3 does not move. The wrong synonyms are rare terms, so they pull in few documents at a cut-off of three. The damage shows up in &lt;strong&gt;where the first relevant hit lands&lt;/strong&gt; — which is the metric a RAG pipeline actually lives on, because the generator only ever sees a truncated list.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why recall is a trap here
&lt;/h2&gt;

&lt;p&gt;Look at the definition. &lt;code&gt;expand&lt;/code&gt; only &lt;strong&gt;adds&lt;/strong&gt; terms, never removes them:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nx"&gt;expand&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;q&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;base&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;tokens&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;q&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="nx"&gt;out&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;base&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;slice&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
  &lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;w&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;base&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;s&lt;/span&gt; &lt;span class="k"&gt;of &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;SYNONYMS&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;w&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="p"&gt;[]))&lt;/span&gt; &lt;span class="nx"&gt;out&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;s&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;out&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A superset of the query can only retrieve a superset of the documents. &lt;strong&gt;Recall cannot fall.&lt;/strong&gt; So "our rewrite improved recall" is close to a tautology for any additive rewrite, and tuning on it will approve every expansion you ever write, including the bad ones.&lt;/p&gt;

&lt;h2&gt;
  
  
  HyDE lost to doing nothing
&lt;/h2&gt;

&lt;p&gt;MRR &lt;strong&gt;0.938 → 0.323&lt;/strong&gt;. Per-query, the pseudo-answer version is worse on five of eight.&lt;/p&gt;

&lt;p&gt;The mechanism is mundane. A HyDE pseudo-answer is query terms plus answer-shaped filler — &lt;em&gt;this, document, explains, typically, generally, process, step, first, then, finally&lt;/em&gt;. Those are &lt;strong&gt;real terms in the corpus&lt;/strong&gt;, and two of my sixteen documents are majority filler:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"The process you should generally follow to file a complaint"&lt;br&gt;
"This document explains what you should typically do first"&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;A pseudo-answer retrieves those because it &lt;strong&gt;looks like them&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The control is the important half: strip the filler and keep everything else, and HyDE's MRR is &lt;em&gt;identical&lt;/em&gt; to the raw query, to machine precision. It is the filler, not the method. A corpus of prose with no boilerplate documents would behave differently — which is the real lesson, since you cannot know that without measuring your own corpus.&lt;/p&gt;

&lt;h2&gt;
  
  
  A rewrite reweights, it does not just add
&lt;/h2&gt;

&lt;p&gt;BM25 scores a term by how rare it is:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;term&lt;/th&gt;
&lt;th&gt;documents containing it&lt;/th&gt;
&lt;th&gt;IDF&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;alternator&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;2.0794&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;automobile&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;1.7047&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;car&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;3.4340&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;you&lt;/td&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;1.0986&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Adding one common word contributes almost nothing. Adding &lt;strong&gt;many&lt;/strong&gt; common words — which is what a pseudo-answer is made of — pulls generic documents to the top.&lt;/p&gt;

&lt;h2&gt;
  
  
  Verification
&lt;/h2&gt;

&lt;p&gt;The page scores from an inverted index, accumulating term by term. The verifier builds a &lt;strong&gt;dense&lt;/strong&gt; term-document matrix and computes each document's score from the BM25 formula directly — the transpose of the page's loop, written from the formula rather than from its code. The relevance labels are retyped, so a label typo would surface as a disagreement instead of being copied along.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;263 asserts, 0 failures.&lt;/strong&gt; 22 in-page checks.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>rag</category>
      <category>search</category>
      <category>llm</category>
    </item>
    <item>
      <title>A Collapsed Encoder Scores a Perfect 1.0 on Alignment</title>
      <dc:creator>Devanshu Biswas</dc:creator>
      <pubDate>Tue, 03 Nov 2026 08:24:44 +0000</pubDate>
      <link>https://dev.to/dev48v/a-collapsed-encoder-scores-a-perfect-10-on-alignment-2a3k</link>
      <guid>https://dev.to/dev48v/a-collapsed-encoder-scores-a-perfect-10-on-alignment-2a3k</guid>
      <description>&lt;p&gt;NT-Xent — InfoNCE, the loss behind SimCLR and most embedding models — asks each view of an item to pick its partner out of a line-up. The line-up is &lt;strong&gt;the rest of the batch&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;👉 &lt;strong&gt;Live, every number measured in your browser:&lt;/strong&gt; &lt;a href="https://dev48.in/dl/day82-contrastive-loss.html" rel="noopener noreferrer"&gt;https://dev48.in/dl/day82-contrastive-loss.html&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That clause is not a footnote. It means the batch size changes the &lt;strong&gt;objective&lt;/strong&gt;, not just the gradient noise.&lt;/p&gt;

&lt;h2&gt;
  
  
  The loss number is not comparable across batch sizes
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;batch&lt;/th&gt;
&lt;th&gt;negatives per row&lt;/th&gt;
&lt;th&gt;loss&lt;/th&gt;
&lt;th&gt;collapse floor log(2N−1)&lt;/th&gt;
&lt;th&gt;partner ranked first&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;N = 2&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;0.0084&lt;/td&gt;
&lt;td&gt;1.0986&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;N = 8&lt;/td&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;td&gt;0.0087&lt;/td&gt;
&lt;td&gt;2.7081&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;N = 32&lt;/td&gt;
&lt;td&gt;64&lt;/td&gt;
&lt;td&gt;0.0379&lt;/td&gt;
&lt;td&gt;4.1431&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;N = 64&lt;/td&gt;
&lt;td&gt;128&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.0817&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;4.8442&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Same encoder, same temperature, same data distribution. The loss climbs anyway, because &lt;strong&gt;the task got harder&lt;/strong&gt;. A loss curve from a run at N=256 cannot be read against one at N=64, and "our loss is lower" across a batch-size change means nothing at all.&lt;/p&gt;

&lt;h2&gt;
  
  
  Collapse maximises the half of the objective people plot
&lt;/h2&gt;

&lt;p&gt;Alignment — the average cosine between the two views of an item — is the metric that gets logged. Here is what it says about an encoder that has learned nothing:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;batch&lt;/th&gt;
&lt;th&gt;alignment&lt;/th&gt;
&lt;th&gt;uniformity&lt;/th&gt;
&lt;th&gt;loss&lt;/th&gt;
&lt;th&gt;partner first&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;well separated&lt;/td&gt;
&lt;td&gt;0.9792&lt;/td&gt;
&lt;td&gt;−0.0229&lt;/td&gt;
&lt;td&gt;0.0014&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;noisy views&lt;/td&gt;
&lt;td&gt;0.6140&lt;/td&gt;
&lt;td&gt;−0.0268&lt;/td&gt;
&lt;td&gt;0.1185&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;views unrelated&lt;/td&gt;
&lt;td&gt;0.1017&lt;/td&gt;
&lt;td&gt;−0.0208&lt;/td&gt;
&lt;td&gt;2.9476&lt;/td&gt;
&lt;td&gt;18.8%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;COLLAPSED&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;1.0000&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;1.0000&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;2.7081&lt;/td&gt;
&lt;td&gt;12.5%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Every embedding is the same point. Alignment is a &lt;strong&gt;perfect 1.0000&lt;/strong&gt;. Only uniformity notices anything is wrong.&lt;/p&gt;

&lt;p&gt;And the collapsed loss is exactly &lt;code&gt;log(2N−1)&lt;/code&gt; — confirmed at five batch sizes and four temperatures. All the logits are equal, so the softmax is uniform over 2N−1 options and &lt;strong&gt;temperature cancels out entirely&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Temperature is a negative-mining knob
&lt;/h2&gt;

&lt;p&gt;Everyone describes τ as sharpening the distribution. True, but the useful reading is about &lt;strong&gt;where the gradient goes&lt;/strong&gt;:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;temperature&lt;/th&gt;
&lt;th&gt;loss&lt;/th&gt;
&lt;th&gt;share of negative gradient on the HARDEST negative&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1.00&lt;/td&gt;
&lt;td&gt;2.6041&lt;/td&gt;
&lt;td&gt;5.0%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.50&lt;/td&gt;
&lt;td&gt;1.8567&lt;/td&gt;
&lt;td&gt;7.3%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.20&lt;/td&gt;
&lt;td&gt;0.4257&lt;/td&gt;
&lt;td&gt;16.8%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.10&lt;/td&gt;
&lt;td&gt;0.0191&lt;/td&gt;
&lt;td&gt;34.1%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.05&lt;/td&gt;
&lt;td&gt;0.0001&lt;/td&gt;
&lt;td&gt;49.4%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.01&lt;/td&gt;
&lt;td&gt;0.0000&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;63.1%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;An independent check with different embeddings measured &lt;strong&gt;98.8% at τ=0.02&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;As τ falls, the share landing on a single negative climbs toward 100% and every other negative stops contributing. At that point it is no longer contrastive learning over a batch — it is &lt;strong&gt;hardest-negative mining with extra steps&lt;/strong&gt;, and all the effort you put into a large batch is being discarded.&lt;/p&gt;

&lt;h2&gt;
  
  
  The numerics are not optional
&lt;/h2&gt;

&lt;p&gt;At τ = 0.01, a cosine of 1 becomes a logit of 100. At τ = 0.001 it becomes 1000, and:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;exp&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;   &lt;span class="c1"&gt;// Infinity&lt;/span&gt;
&lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kc"&gt;Infinity&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="c1"&gt;// Infinity  -&amp;gt; loss becomes NaN&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;NaN is not a large loss. It is no loss at all&lt;/strong&gt; — the gradient is gone and training silently stops learning. &lt;code&gt;logsumexp&lt;/code&gt; with the max shifted out stays finite:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;logsumexp&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;xs&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;m&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="kc"&gt;Infinity&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;x&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;xs&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;x&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;m&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="nx"&gt;m&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;x&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;s&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;x&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;xs&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="nx"&gt;s&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;exp&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;x&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="nx"&gt;m&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;m&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;s&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  How it is verified
&lt;/h2&gt;

&lt;p&gt;The page computes the loss with a stable &lt;code&gt;logsumexp&lt;/code&gt; and differentiates by hand. The verifier computes the softmax &lt;strong&gt;explicitly&lt;/strong&gt; from probabilities, with no &lt;code&gt;logsumexp&lt;/code&gt; anywhere, checks every gradient against a central finite difference of the total loss, and uses a different RNG and different embeddings.&lt;/p&gt;

&lt;p&gt;The gradient shape is checked as an identity rather than a value: &lt;code&gt;softmax_j − [j is the partner]&lt;/code&gt; must sum to zero over a row, be negative on the partner, and positive on every negative. &lt;strong&gt;511 asserts, 0 failures.&lt;/strong&gt; 19 in-page checks.&lt;/p&gt;

</description>
      <category>machinelearning</category>
      <category>deeplearning</category>
      <category>ai</category>
      <category>python</category>
    </item>
    <item>
      <title>Isotonic Calibration Cannot Change Your AUC by a Single Digit. That Is the Point, and Also the Catch.</title>
      <dc:creator>Devanshu Biswas</dc:creator>
      <pubDate>Tue, 03 Nov 2026 08:23:59 +0000</pubDate>
      <link>https://dev.to/dev48v/isotonic-calibration-cannot-change-your-auc-by-a-single-digit-that-is-the-point-and-also-the-273e</link>
      <guid>https://dev.to/dev48v/isotonic-calibration-cannot-change-your-auc-by-a-single-digit-that-is-the-point-and-also-the-273e</guid>
      <description>&lt;p&gt;Isotonic regression answers one question: of all non-decreasing sequences, which is closest to this data in least squares? Not approximately — &lt;strong&gt;exactly&lt;/strong&gt;, in one left-to-right pass.&lt;/p&gt;

&lt;p&gt;👉 &lt;strong&gt;Live, runs in your browser:&lt;/strong&gt; &lt;a href="https://dev48.in/ml/day82-isotonic-regression.html" rel="noopener noreferrer"&gt;https://dev48.in/ml/day82-isotonic-regression.html&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Its main job in practice is &lt;strong&gt;calibration&lt;/strong&gt;: turning scores that rank well into numbers that mean "probability".&lt;/p&gt;

&lt;h2&gt;
  
  
  It cannot touch your ranking metrics
&lt;/h2&gt;

&lt;p&gt;Calibration is a &lt;strong&gt;monotone&lt;/strong&gt; map. A monotone map cannot reorder anything. So AUC is untouched &lt;em&gt;by construction&lt;/em&gt;:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;scores&lt;/th&gt;
&lt;th&gt;AUC&lt;/th&gt;
&lt;th&gt;Brier, raw&lt;/th&gt;
&lt;th&gt;Brier, calibrated&lt;/th&gt;
&lt;th&gt;improvement&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;squash 1.00&lt;/td&gt;
&lt;td&gt;0.8186&lt;/td&gt;
&lt;td&gt;0.1729&lt;/td&gt;
&lt;td&gt;0.1739&lt;/td&gt;
&lt;td&gt;−0.6%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;squash 0.60&lt;/td&gt;
&lt;td&gt;0.8186&lt;/td&gt;
&lt;td&gt;0.1844&lt;/td&gt;
&lt;td&gt;0.1739&lt;/td&gt;
&lt;td&gt;5.7%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;squash 0.35&lt;/td&gt;
&lt;td&gt;0.8186&lt;/td&gt;
&lt;td&gt;0.2047&lt;/td&gt;
&lt;td&gt;0.1739&lt;/td&gt;
&lt;td&gt;15.1%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;squash 0.20&lt;/td&gt;
&lt;td&gt;0.8186&lt;/td&gt;
&lt;td&gt;0.2217&lt;/td&gt;
&lt;td&gt;0.1739&lt;/td&gt;
&lt;td&gt;21.6%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;squash 0.10&lt;/td&gt;
&lt;td&gt;0.8186&lt;/td&gt;
&lt;td&gt;0.2351&lt;/td&gt;
&lt;td&gt;0.1739&lt;/td&gt;
&lt;td&gt;26.0%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;0.8186 in every row&lt;/strong&gt;, to four decimals, while Brier moves 26%. The AUC column is not a coincidence to be checked, it is a theorem to be relied on.&lt;/p&gt;

&lt;p&gt;This cuts both ways. If you are reporting AUC to show that calibration helped, you are reporting the one number that &lt;em&gt;cannot&lt;/em&gt; respond. And the squash-1.00 row shows calibration slightly &lt;strong&gt;hurting&lt;/strong&gt; when the scores were already fine.&lt;/p&gt;

&lt;h2&gt;
  
  
  It overfits, and nothing in it stops that
&lt;/h2&gt;

&lt;p&gt;This is the part the tutorials leave out. Isotonic regression is an exact projection onto a convex cone. It is &lt;strong&gt;not a smoother&lt;/strong&gt;. Nothing in it regularises.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;training size&lt;/th&gt;
&lt;th&gt;blocks fitted&lt;/th&gt;
&lt;th&gt;Brier in-sample&lt;/th&gt;
&lt;th&gt;Brier out-of-sample&lt;/th&gt;
&lt;th&gt;ratio&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;40&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;0.1350&lt;/td&gt;
&lt;td&gt;0.1650&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;1.22×&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;100&lt;/td&gt;
&lt;td&gt;7&lt;/td&gt;
&lt;td&gt;0.1588&lt;/td&gt;
&lt;td&gt;0.1961&lt;/td&gt;
&lt;td&gt;1.23×&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;400&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;0.1671&lt;/td&gt;
&lt;td&gt;0.1764&lt;/td&gt;
&lt;td&gt;1.06×&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2000&lt;/td&gt;
&lt;td&gt;17&lt;/td&gt;
&lt;td&gt;0.1745&lt;/td&gt;
&lt;td&gt;0.1595&lt;/td&gt;
&lt;td&gt;0.91×&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Given a small validation set it will happily fit a step per point. That is not a bug — it is the method doing exactly what it was asked to do.&lt;/p&gt;

&lt;h2&gt;
  
  
  Optimality is characterised, not just computed
&lt;/h2&gt;

&lt;p&gt;The useful thing about a projection is that you can &lt;em&gt;check&lt;/em&gt; the answer without re-running the algorithm. Two conditions:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="c1"&gt;# 1. inside every level set, the fit IS the weighted mean of the data there
#    (otherwise moving that block toward the mean lowers the error
#     without breaking monotonicity - so it was not optimal)
# 2. the residual is orthogonal to the fit:  sum_i w_i (y_i - f_i) f_i = 0
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Both hold exactly, and neither calls PAVA. If your implementation passes these on your data, it is right on your data — independent of whether you trust the merge logic.&lt;/p&gt;

&lt;h2&gt;
  
  
  Verified by a completely different algorithm
&lt;/h2&gt;

&lt;p&gt;PAVA is a merge procedure: a stack of blocks, pooled left to right. The verifier never pools anything. It uses the other classical characterisation — the isotonic fit is the &lt;strong&gt;slope of the greatest convex minorant&lt;/strong&gt; of the cumulative-sum diagram:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// points P_0 = (0,0), P_i = (W_i, S_i); take the lower convex hull;&lt;/span&gt;
&lt;span class="c1"&gt;// the fit is the slope of the hull segment covering each index&lt;/span&gt;
&lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="nx"&gt;n&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;while &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;hull&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nf"&gt;cross&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;hull&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nf"&gt;at&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)],&lt;/span&gt; &lt;span class="nx"&gt;hull&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nf"&gt;at&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)],&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="nx"&gt;hull&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;pop&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
  &lt;span class="nx"&gt;hull&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;i&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A convex hull, not a merge. Both were then checked against a brute-force search over all &lt;code&gt;2^(n−1)&lt;/code&gt; block partitions on 55 sequences — including constants, strict reversals, singletons, negatives and weighted cases.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Largest SSE gap vs brute force: 0.00e+0.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;One assertion worth keeping: pooling on &lt;code&gt;&amp;gt;=&lt;/code&gt; rather than &lt;code&gt;&amp;gt;&lt;/code&gt; is why four equal points become &lt;em&gt;one&lt;/em&gt; block and not four. The fit is identical either way — but the block count is what the overfitting table reads, so the wrong comparison gives you a correct curve with a meaningless x-axis.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;357 verifier asserts, 0 failures.&lt;/strong&gt; 16 in-page checks.&lt;/p&gt;

</description>
      <category>machinelearning</category>
      <category>datascience</category>
      <category>statistics</category>
      <category>python</category>
    </item>
  </channel>
</rss>
