<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: JigsawLabs</title>
    <description>The latest articles on DEV Community by JigsawLabs (@jigsawlabs).</description>
    <link>https://dev.to/jigsawlabs</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4186475%2F7fa8b68c-54a0-4f3e-a560-0f7710027db1.png</url>
      <title>DEV Community: JigsawLabs</title>
      <link>https://dev.to/jigsawlabs</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/jigsawlabs"/>
    <language>en</language>
    <item>
      <title>Where Did 90% of My Photo Go? JPEG Compression, Explained Gently</title>
      <dc:creator>JigsawLabs</dc:creator>
      <pubDate>Thu, 15 Oct 2026 10:23:50 +0000</pubDate>
      <link>https://dev.to/jigsawlabs/where-did-90-of-my-photo-go-jpeg-compression-explained-gently-9ch</link>
      <guid>https://dev.to/jigsawlabs/where-did-90-of-my-photo-go-jpeg-compression-explained-gently-9ch</guid>
      <description>&lt;p&gt;Drag a 4 MB photo into a compressor, and a few seconds later you get a 400 KB file that looks… the same. Nine-tenths of the data is gone and your eye can't find the hole. That should bother you more than it does. &lt;em&gt;Where did 90% of the photo go?&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The answer is one of the most elegant engineering stories in everyday software — JPEG, designed in 1992 and still compressing most of the internet's images. It works because it knows three things about &lt;em&gt;you&lt;/em&gt;: what your eye measures sharply, what it measures poorly, and what it ignores completely. Compression isn't about shrinking data; it's about &lt;strong&gt;knowing which data was never really seen&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Our &lt;a href="https://jigsawlabs.tools/tools/image-shrink/" rel="noopener noreferrer"&gt;Compress Images&lt;/a&gt; tool runs this machinery in your browser. Let's open it up — three tricks, climbed step by step.&lt;/p&gt;

&lt;h2&gt;
  
  
  Trick 1 — Your eye is sharp on brightness, blurry on color
&lt;/h2&gt;

&lt;p&gt;Hold a red pen and a blue pen at arm's length, tips touching. You can see the exact pixel-sharp boundary between light and shadow on each pen — but the boundary between the two &lt;em&gt;colors&lt;/em&gt; is mushier to you than you think. Human retinas carry ~120 million brightness sensors and only ~6 million color sensors. You live in a world of crisp luminance and approximate color, and you've never noticed.&lt;/p&gt;

&lt;p&gt;JPEG's first move exploits this ruthlessly. It converts every pixel from "red, green, blue" into "brightness + two color channels," then &lt;strong&gt;stores the color channels at half the resolution in each direction&lt;/strong&gt; — one color sample per 2×2 block of pixels. That's called chroma subsampling, and it deletes half of all the data before any real compression begins.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fh6q9lspn9xlss1aiq91b.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fh6q9lspn9xlss1aiq91b.png" alt="The original scene split into a full-detail brightness layer and a quarter-size color layer, then recombined — indistinguishable from the original" width="800" height="480"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Look at the recombined panel in the figure. The color layer was stored at a quarter of the size — and you cannot tell. The first 50% of the "missing" photo was never visible to you in the first place.&lt;/p&gt;

&lt;h2&gt;
  
  
  Trick 2 — Every 8×8 block is a recipe of waves
&lt;/h2&gt;

&lt;p&gt;Now the clever part. JPEG chops the brightness layer into little &lt;strong&gt;8×8 pixel blocks&lt;/strong&gt; — 64 numbers each — and asks a strange question about every block: &lt;em&gt;instead of storing your 64 pixel values, what if I stored your recipe?&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Here's the idea in one dimension first. Any curve — say, the brightness along one row of pixels — can be built by &lt;strong&gt;adding up waves&lt;/strong&gt;: one broad smooth wave for the overall trend, plus smaller, faster ripples for the fine texture. This is the famous idea of Fourier, from 1822: complicated signals are sums of simple waves.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjigsawlabs.tools%2Fimages%2Fblog%2Fjpg-anim-waves.svg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjigsawlabs.tools%2Fimages%2Fblog%2Fjpg-anim-waves.svg" alt="Animated: a smooth wave plus fine ripples build a brightness curve; when the fine ripples are dropped, the curve barely changes" width="1000" height="320"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;In two dimensions, the waves become 64 standard patterns — from "completely flat" in the top-left corner, through broad soft gradients, down to a microscopic checkerboard in the bottom-right. This is the &lt;strong&gt;Discrete Cosine Transform (DCT)&lt;/strong&gt;, and the 64 patterns are its alphabet:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5wibom2cqowwpy9phzhx.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5wibom2cqowwpy9phzhx.png" alt="The 64 DCT basis patterns in an 8×8 grid, smooth patterns top-left and finest checkerboards bottom-right" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The DCT rewrites each block as a recipe: "a lot of pattern 1, a pinch of pattern 2, zero of pattern 59…" Crucially, &lt;strong&gt;nothing is lost yet&lt;/strong&gt; — 64 pixel numbers became 64 recipe numbers, and the block can be rebuilt perfectly.&lt;/p&gt;

&lt;p&gt;So why bother? Because photographs are overwhelmingly &lt;strong&gt;smooth&lt;/strong&gt;. Sky, skin, walls, blur — across an 8×8 block (that's a speck a couple of millimeters wide on your screen), brightness barely changes. So in block after block, the recipe says: big amounts of the smooth top-left patterns, and amounts of the busy patterns that are &lt;em&gt;nearly zero&lt;/em&gt;. The DCT doesn't delete anything — it &lt;strong&gt;sorts the information by how much your eye cares&lt;/strong&gt;, smooth and visible at one end, microscopic and invisible at the other. It sets up the kill.&lt;/p&gt;

&lt;h2&gt;
  
  
  Trick 3 — The quality slider is a rounding dial
&lt;/h2&gt;

&lt;p&gt;Now the deletion — and the real meaning of that "quality" slider you've dragged a hundred times.&lt;/p&gt;

&lt;p&gt;Take each recipe amount and &lt;strong&gt;divide it by a step size, then round to the nearest whole number&lt;/strong&gt;. High quality = small steps: dividing 42 by 4 and rounding gives 11 — most of the information survives. Low quality = big steps: dividing 42 by 24 gives 2, and dividing that near-zero fine-texture amount by 24 gives… &lt;strong&gt;0&lt;/strong&gt;. And 0 is the magic word, because a long run of zeros costs almost nothing to store.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyqy4w6dnah6q0wjdhqay.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyqy4w6dnah6q0wjdhqay.png" alt="The same block's recipe divided by small steps and by big steps — at low quality only a handful of the 64 numbers survive" width="800" height="480"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That's the entire quality dial: &lt;strong&gt;how coarsely are the wave amounts rounded.&lt;/strong&gt; Quality 90 rounds gently and keeps fine texture. Quality 45 rounds harder — in the figure the picture still looks fine, at a third of the bytes. Quality 8 rounds so brutally that only each block's average survives, and suddenly you can &lt;em&gt;see the 8×8 blocks themselves&lt;/em&gt; — the famous JPEG blockiness is literally the grid of the algorithm showing through.&lt;/p&gt;

&lt;p&gt;After rounding, the surviving numbers are read out in a zigzag (so the zeros bunch together at the end) and squeezed with classic lossless tricks — run-length counting and Huffman coding, the "say it shorter if it's common" family. But the hard decisions are already made. The file is small because the rounding dial decided which waves you'd never miss.&lt;/p&gt;

&lt;h2&gt;
  
  
  So where did the 90% go?
&lt;/h2&gt;

&lt;p&gt;Half was color detail your retina never had sensors for. Most of the rest was fine-texture wave amounts that were already near zero and got rounded to exactly zero. What's left — the broad shapes, the edges, the brightness your eye actually measures — survives nearly untouched. The photo didn't lose 90% of what you &lt;em&gt;see&lt;/em&gt;. It lost 90% of what you were never seeing.&lt;/p&gt;

&lt;p&gt;One practical note from our own tool: compression this good is also why &lt;strong&gt;re-compressing a JPEG over and over degrades it&lt;/strong&gt; — each pass re-rounds the recipes, and the rounding errors stack. Compress from the best original you have, once, at the quality you need.&lt;/p&gt;

&lt;p&gt;Try the dial yourself and watch the bytes fall as you drag: &lt;strong&gt;&lt;a href="https://jigsawlabs.tools/tools/image-shrink/" rel="noopener noreferrer"&gt;jigsawlabs.tools/tools/image-shrink&lt;/a&gt;&lt;/strong&gt; — free, no signup, and like every JigsawLabs tool it runs entirely in your browser, so your photos never leave your device.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://jigsawlabs.tools/blog/where-did-my-photo-go-jpeg/" rel="noopener noreferrer"&gt;JigsawLabs Blog&lt;/a&gt;. This is the fourth post in a series on the engineering inside &lt;a href="https://jigsawlabs.tools/" rel="noopener noreferrer"&gt;JigsawLabs&lt;/a&gt; tools: &lt;a href="https://dev.to/jigsawlabs/what-if-we-gave-the-browser-a-small-brain-3701"&gt;the 55 KB brain that formats documents&lt;/a&gt;, &lt;a href="https://dev.to/jigsawlabs/how-an-ai-cuts-you-out-of-a-photo-24-million-decisions-at-a-time-3gkj"&gt;how an AI cuts you out of a photo&lt;/a&gt;, and &lt;a href="https://dev.to/jigsawlabs/how-a-browser-reads-a-photo-ocr-explained-step-by-step-3535"&gt;how OCR reads photos&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>imageprocessing</category>
      <category>compression</category>
      <category>math</category>
      <category>webdev</category>
    </item>
    <item>
      <title>How a Browser Reads a Photo — OCR Explained Step by Step</title>
      <dc:creator>JigsawLabs</dc:creator>
      <pubDate>Thu, 15 Oct 2026 10:20:18 +0000</pubDate>
      <link>https://dev.to/jigsawlabs/how-a-browser-reads-a-photo-ocr-explained-step-by-step-3535</link>
      <guid>https://dev.to/jigsawlabs/how-a-browser-reads-a-photo-ocr-explained-step-by-step-3535</guid>
      <description>&lt;p&gt;Take a photo of a printed page and ask your computer to type it out. The job sounds like one step — "read it" — but a computer starts from a place so humble it's almost funny: all it has is a grid of millions of brightness numbers. Nothing in that grid says "letter." Nothing says "word." There is no text in a photo of text.&lt;/p&gt;

&lt;p&gt;Optical character recognition — OCR — is the staircase from those raw numbers up to the sentence you can copy-paste. It has only three stairs, and each one is honest enough to explain completely. Let's climb them with a real example: a phone photo of meeting notes, slightly shadowed on one side, like every photo ever taken in a hurry.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1 — Every pixel becomes ink or paper
&lt;/h2&gt;

&lt;p&gt;First problem: the photo is a mess of grays. The paper isn't white (it's in shadow on the right), and the ink isn't black (cameras soften everything). Before any reading happens, OCR wants a brutally simple world: every pixel is either &lt;strong&gt;ink&lt;/strong&gt; or &lt;strong&gt;paper&lt;/strong&gt;. Nothing in between.&lt;/p&gt;

&lt;p&gt;How do you pick the cutoff? Count the pixels. Make a &lt;strong&gt;histogram&lt;/strong&gt; — a bar chart of "how many pixels have each brightness, from 0 (black) to 255 (white)." A photo of text always shows the same beautiful shape: &lt;strong&gt;two hills&lt;/strong&gt;. A big hill of bright pixels (all that paper) and a smaller hill of dark pixels (the ink). Between them: a valley.&lt;/p&gt;

&lt;p&gt;The cutoff should sit in that valley — and a method from 1979 called &lt;strong&gt;Otsu's method&lt;/strong&gt; finds it with pure arithmetic: try every possible threshold from 0 to 255, and for each one, measure how cleanly it splits the pixels into two tight groups. Keep the threshold with the cleanest split. That's the whole algorithm — a for-loop, no AI. On our sample photo it lands on 104:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F8pf5rzzb9od5bddx76xu.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F8pf5rzzb9od5bddx76xu.png" alt="A shadowed photo of notes, its brightness histogram with two hills and the threshold in the valley, and the clean black-and-white result" width="800" height="480"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Every pixel darker than the valley becomes pure black; everything brighter becomes pure white. The shadow vanishes — because the shadow's pixels, dark as they were, still sat on the bright side of the valley. This step is why decent OCR tools shrug off bad lighting that would ruin a naive "is it dark?" rule.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 2 — Find the text before reading it
&lt;/h2&gt;

&lt;p&gt;Now the page is black ink on white paper, but the computer still doesn't know where the words are. The fix is geometry, not intelligence.&lt;/p&gt;

&lt;p&gt;Black pixels that &lt;strong&gt;touch each other&lt;/strong&gt; get grouped into clumps — each clump is a letter (or a few letters stuck together). Clumps that sit close together on the same baseline become a &lt;strong&gt;word&lt;/strong&gt;. Words sharing a baseline become a &lt;strong&gt;line&lt;/strong&gt;. It's exactly how you'd organize spilled beads: things that are near each other and aligned probably belong together.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fmjog2tbqqsiu9eaqy1rj.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fmjog2tbqqsiu9eaqy1rj.png" alt="The binarized page with blue boxes around lines and mint boxes around each word" width="800" height="440"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The output of this stair is a stack of little image strips, one per line of text, each handed to the reader with a note attached: "this strip contains exactly one line — read it."&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 3 — Reading is a sequence, not a lookup
&lt;/h2&gt;

&lt;p&gt;Here is where old OCR and modern OCR part ways, and the difference teaches a real lesson about machine learning.&lt;/p&gt;

&lt;p&gt;Old OCR tried to cut each word into individual letters and match each letter against stencils — does this blob look like the "B" template? It worked on clean typewriter fonts and fell apart on everything else, because real text refuses to be cut: letters touch, fonts vary, and in cursive scripts the letters &lt;em&gt;flow into each other&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;Modern OCR — the engine in our tool is &lt;strong&gt;Tesseract&lt;/strong&gt;, with its neural "LSTM" reader — stops trying to cut. Instead, a small network &lt;strong&gt;slides along the strip&lt;/strong&gt; like a reading head, a few pixels at a time. At every step it shouts probabilities: "81% this is part of a &lt;em&gt;d&lt;/em&gt;, 9% maybe &lt;em&gt;cl&lt;/em&gt;…" — and crucially, it carries &lt;strong&gt;memory&lt;/strong&gt; of what it has already seen. Having read "Budge…", it strongly expects a &lt;em&gt;t&lt;/em&gt; next. (The memory cell it uses is called an LSTM — "long short-term memory" — which is exactly what it sounds like: a little notebook the network keeps as it reads.)&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F30fyog7514bm59cu08xx.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F30fyog7514bm59cu08xx.png" alt="The reading head sliding across the word Budget, with probability bars at one step, and the repeated shouts collapsing into the final word" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;But wait — the head takes twelve little steps across a six-letter word, so its raw output is a stutter: "B B u u _ d d g _ e t t". A wonderfully simple trick called &lt;strong&gt;CTC&lt;/strong&gt; turns the stutter into text: collapse the repeats, drop the silences. "BBuu_ddg_ett" becomes "Budget". The deep consequence: &lt;strong&gt;nobody ever has to tell the network where one letter ends and the next begins&lt;/strong&gt; — it learns its own timing from examples, which is precisely what made letter-cutting unnecessary.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjigsawlabs.tools%2Fimages%2Fblog%2Focr-anim-scan.svg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjigsawlabs.tools%2Fimages%2Fblog%2Focr-anim-scan.svg" alt="Animated: the reading head sweeps across the word and letters are committed as it passes" width="1000" height="300"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Arabic is the real test
&lt;/h2&gt;

&lt;p&gt;Our tool reads English and Arabic — and Arabic is where the sequence approach proves itself. Arabic script is cursive by design: most letters connect, and a letter changes &lt;em&gt;shape&lt;/em&gt; depending on whether it sits at the start, middle, or end of a word — the same letter can look four different ways. Stencil-matching has no chance. But a reader with memory, sliding along the line (right to left, this time), handles it the way a human reader does: shape-in-context. That's why the tool ships two trained models, one per language, each a few megabytes of learned weights — downloaded once to your browser, cached, and run locally.&lt;/p&gt;

&lt;p&gt;And that sentence is doing a lot of work: &lt;strong&gt;locally&lt;/strong&gt;. Receipts, contracts, IDs, prescriptions — photos of text are the most sensitive images people process online, and the standard OCR site uploads them to a server. Ours doesn't have a server to upload to. The staircase you just climbed — histogram, valley, clumps, boxes, sliding reader — runs inside your tab, and you can watch the network panel stay silent while it works.&lt;/p&gt;

&lt;p&gt;Try it on your own messy photo — English, Arabic, or both at once: &lt;strong&gt;&lt;a href="https://jigsawlabs.tools/tools/image-to-text/" rel="noopener noreferrer"&gt;jigsawlabs.tools/tools/image-to-text&lt;/a&gt;&lt;/strong&gt; — free, no signup, and the photo never leaves your device.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://jigsawlabs.tools/blog/how-ocr-reads-photos/" rel="noopener noreferrer"&gt;JigsawLabs Blog&lt;/a&gt;. This is the third post in a series on the engineering inside &lt;a href="https://jigsawlabs.tools/" rel="noopener noreferrer"&gt;JigsawLabs&lt;/a&gt; tools: &lt;a href="https://dev.to/jigsawlabs/what-if-we-gave-the-browser-a-small-brain-3701"&gt;the 55 KB brain that formats documents&lt;/a&gt; and &lt;a href="https://dev.to/jigsawlabs/how-an-ai-cuts-you-out-of-a-photo-24-million-decisions-at-a-time-3gkj"&gt;how an AI cuts you out of a photo&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>machinelearning</category>
      <category>imageprocessing</category>
      <category>privacy</category>
      <category>ocr</category>
    </item>
    <item>
      <title>How an AI Cuts You Out of a Photo — 24 Million Decisions at a Time</title>
      <dc:creator>JigsawLabs</dc:creator>
      <pubDate>Thu, 15 Oct 2026 10:10:44 +0000</pubDate>
      <link>https://dev.to/jigsawlabs/how-an-ai-cuts-you-out-of-a-photo-24-million-decisions-at-a-time-3gkj</link>
      <guid>https://dev.to/jigsawlabs/how-an-ai-cuts-you-out-of-a-photo-24-million-decisions-at-a-time-3gkj</guid>
      <description>&lt;p&gt;Tap "remove background" and something has to happen to every single pixel.&lt;/p&gt;

&lt;p&gt;A photo from a modern phone has about 12 million pixels. For each one, the tool must answer a question: &lt;em&gt;are you part of the person, or part of the background?&lt;/em&gt; Twelve million questions — and here is the twist that makes this problem genuinely hard: &lt;strong&gt;yes and no are not good enough.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Zoom into the edge of someone's hair. Keep zooming, until you can see individual pixels. Now find the exact pixel where "person" stops and "background" begins. You can't — because there isn't one. A single pixel at the wispy edge of a curl is &lt;em&gt;partly&lt;/em&gt; hair and &lt;em&gt;partly&lt;/em&gt; the wall behind it. Its color is literally a blend: some fraction hair, the remaining fraction wall. The same is true along shoulders, around fingers, through anything slightly out of focus.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq0nmi4wg9xhvyr7cjfln.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq0nmi4wg9xhvyr7cjfln.png" alt="Zooming into a hair edge until single pixels are visible — one highlighted pixel is 62% person, 38% wall" width="800" height="460"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;So the real question the tool must answer, twelve million times over, is not "person: yes or no?" but "person: &lt;strong&gt;how much&lt;/strong&gt;, from 0% to 100%?" Get those percentages wrong and you get the look everyone recognizes from cheap cutouts: helmet hair, jagged halos, edges that look scissored by a hurried child.&lt;/p&gt;

&lt;p&gt;Answering a percentage question millions of times, correctly, in a couple of seconds, on your own device — that is the whole story of a background remover. The rest of this post is just: &lt;em&gt;how?&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1 — What the AI actually draws: the alpha matte
&lt;/h2&gt;

&lt;p&gt;Here is the surprise that unlocks everything else: the neural network never cuts anything. It cannot delete pixels, and it never touches your photo.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;It draws a second image.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;That second image is the same size as your photo, but it's a ghost: pure grayscale, no colors. Each of its pixels holds exactly one number — the answer to the question above for the matching pixel of your photo. White means 100%: "this pixel is fully person." Black means 0%: "fully background." And the grays in between are the whole reason this works: a middle gray at the edge of a curl means "this pixel is about half hair."&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fr9y8cwuija32waeam73z.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fr9y8cwuija32waeam73z.png" alt="Photo, then its grayscale alpha matte, then the cutout on a checkerboard — the AI paints the ghost, the rest is mechanical" width="800" height="460"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This ghost image has a name: the &lt;strong&gt;alpha matte&lt;/strong&gt;. The word &lt;em&gt;matte&lt;/em&gt; is older than computers — it comes from cinema, where artists painted mattes on glass to block out parts of the frame, long before green screens. The film industry's problem a century ago was exactly ours: how do you say, for every point of the picture, &lt;em&gt;how much&lt;/em&gt; of it belongs to the foreground?&lt;/p&gt;

&lt;p&gt;Once you have the matte, "removing the background" stops being intelligent work at all. Your photo supplies the colors; the matte supplies the percentages; gluing them together as an image-with-transparency is one mechanical pass. The person keeps her exact pixels — every strand at 100% — and each half-hair edge pixel becomes half-transparent, which is what lets it sit softly on whatever comes next: pure white for a passport photo, a beach, anything.&lt;/p&gt;

&lt;p&gt;So when you tap the button, the intelligence has exactly one job, and it is weirdly artistic: &lt;strong&gt;stare at a photograph and paint its ghost&lt;/strong&gt; — 12 million careful shades of gray. Everything after that is arithmetic.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 2 — How the ghost-painter thinks: two artists, one canvas
&lt;/h2&gt;

&lt;p&gt;Meet the painter. The tool on jigsawlabs.tools uses a neural network called &lt;strong&gt;MODNet&lt;/strong&gt; — a design that came out of university research on &lt;em&gt;portrait matting&lt;/em&gt;, and the same network we ship to your browser. What makes it worth explaining is that its inner structure mirrors something you already do with your own eyes.&lt;/p&gt;

&lt;p&gt;Look at a portrait and squint. Details vanish; blobs remain. Even blurred, you know exactly where the person is — a squint is enough for the &lt;em&gt;shape&lt;/em&gt;. Now lean in close to one spot — say the boundary of a shoulder. Up close you can trace the edge precisely — but if I only ever showed you that zoomed patch, you couldn't tell me whether it belongs to a shoulder or a shoe.&lt;/p&gt;

&lt;p&gt;Those are two different skills, and they fail in opposite ways. So MODNet is built, literally, as &lt;strong&gt;two artists sharing one canvas&lt;/strong&gt;:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The squinter&lt;/strong&gt; works on a shrunken copy of your photo. Detail is gone, so it can only judge the big picture — and that's the point. Its half of the job: paint a rough, blurry silhouette. &lt;em&gt;The person is HERE.&lt;/em&gt; It is wise about "where" and hopeless about "exactly."&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The edge-tracer&lt;/strong&gt; works close to full resolution, but only cares about a narrow band of the image: the boundary region the squinter pointed at. Along that band it does the delicate work — the strands, the soft blur where a curl meets the wall, the gap between arm and body. It is precise about "exactly" and clueless about "where."&lt;/p&gt;

&lt;p&gt;A third stage — call it &lt;strong&gt;the editor&lt;/strong&gt; — merges the two: the squinter's silhouette everywhere, the edge-tracer's fine grays wherever the two overlap, one clean matte out. (In the research paper these are the &lt;em&gt;semantic&lt;/em&gt;, &lt;em&gt;detail&lt;/em&gt;, and &lt;em&gt;fusion&lt;/em&gt; branches; the paper's real contribution was training all three to cooperate in one pass, fast enough for real time.)&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpnfumxsxn6bvr7kgsmku.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpnfumxsxn6bvr7kgsmku.png" alt="The photo feeding two branches — a blurred silhouette and a full-resolution edge crop — merged by a fusion branch into one clean matte" width="800" height="600"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Here's the sentence worth remembering from this whole section: &lt;strong&gt;the network doesn't succeed because it's huge — it succeeds because the problem was split the way vision actually works.&lt;/strong&gt; Shape is a low-resolution question. Edges are a local question. Neither needs the other's data, so neither wastes capacity on the other's job.&lt;/p&gt;

&lt;p&gt;And how did the artists learn their crafts? The boring, honest answer: practice. The network was trained on many thousands of portrait photos that came with hand-made, pixel-perfect mattes — guess, compare, adjust, millions of times, until its guesses stopped being guesses. There is no rulebook inside that says "hair is thin and wispy"; there are only numbers that drifted, example by example, toward painting good ghosts.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 3 — The whole trick is one line of math
&lt;/h2&gt;

&lt;p&gt;I promised math, and here it is — the entire "put me on a beach" feature, the same line our tool runs for every pixel:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;new pixel = α · person + (1 − α) · new background&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;That α (alpha) is the matte's number for this pixel, written as 0-to-1 instead of a percentage. Read the line aloud and it turns out you already knew it: &lt;em&gt;take α parts of the person, fill the rest with the new background.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Watch it work on three pixels:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A pixel in the middle of the face: α = 1. The formula collapses to &lt;em&gt;new = person&lt;/em&gt;. Untouched — which is why nobody's cheek ever gets "processed."&lt;/li&gt;
&lt;li&gt;A pixel of open sky in the new background: α = 0. It collapses to &lt;em&gt;new = background&lt;/em&gt;. Pure beach.&lt;/li&gt;
&lt;li&gt;And the pixel from the start of this post — the one that was 62% hair over a wall: α = 0.62. The result is a real mix: 62% of the dark hair color plus 38% of whatever now sits behind — sky blue, passport white. &lt;strong&gt;The formula manufactures exactly the in-between color your eye expects at a soft edge.&lt;/strong&gt; That's the whole secret of hair that doesn't look scissored: the edge isn't preserved, it's &lt;em&gt;recomputed&lt;/em&gt; against the new background.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq0boxbbvuf5nqb8jtodq.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq0boxbbvuf5nqb8jtodq.png" alt="The equation computed on real swatches: 0.62 times the hair color plus 0.38 times the beach color equals the edge pixel" width="800" height="460"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Want to &lt;em&gt;feel&lt;/em&gt; it instead of reading it? We built a tiny playground where you drag α and the equation computes live: &lt;strong&gt;&lt;a href="https://jigsawlabs.tools/play/alpha-matte/" rel="noopener noreferrer"&gt;the alpha compositing playground&lt;/a&gt;&lt;/strong&gt; — it runs the exact line above on colors you pick.&lt;/p&gt;

&lt;p&gt;One line, three multiplications and an addition per color channel, twelve million pixels — your phone does it in the blink of a frame. The film industry has a name for this too: &lt;strong&gt;alpha compositing&lt;/strong&gt;, formalized back in 1984 by Porter and Duff, and it is quietly running under every movie green-screen shot, every game HUD, every translucent window on your screen. Our tool didn't invent the glue; it earned the α.&lt;/p&gt;

&lt;p&gt;And that's a clean split worth saying out loud: the neural network's ONLY job was those α values. Everything visible after that — remove, replace, passport white — is this one honest line of arithmetic.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 4 — The pasted look, and the statistics that fix it
&lt;/h2&gt;

&lt;p&gt;Perfect edges are not enough. You have seen the proof a hundred times: a cutout with flawless hair, dropped onto a sunny beach — and still, instantly, unmistakably &lt;em&gt;pasted&lt;/em&gt;. Why?&lt;/p&gt;

&lt;p&gt;Because light is a liar you can't un-hear. The original photo was taken under warm indoor lamps; the beach is cool, bright, blue-shifted. Every pixel of the person still carries the fingerprint of the old room, and your eye — superbly trained by a lifetime of real scenes — notices that the person and the place disagree about what the light is.&lt;/p&gt;

&lt;p&gt;Here's the beautiful part: fixing this needs no neural network at all. It needs six numbers.&lt;/p&gt;

&lt;p&gt;The color "mood" of an image is captured shockingly well by simple statistics: for each color channel, the &lt;strong&gt;average&lt;/strong&gt; (is this image overall warm? dark? blue?) and the &lt;strong&gt;spread&lt;/strong&gt; (are its colors muted or punchy?). Two numbers per channel, three channels — six numbers describe the light's fingerprint.&lt;/p&gt;

&lt;p&gt;So to make a person borrow the beach's light, you shift their statistics toward the beach's statistics. For every pixel of the person, per channel:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;new = (old − person_avg) × (scene_spread / person_spread) + scene_avg&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Subtract the person's average (remove the old room's fingerprint), rescale by the ratio of spreads (adopt the scene's contrast), add the scene's average (put on the new light). This recipe is called &lt;strong&gt;Reinhard color transfer&lt;/strong&gt;, published in 2001 — done in a color space built to match human perception, and twenty-five years later it remains the workhorse because it is instant, predictable, and has no moods of its own.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F0g21kjwh1a36zrwapx1p.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F0g21kjwh1a36zrwapx1p.png" alt="Side by side: the pasted cutout with indoor colors on beach light, and the harmonized version after the statistics shift" width="800" height="520"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;One ingredient matters in practice: &lt;strong&gt;gentleness&lt;/strong&gt;. Match the statistics 100% and the person starts dissolving into the scenery — identity sacrificed to atmosphere. That's why our tool exposes it as a strength slider and applies the shift &lt;em&gt;partially&lt;/em&gt;: borrow the scene's mood, keep your own face. (In the figure, even a one-third strength shift already makes the "pasted" feeling collapse.)&lt;/p&gt;

&lt;p&gt;Step back and look at the full pipeline you now understand: a neural network paints the matte (Step 2), one line of arithmetic recomposes every pixel (Step 3), six statistics make the light agree (Step 4). The "magic" button is a matte, a multiply, and a mood.&lt;/p&gt;

&lt;h2&gt;
  
  
  The part nobody believes: all of it stayed in your tab
&lt;/h2&gt;

&lt;p&gt;Now the twist that this whole series keeps arriving at from different directions.&lt;/p&gt;

&lt;p&gt;Everything you just read about — the ghost-painting network, the twelve million compositing equations, the statistics of light — runs &lt;strong&gt;inside your browser tab&lt;/strong&gt;. Not on our servers. We don't have servers that could do it. When you open the tool, the model travels &lt;em&gt;to you&lt;/em&gt; — about six megabytes, once, cached like any image — and from that moment the pipeline runs on your own device. Open your browser's network panel and watch while it works: your photo never leaves.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fc2brvr1ywz9ubhh607uf.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fc2brvr1ywz9ubhh607uf.png" alt="Two architectures: upload your photo to their server, versus download the model once and the photo stays in your tab" width="800" height="440"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That one architectural choice quietly deletes the business model of an entire tool industry:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Serving another user costs us nothing, so the tool can be &lt;strong&gt;free&lt;/strong&gt; — no signup, no daily limit, no watermark held for ransom.&lt;/li&gt;
&lt;li&gt;Your photo was never sent, so privacy is not a promise we make — it's a &lt;strong&gt;property of the design&lt;/strong&gt;. There is nothing to leak, subpoena, or train on.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The usual way and the browser way differ by a single arrow: either your photo travels to the intelligence, or the intelligence travels to your photo. Everything else — the paywalls on one side, the privacy on the other — is downstream of that arrow's direction.&lt;/p&gt;

&lt;p&gt;Try the whole pipeline yourself — drop a photo, watch the matte do its work, slide the harmony strength and see Reinhard's six numbers move a person between worlds: &lt;strong&gt;&lt;a href="https://jigsawlabs.tools/tools/remove-background/" rel="noopener noreferrer"&gt;jigsawlabs.tools/tools/remove-background&lt;/a&gt;&lt;/strong&gt; — free, no signup, and your photo stays yours.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://jigsawlabs.tools/blog/how-ai-removes-backgrounds/" rel="noopener noreferrer"&gt;JigsawLabs Blog&lt;/a&gt;. This is the second post in a series on the engineering inside &lt;a href="https://jigsawlabs.tools/" rel="noopener noreferrer"&gt;JigsawLabs&lt;/a&gt; tools. The first — &lt;a href="https://dev.to/jigsawlabs/what-if-we-gave-the-browser-a-small-brain-3701"&gt;What If We Gave the Browser a Small Brain?&lt;/a&gt; — explains the 55 KB neural network that formats documents.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>machinelearning</category>
      <category>imageprocessing</category>
      <category>privacy</category>
      <category>webdev</category>
    </item>
    <item>
      <title>What If We Gave the Browser a Small Brain?</title>
      <dc:creator>JigsawLabs</dc:creator>
      <pubDate>Thu, 15 Oct 2026 10:07:32 +0000</pubDate>
      <link>https://dev.to/jigsawlabs/what-if-we-gave-the-browser-a-small-brain-3701</link>
      <guid>https://dev.to/jigsawlabs/what-if-we-gave-the-browser-a-small-brain-3701</guid>
      <description>&lt;p&gt;Here is a question that has been quietly haunting me:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;What is a browser actually capable of?&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;We treat it as a window — a pane of glass we look &lt;em&gt;through&lt;/em&gt; at software running somewhere else. Every "intelligent" thing we do online follows the same ritual: our words, our photos, our files travel to a distant computer, something happens there, and the result travels back. The browser just displays it.&lt;/p&gt;

&lt;p&gt;But the device you are reading this on is a supercomputer by any historical standard, and the browser is the most widely deployed runtime in human history — sitting on billions of machines, capable of running real computation, and mostly used to render other people's servers.&lt;/p&gt;

&lt;p&gt;So the question sharpens into something more interesting:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;What if we gave the browser a small brain? Not a connection to intelligence somewhere else — an actual resident intelligence, living inside the page the way a stylesheet does. What power could we unleash on the devices we already own?&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Hold that question. This post is about the first place it landed for me — and the surprisingly small amount of mathematics it took to answer it.&lt;/p&gt;

&lt;h2&gt;
  
  
  The first place the question landed: documents
&lt;/h2&gt;

&lt;p&gt;I work with documents constantly, and like most people I don't produce them in one sitting — I produce &lt;em&gt;fragments&lt;/em&gt;. Meeting notes on a laptop. Half an outline dictated into a phone. Bullet points pasted from three chats. And every few days comes the moment where the scattered pile has to become a &lt;em&gt;presentable&lt;/em&gt; document: a real one, with a cover page, headings, numbered sections, a table that looks like a table.&lt;/p&gt;

&lt;p&gt;The gap between "text I have" and "document I can send" is tiny in theory and maddening in practice. So the haunting question turned concrete:&lt;/p&gt;

&lt;p&gt;&lt;em&gt;What if a browser tool could look at my scattered notes and understand — even a little — how they should be formatted? What if it lived entirely in the tab: nothing to download, nothing to install, no signup, no upload? How much friction would simply disappear?&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Notice what the last part rules out. The tools that promise this today almost all work the same way: &lt;strong&gt;upload your text to our server, and our AI will handle it.&lt;/strong&gt; That sentence deserves more suspicion than it gets. Notes contain client names, unreleased plans, half-formed ideas — a meeting summary is one of the most sensitive documents a person produces all week, and the standard workflow ships it to someone else's computer to get a cover page stapled on.&lt;/p&gt;

&lt;p&gt;I wanted the intelligence to come to the document, not the document to the intelligence. The answer turned out to be a neural network with 6,471 parameters that weighs 55 KB — smaller than the favicon on some websites. I think of it as a &lt;em&gt;small brain&lt;/em&gt;, and I've come to believe small brains are one of the most underrated ideas in software right now.&lt;/p&gt;

&lt;h2&gt;
  
  
  The landscape: cathedral AI
&lt;/h2&gt;

&lt;p&gt;Today's defining AI systems are cathedrals: hundreds of billions of parameters, running in datacenters, reached through an API. They are genuinely remarkable, and for open-ended work — writing, reasoning, conversation — nothing smaller comes close.&lt;/p&gt;

&lt;p&gt;But the cathedral model quietly became the &lt;em&gt;default&lt;/em&gt; architecture for every intelligent feature, including ones that never needed it. Want to detect whether a line of text is a heading? Send the document to the cloud. The result is a strange bargain that we've all stopped noticing: &lt;strong&gt;to make text prettier, you surrender the text.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;There's a second cost besides privacy: economics. Every cloud call costs the tool's maker money, which is why "free" document tools come with signups, watermarks, daily limits, and pricing pages. The paywall isn't greed, mostly — it's physics. Someone has to pay for the datacenter.&lt;/p&gt;

&lt;p&gt;Flip the architecture and both costs vanish at once. If the intelligence runs on the user's own device, the marginal cost of serving one more user is zero — so the tool can actually &lt;em&gt;be&lt;/em&gt; free — and the data never moves, so privacy isn't a policy, it's a property of the design.&lt;/p&gt;

&lt;p&gt;The question is whether device-sized intelligence is &lt;em&gt;enough&lt;/em&gt; for the job. For document structure, it turns out 55 KB is plenty. Here's why.&lt;/p&gt;

&lt;h2&gt;
  
  
  The problem, precisely
&lt;/h2&gt;

&lt;p&gt;Take this fragment of a messy note:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;QUARTERLY REVIEW
prepared by finance team

Revenue: 410K
Costs: 180K

next steps
- hire two engineers
- renegotiate hosting contract
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You parsed that instantly, without being told: a title, a byline, two key–value pairs, a heading ("next steps"), a list. Formatting a document is exactly this act of &lt;em&gt;seeing structure&lt;/em&gt; — deciding, for every line, what role it plays. Once each line has a role, the rest is mechanical: titles get cover pages, headings get accent colors and numbering, lists get bullets, tables get borders.&lt;/p&gt;

&lt;p&gt;So the whole intelligent part of the problem compresses into one question:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Given a line of text, is it a TITLE, a HEADING, BODY text, a LIST item, a KEY–VALUE pair, a TABLE row — or NOISE (page numbers, boilerplate) that should be thrown away?&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Seven categories. That's it. This is not open-ended reasoning; it's a classification problem — and classification problems this small have been solvable since long before the cathedral era.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fn3jl85u8io3wm457r9a4.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fn3jl85u8io3wm457r9a4.png" alt="Every line of the messy note labeled with one of seven roles: TITLE, BODY, KV, HEADING, LIST, NOISE — label every line and formatting becomes mechanical" width="800" height="460"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  How the small brain works
&lt;/h2&gt;

&lt;p&gt;You could try rules: "if a line is short and ends with a colon, it's a heading." Rules get you surprisingly far — and then betray you. Is &lt;code&gt;Revenue 2024:&lt;/code&gt; a heading, or a key–value pair missing its value? Is a short capitalized line a title, or an angry sentence? Every new rule you add to fix one case breaks another. Rule systems don't degrade gracefully; they &lt;em&gt;argue with themselves&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;The alternative is to let the boundaries be learned from examples. Here's the entire recipe — this is genuinely all of it:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. Measure each line.&lt;/strong&gt; The model never sees words. Each line is reduced to &lt;strong&gt;31 numbers&lt;/strong&gt; — honest, human-readable measurements: How long is it? What fraction is digits? Does it start with a bullet character? End with a colon? Is it ALL CAPS? Does it look like &lt;code&gt;key: value&lt;/code&gt;? Is there a blank line above it? Whereabouts in the document does it sit?&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Add context.&lt;/strong&gt; A line alone can be ambiguous; a line &lt;em&gt;between two other lines&lt;/em&gt; rarely is. (&lt;code&gt;Revenue: 410K&lt;/code&gt; directly above &lt;code&gt;Costs: 180K&lt;/code&gt; is clearly part of a key–value block.) So the model looks at each line together with its neighbors: 31 numbers × 3 lines = &lt;strong&gt;93 inputs&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. One layer of neurons.&lt;/strong&gt; The 93 inputs feed 64 tiny decision units. Each one computes a weighted sum of everything it sees and stays silent unless the evidence crosses its threshold — a "neuron" is nothing more mysterious than that. Their 64 outputs are combined into 7 scores, one per category, and the scores become probabilities.&lt;/p&gt;

&lt;p&gt;That's the whole brain: 6,471 learned numbers. For comparison, the models behind chatbots have hundreds of &lt;em&gt;billions&lt;/em&gt;. This one is roughly &lt;strong&gt;fifty million times smaller&lt;/strong&gt; — because the question it answers is fifty million times narrower.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F15d85qo3zvgqtp7xhbkn.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F15d85qo3zvgqtp7xhbkn.png" alt="The whole network: 93 input numbers (31 per line, three lines of context) feeding 64 neurons, feeding 7 label scores — 6,471 parameters, 55 KB" width="800" height="520"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;On held-out test documents it labels &lt;strong&gt;99.8%&lt;/strong&gt; of lines correctly. A tuned rule-based system reaches about 99.1% on the same test — so the honest claim is not that the network is magic, but that it cleanly wins the ambiguous cases where rules argue with themselves, and it never gets &lt;em&gt;worse&lt;/em&gt; as it learns new document styles. (When it is unsure, the tool falls back to conservative formatting — a wrongly-bolded line is annoying; a deleted one is unforgivable.)&lt;/p&gt;

&lt;p&gt;And because it's 55 KB of arithmetic, it classifies an entire document in about a millisecond, on a phone, offline.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this makes possible
&lt;/h2&gt;

&lt;p&gt;This architecture became &lt;a href="https://jigsawlabs.tools/tools/docforge/" rel="noopener noreferrer"&gt;DocForge&lt;/a&gt;, a free tool I built: paste text (or Markdown), pick a template, and it assembles a finished Word document — cover page, styled headings, tables, even rendered diagrams — entirely inside your browser tab. Open DevTools while it works and watch the network panel: nothing is transmitted. There's no server to upload to; the "AI feature" is those 55 KB, cached with the page.&lt;/p&gt;

&lt;p&gt;The consequences are exactly the two from earlier, now running in reverse:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;It's free without a catch&lt;/strong&gt;, because a million users cost the same to serve as one: nothing. No accounts, no watermarks, no "3 documents per day."&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It's private by construction, not by promise.&lt;/strong&gt; No privacy policy can be violated for data that was never sent.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Back to the haunting question
&lt;/h2&gt;

&lt;p&gt;So — what &lt;em&gt;is&lt;/em&gt; a browser capable of? More than we let it do.&lt;/p&gt;

&lt;p&gt;We treat "AI feature" as a synonym for "cloud feature", but a huge share of everyday intelligence — is this a heading or a list? is this photo's subject in focus? which language is this receipt in? — consists of narrow questions, and narrow questions have small answers. Small enough to ship &lt;em&gt;with the page&lt;/em&gt;, like a stylesheet.&lt;/p&gt;

&lt;p&gt;Imagine that as the default: spellcheck-sized brains embedded in every text box, image tool, and file converter on the web — each one specialized, inspectable, offline, and free, because it costs nothing to run. The cathedrals will keep doing what only cathedrals can do. But most of what we actually need from software intelligence, most days, would fit in a parish chapel — and the parish chapel fits in a browser tab you already have open.&lt;/p&gt;

&lt;p&gt;The document-tools industry built paywalls on top of a cost structure that small brains simply delete. I don't think that industry has noticed yet. The power was on our devices all along; we just kept looking through the window instead of using the room.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://jigsawlabs.tools/blog/what-if-we-gave-the-browser-a-small-brain/" rel="noopener noreferrer"&gt;JigsawLabs Blog&lt;/a&gt;. DocForge is free, has no signup, and runs entirely in your browser: &lt;a href="https://jigsawlabs.tools/tools/docforge/" rel="noopener noreferrer"&gt;jigsawlabs.tools/tools/docforge&lt;/a&gt;. I build free, private, browser-based tools at &lt;a href="https://jigsawlabs.tools/" rel="noopener noreferrer"&gt;jigsawlabs.tools&lt;/a&gt; — if this post's ideas appeal to you, that's where they live.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>machinelearning</category>
      <category>privacy</category>
      <category>browsertools</category>
      <category>webdev</category>
    </item>
  </channel>
</rss>
