DEV Community: Yan Wang

The adaptive threshold that deleted the thing we were looking for

Yan Wang — Sun, 13 Sep 2026 05:45:55 +0000

Finding the four corners of a page in a phone photo is the step that turns a picture of a document into a copy of one. Everything downstream — the perspective warp, the flattening, the PDF — is easy once you know where the page is.

We do that client-side, in a browser tab, on a grayscale copy of the photo scaled to a long edge of 640 px. The pipeline is the textbook one:

grayscale → blur → Sobel → adaptive edge map → Hough lines → best convex quad

Two near-horizontal lines, two near-vertical lines, score every candidate quadrilateral, keep the best. Disclosure up front: this is from LensUp, which is ours. What follows is the part where the textbook pipeline quietly failed, and the one-line change that nearly doubled how often it works.

The failure: the page fills the frame and detection gets worse

The adaptive edge threshold is the standard trick. Take the histogram of Sobel magnitudes, pick the 90th percentile, call everything above it an edge. It adapts to lighting, exposure and camera noise for free.

It also has a property nobody mentions: it is set by whatever has the strongest gradients in the image. On a photo of a document, that is not the page border. It is the letter strokes. Printed text is black-on-white at high spatial frequency — it produces the sharpest gradients in the frame by a wide margin.

The paper-to-desk step, meanwhile, is weak. Measured on SmartDoc 2015, a page lying on light wood clears the surrounding surface by roughly 14 luma levels. That is the signal you actually need, and it is an order of magnitude quieter than the text sitting in the middle of it.

So the adaptive threshold, doing exactly what it was designed to do, sets itself by the text and prices the page border out of the edge map. And it does this worst in the case you most want to work: when the user fills the frame with the page, so text dominates the histogram and there is barely any desk left to argue for a lower threshold.

The fix is not clever. Cap it:

edgePercentile: 0.9,     // gradient magnitudes above this percentile are edges …
minEdgeMagnitude: 14,    // … but never below this …
maxEdgeThreshold: 30,    // … and never above this: print must not price the page edge out

Measured on 300 real frames, with every other guard unchanged:

threshold cap	pages located
220	23.3 %
30	41.0 %

An adaptive threshold with a hard ceiling is no longer fully adaptive, which felt wrong when we wrote it. It is also the single highest-value line in the file.

Telling paper from a block of printed text

Capping the threshold gets the page border back into the edge map. It also lets a lot of rectangles in that are not pages — and the most dangerous one is a dark block of printed text, because it is genuinely a bright-surrounded rectangle with strong edges.

The first instinct is a contrast magnitude: require the inside of the quad to be meaningfully brighter than the outside. We had that set at 14 gray levels, which is the median paper-on-light-wood step from above.

That was the wrong knob. A threshold at the median of true pages rejects about half of the true pages. What actually discriminates is not how big the step is but which way it points:

minContrast: 8,   // 40th percentile of (inside − outside) across each side, in gray levels.
                  // A page on light wood measures 14 at the median, so 14 rejected half of them;
                  // the polarity rule is what keeps text blocks out, not this magnitude.

The polarity rule is the actual test: a quad is a page only if it is brighter inside than outside, sampled at two depths, on all four sides. A text block fails that on the sides where more text continues past it. A page on a desk passes it everywhere.

There is still a case that beats local evidence: a large dark region of print can look like "the desk" if you only sample a few pixels out. So before applying a detection automatically — as opposed to just pre-positioning the handles — we probe farther outside, looking for the page continuing past that edge. If the page continues, the edge was not the page edge.

The guards that are product decisions wearing algorithm clothes

Three of the constants in that file are not computer vision. They are decisions about what to do when we are unsure, and they are the ones I would port to any similar project:

borderMargin: 0.02,        // a side within 2% of the frame edge is "on the border"
applyMinAreaRatio: 0.25,   // a smaller page is only proposed, never applied
minSideSupport: 0.42,      // every side needs this much edge support to be applied

A quad with a side lying on the image border is rejected. If the "page edge" is the edge of the photo, the page is not fully inside the frame, and the honest default is the whole frame rather than a crop that silently cuts off whatever was outside it.

The largest well-supported quad wins. This sounds like a tie-breaker and is actually a correctness rule: it makes an inner rectangle — a photo printed on the page, a bordered table — lose to the page around it.

Below a quarter of the frame, a detection is only a proposal. It pre-positions the four draggable handles and waits. Auto-applying a small quad is how you produce the single worst outcome in this whole category: a confident crop that removes two thirds of somebody's passport.

That last one generalises. The result of this module is a proposal and never the truth; every handle stays draggable, and the caller decides from a confidence value whether to apply it silently. When detection fails outright it returns null, and null means keep the entire original image. A scanner that crops wrong is worse than a scanner that does not crop.

Two features that ship turned off

weakEdgeRecovery: 0,   // Experimental: desktop cost is not yet justified by coverage gains.
contourRecovery: 0,    // Correct proposals improved, but the measured latency gate failed.

Both of these work. Both improve the numbers. Both are disabled in the shipped defaults because they did not clear a latency budget on the hardware people actually hold. Leaving them in the file at 0, with the reason written next to them, has been more useful than deleting them — the next person to look at this does not have to rediscover that the idea was tried and why it is not on.

What I would take away from this

The general shape of the bug is worth more than the specific fix: an adaptive parameter adapts to the strongest thing in your input, and the strongest thing is often not your signal. Text beats page borders. Specular highlights beat document edges. The loudest object in the frame sets your threshold, and if your target is quiet, it gets deleted.

When an adaptive method underperforms, it is worth checking whether it is adapting to something you did not intend before reaching for a better algorithm. In our case the better algorithm was a ceiling of 30.

If you want to watch this run on your own photo, it is at a browser-based document scanner — the detection happens in the tab and your files are never uploaded, so the Network panel is a fair way to check that claim.

Three things that bit us moving a document scanner fully into the browser

Yan Wang — Sun, 13 Sep 2026 03:55:14 +0000

Photographing a page and turning it into a clean, printable PDF is, on paper, a solved problem: read a file, fix the perspective on a canvas, flatten the shading, write a PDF. Browsers have shipped every primitive for that for years, and none of it needs a server.

I work on LensUp, which does exactly that — the whole pipeline runs in the tab, and files are never uploaded. Disclosure up front: it's our tool, and this post is about the parts that were harder than the pipeline itself. None of the three below are in the "how to draw an image to a canvas" tutorials, and all three cost us real debugging time.

1. Web Share can be gone by the time your file is ready

The Web Share API's file variant is the nicest way to hand a generated PDF to whatever the user actually wants to do with it. The naive version looks fine:

shareButton.addEventListener('click', async () => {
  const bytes = await buildPdf(pages);           // takes a while
  const file = new File([bytes], 'scan.pdf', { type: 'application/pdf' });
  await navigator.share({ files: [file] });      // 💥 NotAllowedError
});

navigator.share() requires transient user activation, and transient activation expires. Building a multi-page PDF from full-resolution photos on a mid-range phone takes long enough that by the time you call share(), the activation from the tap is gone. You get a NotAllowedError, the share sheet never opens, and it only reproduces on slow devices — which is the worst possible failure profile.

There is no way to extend the activation. What you can do is decouple "prepare" from "share", and check whether you still have activation before deciding which one you're doing:

let preparedShare = null;  // { file, identity }

shareButton.addEventListener('click', async () => {
  let file;
  if (preparedShare && sameOutputIdentity(preparedShare.identity, snapshotOutputIdentity())) {
    file = preparedShare.file;                   // second tap: no await before share()
  } else {
    preparedShare = null;
    const { pages, settings } = await prepareCurrentOutputPages('share');
    const bytes = await buildPdf(pages, settings);
    file = new File([bytes], `scan-${Date.now()}.pdf`, { type: 'application/pdf' });
    preparedShare = { file, identity: settings.identity };

    // A long render may outlive transient user activation. The next tap shares
    // the prepared file synchronously, only while its content/settings still match.
    if (navigator.userActivation?.isActive === false) {
      toast('Your file is ready — tap share again');
      return;
    }
  }
  await navigator.share({ files: [file], title, text });
  preparedShare = null;
});

navigator.userActivation.isActive is the part worth knowing about. It lets you tell the difference between "this will work" and "this will throw", so you can degrade to an honest two-tap flow instead of showing an error for something the user did nothing wrong in. On a fast device the second tap never happens; on a slow one the user gets a clear "ready, tap again" instead of a failure.

Two details that go with it.

Feature-detect the file variant, not the API. 'share' in navigator tells you nothing about whether files can be shared — that support is separate, and it varies. The only honest probe is to build a real File of the type you intend to send and ask:

const canShareFiles = (() => {
  try {
    const f = new File([new Uint8Array([37, 80, 68, 70])], 't.pdf', { type: 'application/pdf' });
    return !!(navigator.canShare && navigator.canShare({ files: [f] }));
  } catch {
    return false;
  }
})();
if (!canShareFiles) shareButton.hidden = true;

Those four bytes are %PDF. Building the probe file from the real MIME type matters, because canShare can accept one type and refuse another.

AbortError is not an error. When the user opens the share sheet and dismisses it, share() rejects with AbortError. If you surface that as a toast, you are telling people something failed when they simply changed their mind:

} catch (err) {
  if (!err || err.name !== 'AbortError') {
    showShareFailed(err);
  }
}

2. One worker per image beat a worker pool

Finding the page corners in a photo — the geometry that turns a trapezoid back into a rectangle — is the one genuinely CPU-heavy step, and it has no business on the main thread while someone is trying to scroll.

The obvious architecture is a long-lived worker (or a small pool) plus request IDs, so you can match a response to the request that asked for it. We ended up with the opposite: spawn a worker for one image, then terminate it.

export async function detectQuadFromBlob(blob, options = {}) {
  if (!blob || options.signal?.aborted) return null;

  // A worker owns only this image. Completion, cancellation and timeout all release
  // its bitmap/heap; older requests cannot deliver a later request's result.
  if (typeof window !== 'undefined'
      && typeof Worker === 'function'
      && typeof OffscreenCanvas === 'function') {
    let worker;
    try {
      worker = new Worker(new URL('../workers/quad-detect-worker.js', import.meta.url),
                          { type: 'module' });
    } catch {
      /* CSP / unsupported module worker: fall through to the local path. */
    }
    if (worker) return new Promise(resolve => {
      const { signal, ...settings } = options;
      let done = false;
      const finish = value => {
        if (done) return;
        done = true;
        clearTimeout(timer);
        signal?.removeEventListener('abort', abort);
        worker.terminate();
        resolve(value);
      };
      const abort = () => finish(null);
      const timer = setTimeout(abort, 10000);
      const fallback = () => { if (!done) finish(detectLocal(blob, options)); };

      worker.onmessage = e => finish(e.data?.detection ?? null);
      worker.onerror = fallback;
      worker.onmessageerror = fallback;
      signal?.addEventListener('abort', abort, { once: true });
      if (signal?.aborted) { abort(); return; }
      try { worker.postMessage({ blob, options: settings }); } catch { fallback(); }
    });
  }
  return detectLocal(blob, options);
}

Three reasons this turned out better for this particular job:

Stale results become structurally impossible. With a shared worker, a response from the image the user already replaced can arrive after you've moved on, and you are one forgotten ID comparison away from cropping photo B by photo A's corners. Terminating the worker deletes that bug class instead of guarding against it.

Memory releases deterministically. A decoded ImageBitmap from a 12-megapixel photo is tens of megabytes. terminate() takes the whole worker heap with it, which is a much shorter argument than reasoning about when the bitmap becomes unreachable inside a worker that keeps running. The worker side still closes it explicitly, because the tab may be doing several things at once:

self.onmessage = async ({ data }) => {
  let bitmap;
  try {
    bitmap = await createImageBitmap(data.blob);
    self.postMessage({ detection: detectQuadFromSource(bitmap, data.options) });
  } catch {
    // The importer keeps the full original image on every detection failure.
    self.postMessage({ detection: null });
  } finally {
    bitmap?.close();
  }
};

Cancellation is just terminate(). No cooperative abort checks inside the detection loop, no message protocol for "never mind".

The cost is real — you pay worker startup per image, and on a cold module worker that is not free. For a user importing a handful of pages, that cost is invisible; if you were detecting corners on a video stream at 30fps, you would want the pool and the request IDs.

Note what the failure path does: every error resolves to null, and null means keep the full original image. A scanner that crops wrong is worse than a scanner that doesn't crop, so the degraded state is "you get your whole photo" rather than "you get two thirds of your passport".

3. "Compress to 200 KB" is a search problem, not a setting

Government portals and university systems love a hard byte ceiling: PDF, under 200 KB, colour, A4. Developers see that requirement and go looking for the quality parameter that produces 200 KB.

There isn't one. The size of an encoded JPEG is a function of the image content as much as the quality setting — a dense page of small text and a mostly-white form at the same quality can differ several-fold. The only thing you can do is encode, measure, and step:

async function encodeTowardTarget(canvas, targetBytes) {
  let best = null;
  for (const q of [0.92, 0.85, 0.78, 0.7, 0.6, 0.5, 0.42, 0.35]) {
    const blob = await new Promise(r => canvas.toBlob(r, 'image/jpeg', q));
    best = blob;
    if (blob.size <= targetBytes) break;   // first one that fits wins
  }
  return best;                              // may still exceed the target
}

Two things follow from that, and both are product decisions rather than technical ones.

The loop has to terminate somewhere, which means the result can still be over the ceiling. You either keep degrading until the page is unreadable, or you stop and hand back something too big. We stop, which makes this a best-effort operation — and if you are building something similar, say that in your UI. Telling a user you will hit an exact byte count is a promise the format does not let you keep.

And if you are the one filling in the form: check the file, don't trust the label. A tool that claims an exact size is either lying or about to destroy your document's legibility.

The part that is genuinely easier client-side

For all three of the above, the reward is worth it. The documents people scan are passports, signed contracts, medical forms, payslips — the most sensitive paper most people own. Doing the work in the tab means the server only ever ships HTML, JavaScript and translations; it never receives a pixel of the document.

That claim is also checkable, which is the main thing I'd push for in this category: open DevTools → Network, clear the log, scan something, export it, and watch whether your file's bytes leave. If a "client-side" claim is real, the Network tab shows it. If it isn't, you'll see a POST with your document in it. Worth doing to any tool that touches your paperwork — including ours.

If you want to poke at the implementation described above, it's running at a browser-based document scanner.