What “local OCR” should mean in a web application

“Runs in your browser” is easy to put on a landing page. It is harder to make it a property of the whole OCR pipeline.

An OCR page can have a static interface and still send the selected file to an API. It can keep the image local but put the extracted text into error reporting. It can avoid both and still leak a protected PDF password through a URL or form submission. If privacy is part of the product, the useful question is not where the JavaScript was downloaded from. It is: which user-derived bytes can cross a process or network boundary, and why?

Here is the approach I use for a browser-only OCR tool.

Start with a small threat model

For an image or PDF OCR session, treat all of these as sensitive:

• source bytes;
• filenames and MIME metadata;
• PDF passwords;
• rendered pages and crops;
• recognized text and table output;
• parser and model errors that could contain user-derived values.

The default rule is that none of them may be placed in fetch, forms, beacons, analytics payloads, URLs, browser storage, or logs. Downloading application code, OCR models, and a public character dictionary is different: those resources are the same for every user and contain no document data.

That distinction makes the network policy testable. A test can select a uniquely marked fixture, perform OCR, and fail if the marker or source bytes appear in any outgoing request.

Move inference off the main thread

OCR is both CPU- and memory-heavy. Running it on the UI thread creates frozen progress indicators and tempts developers to move the whole job to a server.

A dedicated Web Worker gives a better local architecture:

1. The page decodes a selected file into an ImageBitmap.
2. Ownership of the bitmap is transferred to the worker.
3. The worker loads ONNX Runtime Web and the selected detection and recognition models.
4. It reports coarse progress states such as model loading, detection, and recognition.
5. It returns structured lines and then closes the bitmap.

Transferable objects matter here. Copying a large RGBA buffer between the main thread and a worker can briefly double its memory cost. Transferring an ImageBitmap avoids that copy, while explicitly closing it keeps repeated batches from accumulating GPU-backed resources.

Put bounds before decoding

File size is not enough to predict decode cost. A compressed image can expand into tens or hundreds of megabytes of pixels. A practical pipeline checks:

• accepted MIME types and extensions;
• compressed byte size;
• decoded width, height, and pixel count;
• a maximum working dimension for inference;
• a separate hard ceiling for safe decoding.

For formats with no universal browser decoder, isolate the decoder. HEIC can run in a short-lived worker; TIFF and BMP can have explicit dimension and buffer checks. After decoding, constrain the bitmap before inference. The OCR detector rarely needs a full 12-megapixel phone photo to find lines of text.

The important failure behavior is deterministic: reject an unsafe image before allocating another full-size buffer, return a localized error, and release anything already created.

Separate text detection from recognition

A useful local OCR pipeline is not one giant black box. Mine has two model stages:

• a detector produces a text probability map;
• connected regions become candidate boxes and nearby boxes are merged;
• each crop is deskewed into a normalized canvas;
• a recognition model produces character probabilities;
• CTC decoding turns those probabilities into text and confidence values.

The detector can operate on a bounded, resized image. Recognition still maps boxes back to the source bitmap, so each line gets a cleaner crop. This also makes table-oriented output possible: the UI retains text plus geometry instead of receiving one opaque string.

Two recognition models are a reasonable product tradeoff. A small model reduces first-use download and latency; a larger model can be loaded only when the user asks for higher accuracy. “High accuracy” should describe the relative model choice, not promise perfect OCR.

PDFs need their own boundary

A PDF workflow is not just an image workflow with a different file extension. It introduces page ranges, passwords, malformed structures, and potentially hundreds of render operations.

Render only selected pages, cap the document size and page count, and keep the password in memory. Each page should be converted to a bounded bitmap and passed through the same worker pipeline. Release page canvases as soon as their OCR result is complete.

If a user can pause or stop a batch, make that a real state transition. It should stop scheduling new pages, not merely hide a spinner while work continues in the background.

Be precise about offline support

“Works offline” does not mean the first visit works without a network. The application shell and a chosen OCR model have to be downloaded first.

A service worker can cache immutable application resources and models, but model versions need explicit names. When a model changes, old and new caches should not be confused. The UI should say that models load on demand and that offline reuse begins after caching.

Test offline behavior after a successful warm-up, then reload with the network disabled. Also test a model the user has not cached: that failure should be clear rather than silently switching to an unrelated model.

Translate operational text, not just the heading

OCR interfaces contain many strings that appear only during work: decode failures, password prompts, page progress, model downloads, pause states, empty results, and export errors. A translated landing page with English worker errors is not a translated product.

One approach is to keep worker errors as stable codes and translate them in the UI. If a worker must produce fallback text, verify every public error path in every locale. Missing messages should fail the build rather than fall back silently on an indexable page.

Tests that support the privacy claim

The highest-value tests are not snapshots of the hero section. I keep checks for:

• recognized text from a repository-authored public fixture;
• Fast and higher-accuracy model selection;
• a PDF page range and a password-protected PDF;
• stop, continue, clear, and batch export states;
• oversized and malformed input;
• zero user-derived network requests;
• offline reload after model caching;
• memory cleanup after replacing or clearing files;
• the same workflow in every interface language.

No browser test proves that all future code will be private. It does make a privacy regression visible in the same place as a broken download button.

The reference implementation behind these notes is BrowserOCR. The useful idea is broader than one product: local processing is an end-to-end constraint, not a badge attached to a client-side upload form.