Short answer: inspect metadata at upload, normalize orientation into pixels, and postpone destructive cropping until you have a reason to crop. For a scanned receipt expense app, that policy keeps OCR predictable while preserving an original that can be audited later.
A one-person SaaS lives on revenue per hour. Every image branch I do not have to debug is an hour I can spend shipping the next weekly release. The tempting shortcut is to crop and rotate in the phone client, then upload the result. It saves a request, but it also hides how the pixels were produced. A server-side intake record gives me a repeatable decision instead.
| Choice | Good fit | Cost or risk |
|---|---|---|
| Normalize on upload | OCR, search, duplicate detection | More CPU and storage work up front |
| Normalize on demand | Rarely viewed archives, cheap ingestion | First viewer or OCR job pays the latency |
| Crop before storage | Strict privacy or fixed document frame | Loses context and makes reprocessing harder |
| Keep original, derive crop | Audit trail, changing OCR models | Extra object and lifecycle rules |
For receipts, I choose upload-time metadata inspection and orientation normalization, then retain the source and a derived working image. Crop is a policy decision, not a default image operation.
What should receipt capture inspect before rotation and crop?
Start with the bytes, not the filename. A camera can label a JPEG as .jpg while the content has a different media type, and a renamed HEIC file is still HEIC. Decode the image with a library that validates the format, dimensions, and pixel limits. MDN's media format guidance is a useful baseline for understanding what browsers can actually decode.
Metadata inspection has two jobs. EXIF orientation tells you how a viewer should display the sensor pixels; it does not necessarily change those pixels. EXIF also may contain GPS, device, and timestamp fields that an expense system does not need. Copy only the fields required for audit, and strip the rest from derivatives. Keep a hash of the original bytes so a later crop can be tied back to the upload without retaining private metadata in every output.
I record width, height, MIME type, byte length, orientation, and a content hash in one intake row. A missing orientation is normal. Treat it as the default orientation rather than an error. A malformed segment, an impossible dimension, or a decompression limit breach is different: reject it with a useful client message and do not enqueue OCR.
How does a predictable rotation and crop pipeline work?
The pipeline should be idempotent. Given the same source hash and policy version, it should produce the same normalized pixels. That matters when a queue retries after a worker timeout. It also makes a bad OCR result diagnosable: you can compare the model input with the retained source instead of guessing which mobile SDK transformed it.
Here is the shape of the boundary in TypeScript. The decoder and encoder are deliberately generic; use a maintained implementation in your runtime.
type ImageMeta = { mime: string; width: number; height: number; orientation: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8; bytes: number };
type CropPolicy = { mode: "none" | "receipt-bounds"; paddingPx: number };
function chooseTransform(meta: ImageMeta, policy: CropPolicy) { return { rotatePixels: meta.orientation !== 1, cropPixels: policy.mode === "receipt-bounds", keepSource: true, policyVersion: "receipt-v2" }; }
A worker applies rotation first, then detects receipt bounds on the normalized image. If bounds confidence is low, it should produce an uncropped derivative and mark the job for review. Do not silently trim a hand, a second receipt, or a tax line. A conservative full-frame OCR pass is usually cheaper than a support ticket.
The output should carry a relationship, not a replacement: sourceHash, transformVersion, and cropBox belong beside the derivative object. Store the crop box in pixel coordinates after rotation. Normalized coordinates look portable, but rounding them differently across platforms creates one-pixel drift that is hard to reproduce.
Where do mobile photos fail in production?
The failures are ordinary. Portrait pixels plus an EXIF value of 6 can appear sideways to a decoder that ignores metadata. A very large panorama can pass a file-size limit and still exhaust memory while expanding. A browser preview can show a cropped canvas while the uploaded blob remains uncropped. Each case is a contract mismatch, not an OCR mystery. I don't ask the OCR vendor to explain that mismatch; I trace the bytes through each boundary. The intake event records the original hash, the decoder records the dimensions it accepted, and the transform event records the exact orientation and crop policy. When a receipt is later disputed, those three records answer a much narrower question than a screenshot ever could: which pixels did extraction actually receive, and why?
I use fixtures for every orientation value, two color profiles, transparent PNG input, and a receipt photographed against a busy table. The test asserts pixel dimensions after rotation, not only that a thumbnail looks right. It also verifies that the source hash stays stable when a new crop policy is deployed.
Observability needs the same vocabulary as the data model: intake rejection reason, decode milliseconds, normalized dimensions, crop confidence, queue age, and OCR completion. Keep ratios and latency histograms, not raw receipt images in logs. A request ID that follows the upload, transform, and OCR events is enough to trace a single failure.
Three words: preserve the source.
Measure twice.
When is on-demand processing the better trade?
Upload-time work is a poor fit for an app that accepts huge image bursts, has strict ingest latency objectives, or rarely runs OCR. In those cases, store a validated original and enqueue a low-priority transform when the user opens the expense or requests extraction. Set a bounded queue and show an explicit processing state; do not make the first screen wait on a full-resolution decode.
On-demand work is also useful when crop rules change often. You can replay derivatives from the source instead of asking users to upload again. The catch is operational: storage lifecycle, retention, and privacy controls now cover both the source and every derivative. If your team cannot monitor that queue, normalize on upload and keep the policy simple.
Your mileage may vary with device cameras and browser support. I am not sure a single crop detector will behave consistently on thermal-paper receipts, so I would gate automatic cropping behind a confidence threshold and measure manual corrections before tightening it.
A decision rule for a small expense product
Choose upload-time normalization when OCR is part of the critical path, input volume is predictable, and you can afford a short worker step. Choose on-demand processing when ingestion must stay fast, OCR is occasional, or crop policy is still experimental. In both cases, inspect metadata, validate decoded dimensions, and retain a source reference.
That is the boring architecture I want as a solo founder. It outsources undifferentiated image plumbing to maintained codecs and a queue, while the product code owns the receipt-specific policy and review experience. Ship weekly. Change one policy version at a time.
Top comments (0)