DEV Community: tony huang

Local-first photo editing in the browser: what the pipeline split actually looks like

tony huang — Mon, 19 Oct 2026 19:08:19 +0000

If you build anything in the browser that touches user photos, you eventually run into the same wall: the cheap operations want to be local, and the expensive ones cannot be.

Here is how I ended up splitting the pipeline, and the constraints that forced each decision.

The cost asymmetry nobody mentions in the docs

A brightness slider is a loop over pixels. A crop is a coordinate transform. A JPEG re-encode is a codec call. None of these need a server, and if you put them behind one you have added latency, bandwidth and a privacy problem to something that costs you nothing to run locally.

A diffusion step is different. It needs a GPU, it costs real money per call, and there is no version of it that runs acceptably in a browser tab today. That is not a temporary limitation of my code — it is the state of the hardware.

So the pipeline naturally splits: everything deterministic runs in the client, everything generative runs on a server, and the boundary between them is the honest place to draw your pricing model.

What the local layer actually costs

Canvas-based operations are fast until they are not. The three things that bit me:

1. getImageData is a synchronous trap. Every pixel-level pass that reads back from the canvas stalls the main thread while the GPU drains. On a 4000×3000 photo that is a visible freeze. The fix is to do the pixel work once per adjustment change, not once per frame, and to debounce slider input — otherwise you are running the full loop on every mousemove event.

2. Re-encoding is the real cost, not the filter. Applying a filter is milliseconds; encoding the result to JPEG at quality 0.9 to show a preview is where the time goes. Previewing at display resolution and only re-encoding at full resolution on export makes the interaction feel instant.

3. Memory is worse than you think. An RGBA buffer for a 12-megapixel photo is 48 MB. Keep a source copy, a working copy and an undo stack and you are at a few hundred megabytes on a phone. Mobile Safari will kill the tab without ceremony. Downscaling the working copy to the viewport size and keeping the original as a compressed blob is the only thing that survived real testing.

None of that is exotic, but it means "local-first" is an architectural commitment rather than a checkbox.

The network panel is the honest test

The nice side effect of this split is that it is verifiable from outside. Open DevTools, switch to the Network tab, and:

Apply a tone adjustment. Requests appearing there means the claim is false.
Use the generative feature. Requests are expected — that part really does cost money per call.
Export. If the only difference between the free and paid path is this step, you have found the business gate, which is a separate thing from a technical limit.

A tool that passes the first check and is open about the second is telling you the truth about its own cost structure. That is a better signal than any badge.

The privacy argument is a side effect

I did not start with a privacy pitch. But once the local layer exists, the useful consequence follows: photos that never leave the device are photos people are willing to use. Scans of documents, pictures of kids, product shots under NDA — the category that gets excluded from every cloud editor by default becomes available.

That is a real product difference, and it comes from the architecture, not from a policy page.

Where the split breaks

Two honest limits, because anyone claiming otherwise is selling something.

WebAssembly models are not a shortcut. You can run a small model in the browser, and people do. But it is slow enough to be unusable for anything interactive on a laptop, and it drains the battery doing it. For the operations people actually want (restoration, background removal, upscaling, style generation) the server is the only option that produces good output in reasonable time.

The boundary moves. Every year a model that needed a GPU gets smaller. The split I have today will be wrong in a couple of years, and rebuilding around a new boundary is cheaper than pretending the old one is permanent.

I build aiphotoeditor.best along these lines: the local layer (tone, filters, crop, resize, format conversion, compression) is free and unlimited because it costs nothing to run; the AI layer is capped at three free edits a day and then one-time credit packs, because each call costs money. If you want to check the split for yourself, the network panel test above works on it too.

What it costs to run an AI baby-photo generator: ~4¢ an image, and an honest promise

tony huang — Thu, 15 Oct 2026 15:00:10 +0000

I build small browser-first tools. One of them, OurBaby, answers the oldest question couples ask — what will our baby look like? — by blending one photo of each parent into an imagined baby portrait.

Two questions come up from other developers every time I describe it, so here are both answers with real numbers.

The unit economics (real numbers, October 2026)

Generation runs on a multi-image model of the "nano-banana" class via fal.ai. The actual billed cost per image, from my fal usage export:

$0.0398 per generated image (identity-blending edit, two reference photos)
Same engine for every scene — studio, Christmas, beach, family-of-three

So the math is simple: a $2.99 pack of 5 photos is ~80%+ gross margin, and the free tier is a bounded acquisition cost rather than a leak. The free tier is deliberately small and capped three ways:

1 free preview per visitor IP per day
a per-site daily spend circuit breaker (cost-estimate accumulation in KV)
no API key in the deployment → the endpoint returns a waitlist 503 instead of ever going bare

That last one matters. An earlier project of mine shipped an AI endpoint without a daily cap and someone found it within hours — the cap is the difference between a marketing cost and an incident.

The pipeline, in one paragraph

Photos never touch my servers as files: the browser downscales each to a 768px JPEG, sends them as data URIs, and a Cloudflare Pages Function makes a single queued call to the model with both references. The interesting part is not the plumbing, it's the prompt: the model is told to inherit specific features (eye color and shape, nose, smile, hair color and texture, skin tone) from both references, rather than averaging two faces — averaging is why first-generation "baby generators" produced children that looked like nobody.

The part every "baby predictor" gets wrong

Here's where I have to argue against my own product category. A baby's face cannot be predicted — by my tool or any other. Appearance is polygenic: eye color alone involves several genes (the OCA2/HERC2 region does most of the work, not all of it), and each parent passes on one of two copies of every gene. The recombination is random. Even two full genomes don't compute a face.

What a good generator can honestly claim is fidelity to the parents — the imagined child should visibly inherit from both of you — not prediction of the child. So the copy on the site says exactly that: the result is a fictional, imagined person, and the page even explains why prediction is impossible. It turns out that honesty is also good SEO: it's the question people actually type.

Privacy structure, since your users' faces are your users

Photos go encrypted to the model provider only to generate the image
No gallery, no public results wall — if you show other people's uploads, assume yours joins them
The generated images live on the provider's temporary storage, not in my database

If you're building anything that eats faces (headshots, restorations, family photos), I'd treat this as the baseline, not a feature.

What I'd tell you before you build in this space

Bound your free tier with per-IP and global daily caps on day one
Label outputs as fictional — it's the difference between a keepsake and a scam
Put the honest limit in the title of your main content page; being straight about what AI can't do outperforms promising predictions

If you want to see the result, the generator is at ourbaby.shop — first preview free, no signup, and the free tier is documented in plain numbers.

AI Photo Restoration Keeps Changing Faces — Here's the Pipeline That Stops It

tony huang — Wed, 14 Oct 2026 15:37:54 +0000

If you've run old family photos through an AI restorer, you've probably hit the failure mode nobody advertises: it changes faces. Heavy restoration on a low-resolution scan doesn't recover a face — it re-draws a plausible one. You end up with a stranger wearing your grandmother's expression.

Why faces drift

Face restoration models are trained to produce plausible faces. With a high-signal source, "plausible" and "correct" converge. Below a certain signal level — small scan, compression, damage sitting on the features — the model stops restoring and starts re-drawing. It's a fidelity-versus-strength trade-off baked into every tool of this class.

The pipeline

Scan at 600 dpi, in color — color even for black-and-white originals, because the stain and tone information helps separate damage from image. A flat daylight photo of the print beats nothing, but a real scan beats everything.
Repair damage first, enhance faces second, upscale third — separate passes, not one mega-pass. Scratches and fading act like added noise, and face restoration running on top of damage is where the drift starts.
Bias toward fidelity, not strength — the "beautiful" results that swap identity are usually high strength on low-res input.
Verify at 100% zoom on eyes, smile, hairline — if any of the three changed, reject the pass and re-run with less strength. That's the identity check.
Colorize last — colorization is a guess about the past, not a record of it. Keep the black-and-white original as the archival master.

The hard ceiling

Past roughly half of the facial detail missing, no model can recover identity — anything it produces is invention. That's manual-restoration territory. Below that threshold, the order of operations above is the difference between a sharper version of the same person and a stranger.

What I built with this

I packaged this workflow into AI Photo Editor: free local adjustments (crop, tone, contrast — nothing uploaded) plus a few free AI edits a day for enhance / restore / background removal. The local layer handles the prep steps without spending an AI credit, and the AI pass only ever sees the cleaned scan.

Honest limits: it won't fix a photo beyond the reconstruction threshold above, and keeping the period look is the goal by design — if you want a "modernized" face, this is the wrong tool.

Building browser-only tools that never upload your files (the local-first web stack)

tony huang — Wed, 30 Sep 2026 06:09:06 +0000

Most online tools quietly ship your files to a server: your photos, your invoices, your subtitle files, your client documents. There is a different way — build the whole tool client-side, so nothing ever leaves the device. I've been shipping a family of free tools built on this principle, and this post is the practical stack behind them.

The core pattern

Static site export. The app is a Next.js static export on Cloudflare Pages — there is no backend to receive your files in the first place. Privacy is structural, not a policy promise.
Canvas API for anything pixel-shaped: downscaling, palette quantisation, crops.
WebAssembly for codecs that never shipped in browsers: HEIC decoding, WebP/AVIF encoding, GIF optimisation.
WebCodecs + seeking for video work without ffmpeg.
Plain text processing where "format" just means text with rules.
Web Workers so the main thread keeps breathing during heavy transforms.

What this looks like in practice

Image format conversion — wasm builds of libheif/libwebp mean you can convert HEIC or WebP entirely locally, with batch support and no file-size ceiling. Live example: ImageConverter.
GIF compression — scale, colour-reduce and drop frames in a worker. Live example: GifCompressor.
Video frames — the browser can decode and seek video natively now: pull evenly spaced frames with timestamps, grab one exact frame, or scan a long clip for motion. No ffmpeg install, nothing uploaded. Live example: FrameExtractor.
Pixel art conversion — canvas downscale + palette quantisation turns photos into game-ready sprites. Live example: PixelArt.
Subtitle repair — SRT/VTT/ASS are just text with rules, so parsing, resyncing and merging run instantly client-side. Live example: SubToolbox.
Calculators — pure JavaScript, zero I/O: aspect ratios (https://aspectratio.site), marketplace seller fees (https://sellerfees.org), construction materials (https://buildcalcs.best), electrical/HVAC tables (https://tradecalcs.best), even baby wake windows (https://wakewindows.best).

The honest trade-offs

Client-side is not free of constraints. Large files on low-RAM phones will hurt; you're bounded by the device instead of a server farm. Heavy AI inference is still mostly a server story (though small models in WASM are getting real). And some features — accounts, sync, sharing — reintroduce a backend by definition.

But for a huge category of "give me a file, give me a different file back" problems, the browser is now enough. The user gets instant results, offline capability, and privacy that isn't just a checkbox. If you're building tools in 2026, it's worth asking first: could this run entirely on the client? For us, the answer is usually yes — and it makes the product better, not just cheaper.

All the tools linked above are free, require no sign-up, and run 100% in your browser. Feedback welcome in the comments.