Routing Image Work Between Canvas and Sharp by Capability

Why this matters

"Browser first" sounds like one Boolean. A real image pipeline has at least four questions:

• Can the browser decode this input format?
• Can Canvas encode the requested output format?
• Does the product accept the browser encoder for this operation?
• What happens when an apparently supported operation fails at runtime?

If those decisions leak into UI components, batch loops, and API callers, the two processing paths drift. The interface may say an image stayed local while it actually uploaded, or a supported browser silently returns a different output format.

I tested a small capability router that keeps those questions in one place. Common work goes through Canvas. Sharp remains the compatibility path. The result records which engine actually produced each file.

What I built or tested

The browser adapter starts with two explicit sets:

const browserInputs = new Set([
  "image/jpeg",
  "image/png",
  "image/webp",
]);

const browserOutputs = new Set([
  "jpeg",
  "png",
  "webp",
]);

That excludes AVIF, HEIC, and HEIF inputs from the browser path. It also excludes AVIF output. These are capability decisions for this implementation, not claims about every browser.

The router then adds runtime and product constraints. It returns false during server-side rendering, probes WebP Canvas export, and keeps selected PNG compression operations on Sharp even though Canvas can emit PNG.

Setup

Define the routing input independently of either processor:

type ProcessingRequest = {
  file: File;
  outputFormat: "jpeg" | "png" | "webp" | "avif";
  toolSlug?: string;
};

The predicate needs no pixels. It only decides whether the browser path is worth attempting.

JPEG and PNG output are treated as supported once the runtime is a browser. WebP is probed with a one-pixel Canvas because some implementations can accept an output MIME argument yet return a different data URL. The probe verifies that the returned URL begins with the requested MIME type, then caches the Boolean.

This is capability detection rather than user-agent detection. The code asks the current runtime what it can produce instead of maintaining a browser-version table.

Step-by-step walkthrough

The decision order is intentionally cheap:

The predicate chooses an attempt; successful processing determines the recorded engine.

Reject non-browser runtimes

The same module can be imported during server rendering. Checking typeof window first prevents DOM capability code from running there. The server path is the safe default.

Check the input before the output

Canvas output support does not imply that the browser adapter can decode every accepted upload. An AVIF input requesting PNG therefore goes to Sharp even though PNG output is available.

Preserve product exceptions

The router keeps PNG compression on the server for three tool routes: the dedicated PNG compressor, the universal compressor producing PNG, and batch compression producing PNG.

This is the subtle part. A generic canCanvasEncode("png") check would say yes. The product has chosen a different implementation for compression behavior, so the exception belongs beside capability routing and needs a test of its own.

Probe uncertain exports once

For WebP, the adapter creates a tiny Canvas and calls toDataURL("image/webp"). It accepts the path only when the returned data URL declares WebP. The result is cached by output format, avoiding a probe for every file.

Fall back after a real browser failure

Passing the predicate does not guarantee that decoding, allocation, drawing, or encoding will succeed. The client orchestration therefore wraps the complete browser operation in try/catch. On failure it logs a warning and calls the server adapter with the original request.

This keeps the predicate simple. It does not try to predict available memory or every decoder edge case.

What went wrong

The first tempting model was "JPEG, PNG, or WebP means browser." The experiment disproved that shortcut in several ways.

A PNG input on the dedicated PNG compression route went to the server even though both its input and output appeared in the browser sets. An AVIF input requesting PNG also went to the server, but for a different reason: unsupported input decoding. JPEG requesting AVIF failed on output capability. Removing the browser runtime failed before any format test.

The cases look identical from the final Boolean, but they represent different constraints. That is why the routing predicate should remain readable rather than collapse into one opaque capability flag.

There is another limitation: the runtime fallback catches every browser-processing exception. A decode failure and a transient Canvas allocation failure both attempt the server. That is useful for compatibility, but it can add an upload after local work has already begun. The UI and telemetry must report the engine that completed, not merely the engine attempted first.

Fix or mitigation

Return an engine on the result:

type ProcessedResult = {
  processor: "browser" | "server";
  blob: Blob;
  width: number | null;
  height: number | null;
};

For a batch, reduce the recorded engines into browser, server, or mixed. A mixed label is more honest than assigning the whole batch to whichever path processed the first item.

Keep the server API behavior compatible with the browser request. It should accept the same quality, output-format, tool, and resize intent. Otherwise fallback changes semantics instead of only changing execution location.

Finally, make product exceptions named functions. A helper such as shouldKeepPngOnServer explains intent in review and gives future changes one obvious location.

Trade-offs

Browser processing avoids an upload for eligible files, but it consumes client memory and CPU. Large inputs can still fail after passing the format predicate. The server fallback improves completion rate at the cost of network transfer and server resources.

Canvas and Sharp also do not promise byte-identical output. The router aligns product intent and engine selection; it does not make two encoders equivalent. Verify format, dimensions, alpha handling, and other required properties rather than comparing hashes.

The WebP probe detects output support in the current runtime, but it does not prove acceptable quality for every image. A successful one-pixel export is a capability signal, not a visual benchmark.

Catch-all fallback can hide browser regressions if warnings are ignored. Count fallback reasons in observability, and fail tests when a route expected to stay local begins uploading.

How I verified it

I imported the production router into a Node 22 experiment with minimal window, document, and Canvas probe stubs. The assertions observed:

• JPEG to JPEG selected the browser;
• JPEG to a successfully probed WebP selected the browser;
• AVIF input selected the server;
• AVIF output selected the server;
• PNG compression selected the server; and
• removing window selected the server.

The production batch summarizer also returned mixed for one browser and one server result.

Repository end-to-end tests provide the next evidence layer. They download and decode a browser-produced JPEG-to-PNG result, a server-produced AVIF-to-PNG result, and both outputs from a mixed batch. That verifies the engines beyond the pure routing predicate.

Conclusion

A browser-first pipeline is safest when "first" means "attempted under an explicit contract," not "always used for familiar extensions."

List supported inputs and outputs, probe uncertain runtime behavior, preserve named product exceptions, fall back around the complete local operation, and record the engine that actually completed. The resulting capability table is small, testable, and far easier to evolve than format checks scattered across the interface.