A photo of a dog can become two very different coloring pages. One keeps the fur, fabric and small edges close to the source. The other redraws the dog with broader outlines and more room for crayons.
While building ColorWillow, a printable coloring site for parents and teachers, we kept both approaches: a free conversion running on the user's device, and an AI service that can simplify and restyle a photo.
The interesting engineering work was deciding what each path should promise, how much work belongs in the browser, and what happens when a remote generation takes longer than the page stays open.
The same photo reveals the tradeoff
Start with this dog resting in a person's arms.
Photo by Kristin Hardwick on StockSnap, CC0 1.0. Resized for this example.
Free conversion. Fine fur and the checked shirt remain visible as many small lines.
AI conversion with Classic style, Medium detail, Standard lines and Keep background. It leaves larger white areas, but changes the dog's facial features and coat markings.
That last point matters. A cleaner illustration can be easier to color while being less faithful to the individual animal. We show both methods so people can choose based on their own photo.
Running a line drawing model in the browser
Our free photo to coloring page converter uses an ONNX line drawing model executed through ONNX Runtime Web's WASM backend.
The local model is derived from Informative Drawings by Caroline Chan, Frédo Durand and Phillip Isola. We retain the model and runtime license notices with the distributed assets.
Although the interface calls this mode Free and the remote mode AI, the local path also uses machine learning. The distinction is where the processing happens and how much the picture is reinterpreted.
The pipeline is:
- Decode the selected image and fit it within a bounded working size.
- Composite transparency onto white.
- Convert RGBA pixels into a normalized RGB tensor.
- Run the model in a dedicated Web Worker.
- Turn its single channel output into a grayscale PNG.
Here is the session setup, shortened from our worker:
import * as ort from 'onnxruntime-web/wasm';
// This code already runs inside our own Web Worker.
ort.env.wasm.numThreads = 1;
ort.env.wasm.proxy = false;
ort.env.wasm.wasmPaths = assetPrefix;
const session = await ort.InferenceSession.create(modelBytes, {
executionProviders: ['wasm'],
graphOptimizationLevel: 'all',
enableCpuMemArena: true,
enableMemPattern: false,
});
We use one inference thread in this implementation. The ONNX Runtime configuration guide explains the additional cross origin isolation requirement for WASM multithreading. Running inside our own worker lets us control cancellation and its lifetime directly; we do not enable the runtime's additional proxy worker.
The model file is about 8.6 MB, excluding the runtime. We fetch it from a versioned asset directory and verify its expected byte length and SHA-256 before creating a session. The JavaScript runtime and WASM binaries must also come from the same build, as the ONNX documentation specifies.
The first conversion includes downloads and initialization. WASM moves inference onto the device, but loading the page and its assets still needs network access.
Preprocessing and memory need explicit limits
A successful session initialization does not guarantee a useful image. The model expects RGB channel planes, while canvas gives us interleaved RGBA pixels.
Our conversion code preserves the aspect ratio, pads the working dimensions to multiples of four with a minimum of 32, and uses reflected pixels at the boundary. After inference, it removes that padding.
The channel conversion looks like this, with padding omitted for clarity:
const plane = width * height;
const data = new Float32Array(3 * plane);
for (let i = 0; i < plane; i++) {
data[i] = rgba[i * 4] / 255;
data[plane + i] = rgba[i * 4 + 1] / 255;
data[2 * plane + i] = rgba[i * 4 + 2] / 255;
}
const input = new ort.Tensor('float32', data, [
1, 3, height, width,
]);
The current device policy uses a 512 pixel maximum working edge on phones and tablets, and 1024 on desktop. These are inference dimensions; they do not imply that every source photo gains printable detail at that resolution.
Memory use includes decoded pixels, tensors, intermediate activations, the model and PNG encoding. We close image bitmaps, dispose tensors and shrink temporary canvases when their work finishes.
We also separate line thickness adjustments from inference. A small grayscale buffer lets us strengthen neighboring dark pixels without running the model again. On phones and tablets we terminate the worker after a result; on desktop we allow a short idle window before releasing it. The grayscale buffer can survive that worker teardown so an adjustment does not need to reload the model.
Free conversion does not upload the selected photo. Choosing to save its result to an account is a separate upload action. The remote AI path has a different data flow, so the interface explains it separately.
AI gives more control over the drawing
The AI photo to coloring page converter can request simpler details, a simplified background, different line weights and illustration styles.
These are structured product options. The server validates them and builds the generation instructions; it does not accept an arbitrary prompt from the browser.
That gives each control a specific job. Simple detail asks for larger coloring areas. Bold lines strengthens contours. Simplifying a background should preserve the main subjects. A style choice should keep the output usable as black outlines on white paper.
Those instructions express intent, not a guarantee of exact preservation. In the dog example, the AI reduced texture but changed recognizable details. Comparing the result with the original remains part of using the tool.
Remote generation needs a durable job
The browser path can finish with a local PNG. Remote generation introduces an upload, an external provider, private storage and credits.
Our backend runs on Cloudflare Workers. A Durable Object with SQLite storage coordinates generation state and credit reservations, while private R2 storage holds originals and results.
The job progresses through queued, dispatching, uncertain or pending output states, and finally a terminal result. An uncertain provider response stays unresolved while the system checks what happened. A browser timeout alone does not prove that generation failed.
Three details make this manageable:
- Repeated submissions have an identity. Reusing a job ID with the same input and options returns the existing job. Reusing it with different parameters is rejected.
- Processing uses a lease. A worker claims the job temporarily, and state changes check its lease token so an older attempt cannot complete over a newer claim.
- Completion follows storage. The runner validates and normalizes the returned image, records output metadata, writes it to private storage and confirms the stored object before completing the job and consuming the credit reservation. Confirmed failures release the reservation.
The output write and the database transaction are separate operations. Keeping an intermediate output state gives recovery code something concrete to reconcile if execution stops between them.
Private images also need authorization at retrieval time. We keep them separate from the public coloring library and check the requesting owner's access before serving them.
Choosing the path by the result
For this product, local inference is useful when someone wants a free outline that stays close to the photo, with processing on their device. Remote AI is useful when they want broader coloring areas or a different illustration style and accept that details may change.
Both paths end with a practical task: inspect the picture, download it or print it, and color it on paper.
The wider ColorWillow library of printable coloring pages is available when a ready made illustration suits the activity better. Photo conversion adds another starting point: a familiar pet, object or moment that someone wants to turn into a page of their own.
Top comments (0)