Filling a region by writing four bytes: region-id PNGs and an SkSL lookup shader

A paint-by-number canvas has an awkward shape as a rendering problem. The image is large. The number of independently colorable regions is large. Every tap recolors exactly one of them, and the result has to appear within a frame while a pinch-zoom gesture may be running.

The obvious implementations all fail in a specific, instructive way. What actually works is to stop treating "fill a region" as an image operation at all: the pixels never change. Only a tiny lookup table changes.

Why the obvious approaches fail

Flood fill on tap. Scan-line fill from the touch point across a full-resolution image is tens of milliseconds of JS work on a large canvas, on the JS thread, while the gesture is live. Worse, it has to be redone on every reload unless you also persist the filled bitmap.

One vector path per region. Cleanest conceptually — SVG paths, hit-test, set fill. It collapses on two fronts: the path data for a detailed illustration is far heavier than a compressed PNG, and hit-testing many complex paths per tap is not cheap either.

One texture per region, or a full-canvas repaint. Either you allocate an absurd amount of GPU memory, or you rebuild a large texture per tap and upload it. Uploading a multi-megabyte texture on every tap is exactly the thing you cannot do sixty times a minute.

The common mistake in all three: they treat the region geometry as mutable. It is not. Geometry is fixed at authoring time. The only thing that changes when the user taps is which color a region maps to — a value with a few bits of entropy.

The representation

Region geometry is baked once into a PNG where the pixel channels encode a region id rather than a color:

R = id >> 8   (high byte)
G = id & 255  (low byte)
B = 255       marks an outline pixel
A = 255
id = 0        outline / background

so id = (R << 8) | G. Decoding on the CPU is a single linear pass:

export function decodeIdMap(pixels: Uint8Array, w: number, h: number): Uint16Array {
  const n = w * h;
  const ids = new Uint16Array(n);
  for (let i = 0, p = 0; i < n; i++, p += 4) {
    // outline pixels (B=255) encode id 0 even if R/G carry values
    if (pixels[p + 2] === 255) { ids[i] = 0; continue; }
    ids[i] = (pixels[p] << 8) | pixels[p + 1];
  }
  return ids;
}

Fill state is one byte per region — colorIndex + 1, with 0 meaning unfilled:

export function applyFill(state: Uint8Array, regionId: number, colorIndex: number): boolean {
  if (regionId < 1 || regionId >= state.length) return false;
  if (state[regionId] !== 0) return false;
  state[regionId] = colorIndex + 1;
  return true;
}

And the bridge between the two is a palette LUT: an (regionCount + 1) × 1 RGBA texture where texel id holds the current display color of region id.

So filling a region is: write one byte of state, rebuild a one-pixel-tall texture, upload it. The large id texture is never touched, never re-scanned, never re-uploaded. That is the whole trick.

The shader

The renderer is a single Skia RuntimeEffect fed two image shaders — the id texture and the LUT:

uniform shader idTex;
uniform shader lutTex;
uniform float showEdge;
uniform float hlMode;
uniform float time;

half4 main(float2 xy) {
  half4 c = idTex.eval(xy);
  if (c.b > 0.5) {                       // outline marker
    return showEdge > 0.5 ? half4(0.11, 0.11, 0.12, 1.0) : half4(1.0, 1.0, 1.0, 1.0);
  }
  float id = floor(c.r * 255.0 + 0.5) * 256.0 + floor(c.g * 255.0 + 0.5);
  if (id < 0.5) return half4(1.0, 1.0, 1.0, 1.0);
  half4 L = lutTex.eval(float2(id + 0.5, 0.5));
  float flag = floor(L.a * 255.0 + 0.5);
  if (flag > 253.5) {                    // 254 = unfilled base, 255 = filled
    return half4(L.rgb, 1.0);
  }
  // flag == 253: unfilled region belonging to the active color index
  if (hlMode < 0.5) {
    return half4(0.80, 0.80, 0.83, 1.0);
  } else if (hlMode < 1.5) {
    float cs = 22.0;
    float ch = mod(floor(xy.x / cs) + floor(xy.y / cs), 2.0);
    half3 col = ch < 0.5 ? half3(0.87, 0.87, 0.90) : half3(0.73, 0.73, 0.78);
    return half4(col, 1.0);
  }
  float pulse = 0.5 + 0.5 * sin(time * 4.0);
  half3 tint = mix(half3(0.90, 0.90, 0.92), L.rgb, 0.30 * pulse);
  return half4(tint, 1.0);
}

Note the LUT's alpha channel is not alpha. It is a three-state tag: filled, unfilled, unfilled-and-belongs-to-the-currently-selected-color. Highlighting the regions the user should be tapping next is therefore also just a LUT rebuild — no second texture, no second pass, no per-region draw calls. The CPU side that produces the tag:

export const LUT_FLAG_FILLED = 255;
export const LUT_FLAG_UNFILLED = 254;
export const LUT_FLAG_HIGHLIGHT = 253;

buildRenderLut walks ids once and writes RGB plus one of those three flags per texel. That function is pure — no React, no Skia, no storage — so the entire visual state machine is unit-testable without a renderer.

Hit testing, with intent correction

The CPU keeps the decoded Uint16Array, so a tap is an array index. But a raw index is a bad user experience: fingers are wide, regions are small, and a one-pixel miss lands on a neighbor. So the pick does a histogram inside a radius and prefers eligible regions:

for (let y = y0; y <= y1; y++) {
  const dy = y - cy;
  const row = y * w;
  for (let x = x0; x <= x1; x++) {
    const dx = x - cx;
    if (dx * dx + dy * dy > r2) continue;
    const id = ids[row + x];
    if (id === 0) continue;
    counts.set(id, (counts.get(id) ?? 0) + 1);
  }
}

The caller decides what "eligible" means, and it is stricter than "unfilled":

const eligible = (id: number) => {
  if (fsIsFilled(st.fills, id)) return true;  // already filled = not selectable
  const r = st.meta!.regions[id - 1];
  return !r || r.colorIndex !== active;       // wrong color number = not selectable
};
const radius = Math.max(1, Math.round(PICK_RADIUS_BASE / (s / (base.value || 1))));

Radius shrinks as the user zooms in, so correction is generous at fit scale and precise when they are working on detail. Tapping the wrong region simply returns false from the store and produces no haptic — a miss costs nothing.

Costs and boundaries

The color-space trap is the one that will cost you a day. The id texture must be uploaded with no color space attached. If any sRGB conversion touches it, the R and G channels — which are not colors, they are an integer — get rewritten, and ids decode to garbage. In this codebase the id PNG is decoded, read back as RGBA_8888 / Unpremul, and reconstructed with Skia.Image.MakeImage from raw bytes for exactly this reason.

Sampling must be nearest, mipmaps off. Linear filtering interpolates between adjacent ids and invents region ids that do not exist — typically producing a thin halo of a random unrelated region along every boundary. There is no way to detect this from the shader; you just get wrong colors at edges.

Byte order is a contract, and contracts get violated. An early asset generator wrote R = low, G = high; the shipping pipeline is the opposite. Nothing crashes when they disagree — ids simply map to unrelated regions and the picture fills in wrong. If you adopt this technique, write the byte order down in one place, and have the asset generator and the shader both quote it.

Hard ceiling of 65,535 regions. Two bytes. The B channel is spent on the outline flag, so there is no third byte to borrow without another convention.

Memory. The decoded id map is a Uint16Array of width × height — two bytes per pixel, resident for as long as the page is open, plus the id texture itself on the GPU. This buys O(1) hit testing but it is not free, and it is the main reason page teardown has to actively drop state.

Antialiasing is not available. Region ids cannot be blended, so boundaries are hard. The design absorbs this by baking a dark outline into the artwork (the B = 255 path in the shader) which visually hides the aliasing. On an illustration without outlines this technique would look noticeably crunchy.

LUT rebuild is O(regions), not O(1). Each tap rebuilds the whole LUT and allocates a new SkImage. It is small and fast, but it is a per-tap allocation, and at very high region counts you would want a partial texture update instead.

Debug hooks must be __DEV__-gated. The canvas here reads a command file to script fills and zoom levels for screenshot automation. In Release __DEV__ is false and the block compiles out — which is correct, and also means any test relying on it silently does nothing against a release build.

This is the rendering path used by Numbrush.