Canvas getImageData RGB values are quantized with premultiplied-alpha

yoya — Wed, 21 Apr 2021 14:01:03 +0000

When RGBA data including transparent colors is obtained from Canvas using getImageData, the RGB values may differ from the imagine values. The behavior described in this entry is not a bug, but conforms to the specification.

sample image

% magick -size 256x256 xc:white -alpha on \
         -channel RGB -fx "i/w"           \
         -channel A   -fx "j/h"   RGBA.png

RGBA.png	RGB.png	A.png

problem

If we use drawImage to draw an image file from this image file onto the Canvas and getImageData, we get the following data.
Since RGBA does not change in appearance, the RGB separated from it.

% magick RGBA-canvas.png -alpha off RGB-canvas.png

RGBA-canvas.png	RGB-canvas.png

As you can see, the upper part of the image, where the A value is small, has collapsed RGB values.
To make it easier to see, let's zoom in vertically at x8.

RGB-canvas-x8.png

A:0 RGB: all 0
A:1 RGB: (div 2) 0 or 255
A:2 RGB: (div 3) 0, 128(0x80), 255(0xFF)
A:3 RGB: (div 4) 0, 85(0x55), 170(0xAA), 255(0xFF)
...

The smaller A is, the more coarse the RGB values will be. This phenomenon of jumping values is also called quantization.

why

This is because Canvas keeps the ImageData RGBA as premultiplied-alpha.

GPUs prefer premultiplication
- http://www.realtimerendering.com/blog/gpus-prefer-premultiplication/

To explain premultiplied-alpha briefly, it multiplies alpha by A, so if you multiply RGB by A in advance, you can skip the multiplication. It is also widely used for speeding up the compositing process.

RGBA(straight alpha): 255, 128, 64, 128

the RGBA values above, multiply RGB by 128(/255) to get premultiplied.

RGBA(premultiplied alpha): 128, 64, 32, 128

Since getImageData restores the RGB values based on the RGB values that have been divided by A, A:0 means that all RGB values are 0, and A:1 means that only two RGB values, 0 and 255, can be restored.

work around

There is no workaround if you use getImageData to retrieve RGBA that has been put into Canvas.
If you want to get the transparency of an image file accurately, you can use a decoder such as PNG.js, or if you import PNG32 using a WebGL2 texture without using the 2d context of Canvas, you can get ImageData with RGBA intact.

How can I stop the alpha-premultiplication with canvas imageData?
- https://stackoverflow.com/questions/23497925/how-can-i-stop-the-alpha-premultiplication-with-canvas-imagedata

notes

There seems to be a request to change the specification because the values are degraded and inconvenient. However, I think it's probably not possible to do so, because it would mean either throwing away the advantages of premultiplied and making it slower, or keeping two RGBAs in memory, one for straight and one for premultiplied.

ImageData alpha premultiplication #5365
- https://github.com/whatwg/html/issues/5365

reference

japanese edition

https://zenn.dev/yoya/articles/d874880e1820a5