Every encoder throws resolution away on purpose, and every decoder has to put it back. Segmentation masks, autoencoder reconstructions, GAN samples, diffusion decoders, super-resolution — all of them need an operator that takes a small feature map and returns a bigger one, with learnable weights rather than a fixed interpolation rule.
The operator invented for that job is almost universally mis-named. It is called deconvolution, and it deconvolves nothing. It does not invert the convolution. It only reproduces its shape.
What it actually is, exactly and with no hedging: the transpose of the convolution matrix.
Convolution is linear, so it has a matrix — a sparse banded O x I thing whose rows are shifted copies of the kernel. Then:
y = C x the convolution
x' = Cᵀ y the transposed convolution
That is the whole definition. The page builds C explicitly and checks it.
Live, every pixel computed by hand-written loops: https://dev48.infy.uk/dl/day62-transposed-convolutions.html
Kill the word "deconvolution" with two lines
If it were an inverse, CᵀC would be the identity. Build C, multiply, look:
CᵀC != I
The page measures how far off it is. That single comparison ends the naming argument permanently, and it takes two lines to run.
The adjoint test is the strongest check you can write
For any x and y, the transpose satisfies:
<Cx, y> == <x, Cᵀy>
Both sides are scalars. If your transposed convolution is implemented correctly, they agree to floating-point noise for random x and y. If you have an off-by-one in the scatter loop, they do not.
This is worth more than any visual check, because it fails loudly on exactly the bugs that produce plausible-looking output.
The checkerboard is not noise. It is a tap count.
The famous checkerboard artifact is usually shown as a picture and explained with a hand-wave. It is completely countable, before any training happens.
Run the scatter loop and count how many kernel taps land on each output position. When the kernel size is not divisible by the stride, some positions receive more taps than their neighbours — in a fixed, repeating pattern. That pattern is the checkerboard.
The page counts taps per output cell and shows the grid. Change k and s and watch the artifact appear and vanish as divisibility changes. No training, no data, no randomness — just arithmetic.
Three fixes, and the one the tap count does not cover
-
Make
kdivisible bys. The tap count goes uniform and the artifact disappears from this cause. - Resize then convolve. Nearest-neighbour upsample followed by a stride-1 convolution — same kernel, same parameter count, same output size, no scatter.
-
Sub-pixel convolution with ICNR. A convolution to
s²channels followed by a pixel-shuffle, which is a permutation, initialised so the shuffle starts out flat.
The honest caveat: uniform tap counts remove the architectural checkerboard. They do not stop a network from learning a periodic artifact if the loss rewards one. The tap count is a necessary condition, not a guarantee.
Output sizes, both directions, and why the inverse is ambiguous
Forward, the output size is determined. Backward, it is not — several input sizes map to the same output under the same k, s, p. That ambiguity is why frameworks make you pass output_padding, and it is another reason the operator cannot be an inverse: inverses are unique.
The page sweeps the whole reachable configuration space of k, s, p and reports which combinations are artifact-free.
Plus the decoder with its backward pass written out, and the zero-insert formulation — correct, instructive, and a deliberately bad implementation, shown so you can see why nobody ships it.
Top comments (0)