A trained classifier hands you a label and nothing else. It says ring, with 0.97 confidence, and you have no way to know whether it looked at the ring or at a watermark in the corner of every ring photo in your training set.
Grad-CAM (Selvaraju et al., 2016) pulls that answer out of the network you already have. No retraining, no architecture change, no extra parameters. The whole method is three lines of calculus once you know where to tap.
Where to tap, and why it has to be there
The last convolutional layer is the deepest place in the network where activations still have coordinates. A value at cell (i,j) of feature map A^k still means "there, in the image". Everything after it — global average pooling, the classifier — deliberately throws that geometry away, which is exactly what makes the classifier translation-invariant and exactly why you cannot ask it where it looked.
So tap the layer that still knows.
Backpropagate the logit, not the loss
This is the part that trips people who have only written training loops. Training backpropagates the loss and stops at the parameters. Grad-CAM backpropagates a single logit y^c and stops at the activations. Different seed, different destination.
def tap_grad(net, cache, c, tap):
"""dy^c / dA for the activations of layer `tap`. Seed = one-hot on logits."""
Z, K = 8*8, net.C3
# y^c = SUM_k Wd[c,k] * mean(a3[k]) -> dy^c/da3[k,i] = Wd[c,k] / Z
da3 = np.repeat(net.Wd[c] / Z, Z).reshape(K, 8, 8)
if tap == 2: return da3
da2 = conv_backward(net.c3, cache.a2, relu_back(cache.z3, da3))
if tap == 1: return da2
return conv_backward(net.c2, cache.a1, relu_back(cache.z2, da2))
Two things must be true and are routinely got wrong. The seed is a one-hot vector on the logits, not p - y. And there is no softmax anywhere in the chain — the softmax normalises across classes, so once the model is confident the gradient through it is nearly flat and most of your signal divides out.
Grad-CAM itself
def grad_cam(net, cache, c, tap=2):
A = cache.acts[tap] # (K, h, w) the feature maps
dA = tap_grad(net, cache, c, tap) # (K, h, w) dy^c / dA
alpha = dA.mean(axis=(1, 2)) # (K,) ONE number per channel
L = (alpha[:, None, None] * A).sum(0)
return np.maximum(L, 0) # ReLU: keep only positive influence
Pool each channel's gradient map into a single importance α. Weight the feature maps by it and sum. ReLU.
The ReLU is not cosmetic. α < 0 means the channel is evidence against the class, and without the clamp the map blends "this is why it's a ring" with "this is why it isn't" into one indistinguishable blur. Turn it off and the negative regions are genuinely informative — they are just not the question you asked.
The output is an 8×8 map, the size of the layer you tapped. It is defined in feature-map coordinates, not pixel coordinates, which matters in a moment.
The correctness test that is algebra, not eyeballing
"It looks right" is the worst possible test for an explanation method — a subtly wrong backward pass produces a heat-map that looks completely plausible. So use a test that cannot be fooled.
On a global-average-pool + one-linear-layer head the algebra is forced. Since y^c = Σ_k w_k^c · (1/Z) Σ_ij A^k_ij, we get ∂y^c/∂A^k_ij = w_k^c/Z for every cell, and averaging Z identical numbers gives α_k^c = w_k^c/Z. The 1/Z is global, so after normalisation the map is bit-identical to CAM:
Z = 8 * 8
cam_map = np.maximum((net.Wd[c][:, None, None] * cache.a3).sum(0), 0) # plain CAM
gc_map = grad_cam(net, cache, c, tap=2) # Grad-CAM
assert np.allclose(grad_cam_alpha(net, cache, c) * Z, net.Wd[c], atol=1e-10)
assert np.allclose(norm01(gc_map), norm01(cam_map), atol=1e-10)
That is the honest framing of what Grad-CAM is: a strict generalisation of CAM, not an approximation of it. CAM only works if your head is exactly GAP → one linear layer; any fully-connected head, any pooling variant, and you have to rebuild and retrain the network. Grad-CAM works on any architecture and any layer, and collapses to CAM exactly where CAM is legal. If your implementation fails that assertion at 1e-10, it is broken.
Behind that sits the usual gradient discipline — central finite differences on every parameter against the loss, and on every feature-map entry against the logit:
def check(f, arr, i, analytic, eps=1e-5):
old = arr[i]
arr[i] = old + eps; fp = f()
arr[i] = old - eps; fm = f()
arr[i] = old
numeric = (fp - fm) / (2 * eps) # central difference, O(eps^2)
rel = abs(analytic - numeric) / max(1e-8, abs(analytic) + abs(numeric))
assert rel < 1e-5, (analytic, numeric, rel)
The upsampling bug almost every tutorial ships
Nearly all of them do cv2.resize(cam, image.shape), which maps map-corner to image-corner. That is wrong.
Cell j of an 8×8 map produced by a stride-2 stack with a 9×9 receptive field is the response of a window centred on pixel 2j+4. The map covers pixels 4…18 of 24 — not 0…23. A corner-to-corner stretch shifts the entire explanation by up to 4 pixels here, and on a 224×224 input that is a 16-pixel lie about where the model looked.
# receptive-field geometry of a stride-1/2 stack, computed once
# jump = product of strides = 2
# rf = 1 + SUM (k-1) * jump_before = 9
# start = centre of cell 0 = (rf - 1) / 2 = 4
def upsample_cam(m, stride, offset, H, W):
"""cell j belongs at pixel j*stride + offset; clamp outside the band."""
src = (np.arange(H) - offset) / stride
src = np.clip(src, 0, m.shape[0] - 1)
return bilinear_sample(m, src, src)
Testing an explanation behaviourally
Two tests separate a real explanation from a pretty picture, and both are cheap.
The pointing game: build scenes with two objects of known classes at known positions, run one forward pass, and ask for each class in turn. The peak must land on that class's object.
hits = 0
for scene in two_object_scenes(30): # a bar and a ring, disjoint
cache = forward(net, scene.image) # ONE forward pass
for c, box in enumerate(scene.boxes):
peak = argmax_pixel(upsample_cam(grad_cam(net, cache, c)))
hits += dist(peak, box) < dist(peak, scene.boxes[1 - c])
assert hits / 60 > 0.90 # trained
# random weights: ~0.50 — the map stops being class-conditional
A trained model scores above 90%. The identical architecture with random weights scores about 50% — coin-flip — because its map does not depend on the class at all. That is the weight-randomisation sanity check, and it is the one test that catches a method which is really just an edge detector wearing a lab coat. Run it. Several published saliency methods fail it.
And the reason to do any of this at all: Grad-CAM is conditioned on the model, so it exposes shortcuts. Poison a training set with a small watermark in the corner of one class's images and you get a model that is ~100% accurate and looks straight at the watermark. The accuracy number tells you nothing. The heat-map tells you the whole story.
Grad-CAM++ and saliency, briefly
Grad-CAM++ keeps the skeleton and changes only how α is formed — a per-pixel coefficient built from higher derivatives of exp(y^c), with negative gradients clipped. The exponential factors out of the ratio, so it is a closed form in g with no second backward pass:
num = g ** 2
den = 2 * g ** 2 + (A * g ** 3).sum(axis=(1, 2), keepdims=True)
a_ij = np.where(den != 0, num / den, 0)
alpha = (a_ij * np.maximum(g, 0)).sum(axis=(1, 2)) # positive gradients only
That per-pixel weighting is what lets it spread across several instances of a class instead of locking onto the single strongest one. Plain saliency is the other extreme — the same backward pass carried all the way to the pixels, giving |∂y^c/∂x|: high resolution, visibly noisy, and barely class-dependent.
In practice
On a real backbone you reimplement nothing. Two hooks:
feats, grads = {}, {}
layer = model.layer4[-1] # last conv block
layer.register_forward_hook (lambda m,i,o: feats.update(a=o))
layer.register_full_backward_hook(lambda m,gi,go: grads.update(g=go[0]))
logits = model(x) # x: (1,3,224,224)
model.zero_grad()
logits[0, c].backward() # the LOGIT for class c, not the loss
alpha = grads['g'].mean(dim=(2, 3), keepdim=True) # (1,K,1,1)
cam = torch.relu((alpha * feats['a']).sum(1)) # (1,7,7)
cam = F.interpolate(cam[None], size=x.shape[-2:], mode='bilinear',
align_corners=False)[0, 0]
cam = (cam - cam.min()) / (cam.max() - cam.min() + 1e-8)
Four traps worth naming. Zero the gradients or you accumulate the last call's. Backprop the logit, never the loss and never the softmax probability. align_corners=False and, if you care about precision, the receptive-field offset rather than a plain resize. And run the randomisation check every time you point this at a new architecture.
Then the honest caveat: a heat-map is a hypothesis about the model, not proof about the world. It tells you which spatial regions the class score was sensitive to at the layer you tapped. It does not tell you the model understands rings, and it cannot tell you the model is right.
What the page does
A real 3-layer CNN with a real backward pass trains live in your browser — 729 parameters, no libraries — and every heat-map is computed by hand from its own gradients. Ask it for a different class and watch the map move to that class's object. Switch to the poisoned model and watch a ~100%-accurate classifier stare at the corner watermark. Randomise the weights and watch the map turn to noise. Toggle the final ReLU off to see the evidence it was hiding.
The self-check panel runs live on every load and reports what it actually measured:
dL/dw vs central differences max rel err 7.87e-7 PASS
dy^c/dA vs central differences max rel err 3.14e-8 PASS
alpha_k^c * Z - w_k^c max abs err 3.11e-15 PASS (Grad-CAM == CAM)
sum(softmax) 1.000000000000
One footnote on that first line, because it bit me while checking this page. The parameter gradient check runs on untrained weights on purpose. Once the model converges the true gradient decays to about 1e-5 while central differences bottom out around 1e-11 — so a relative error computed there measures floating-point round-off, not your backward pass, and reports a scary 1.17e-4 for code that is correct to 1e-11 absolute. Gradient-check at initialisation, or grade on a mixed absolute/relative tolerance. Do not "fix" a backward pass that was never broken.
Top comments (0)