DEV Community

Devanshu Biswas
Devanshu Biswas

Posted on

I Tried to Reproduce Attention Sinks and Got Nothing. The Null Result Is the Interesting Part

#ai #llm #machinelearning #programming

Here is a result everyone repeats: a sliding-window KV cache collapses when the first few tokens fall out of it, and keeping just 4 of them fixes it. That is StreamingLLM (Xiao et al., 2023), and it is real.

I set out to reproduce it and measured no effect at all. Working out why taught me more than a clean reproduction would have.

Everything below is computed live: https://dev48.infy.uk/ai/days/day64-attention-sinks.html

Why the first tokens get anything in the first place

Softmax must sum to 1. Not "roughly 1" — exactly 1, always, whether or not any token in the window is relevant to the query:

const attn = scores.map(s => Math.exp(s - max));
const Z = attn.reduce((a, b) => a + b, 0);
return attn.map(a => a / Z);       // sums to 1. There is no "attend to nothing".
Enter fullscreen mode Exit fullscreen mode

There is no null option. The mass has to land somewhere, and empirically it lands on the earliest positions.

Part of that is pure causality and nothing to do with content. In a 512-token stream, position 0 is visible to 511 later queries; the final position is visible to none. The page separates the two explanations and shows visibility alone accounts for a large part of the ordering — position 0 receives far more than the midpoint, which receives more than a late position.

The measurement that failed

The claim to test: at a fixed cache budget, keeping 4 sink tokens plus a shorter recent window beats a pure sliding window.

Same budget, 512-token stream, random q/k:

policy attention mass retained
sliding window, 128 entries 46.1%
streaming, 4 sinks + 124 recent 46.1%

Nothing. Not a small win — no win. The two policies are indistinguishable.

Why, and what it says about the real thing

With random q/k, the first four positions are ordinary tokens that happen to be early. Trading four recent tokens for four ordinary early ones is a wash, and it should be.

In a trained model those positions are not ordinary. The model has learned to route mass to them — they act as a place to dump attention when nothing is relevant, which is why the paper describes them as sinks rather than as "the first tokens". The sink is learned, not geometric.

So the page adds one declared parameter standing in for that learned routing:

// a bias the model has learned toward the sink positions.
// NOT emergent here - declared, so the page cannot claim to have discovered it.
if (pos < S) score += sinkBias;
Enter fullscreen mode Exit fullscreen mode

Turn it on and the fix becomes decisive at an identical budget — streaming retains far more mass and shifts the distribution far less than the sliding window. Turn it off and the null result comes back.

That is the honest version of the finding, and it is a better lesson: StreamingLLM works because of what the model learned, not because the first tokens are first. A reproduction that skipped the null result would have taught the wrong mechanism.

The elbow at 4, reproduced instead of quoted

The paper says 4 sinks. Rather than repeat it:

sinks mass retained
0 → 4 large gain
4 → 16 small gain

Diminishing returns are visible, and the elbow is where the paper puts it. The number is worth having because it was re-derived.

Structural checks, because a plausible curve is not a correct one

  • S = 0 must reduce exactly to the sliding window. It does — 0 difference. If it did not, the two policies were never comparable.
  • Dense must retain exactly 100% at shift 0. The reference has to be a reference.
  • Total variation must be a metric — zero on identity, symmetric, bounded by 1. All checked.
  • RoPE must stay relative. ⟨R(q,m), R(k,n)⟩ = ⟨R(q,m−n), R(k,0)⟩ over 500 random pairs, worst relative difference ~1e-16. Every position claim on the page rests on that identity holding.
  • Attention sums to 1 to 1e-16, is causal, non-negative, at every position tested.

What I would keep

  • Simulating a mechanism is not simulating a trained model. If the effect depends on learned behaviour, a random-weight simulation will return nothing, and that nothing is information.
  • Declare the thing you are standing in for. A named sinkBias is honest. Quietly tuning the initialisation until the curve looks like the paper's is not.
  • A null result on a famous claim usually means you have modelled the wrong mechanism. That is worth chasing, not deleting.

Part of a from-scratch series — one AI concept a day, measured in-browser: https://dev48.infy.uk/aifromzero.php

Top comments (0)