DEV Community: Mathias Leonhardt

Transformer Attention Is Hopfield's 1982 Update Rule (And What That Tells Us About LLM Memory)

Mathias Leonhardt — Thu, 04 Jun 2026 15:40:53 +0000

Hopfield's associative-memory equation from 1982 and the scaled dot-product attention from Vaswani 2017 are the same operation. One substitution turns one into the other. The 2024 Nobel Prize in Physics — to Hopfield and Hinton — is the academic acknowledgement that the mathematics behind today's LLMs was already written four decades ago, in a different vocabulary.

This is a condensed write-up of the longer, interactive piece at ki-mathias.de/en/hopfield.html. Seven chapters there, five live MNIST demos. Here I focus on the four steps where the story has interesting empirical edges.

1. The identity

Modern Hopfield (Ramsauer et al., 2020) writes the update rule as

v ← X · softmax(β · Xᵀv)

where X ∈ ℝ^(N×p) is the matrix of stored patterns and β > 0 is an inverse-temperature parameter.

Scaled dot-product attention (Vaswani et al., 2017) writes

Attention(Q, K, V) = V · softmax(Kᵀ Q / √dₖ)

Set Q = v, K = X, V = X, and β = 1/√d_k. The two equations become identical. Not analogous. Identical. Same operation, written in two different notations.

In a Transformer, K and V are independent learned projections of the same input rather than the same matrix, and Q is yet another projection. Those are extra learnable transformations around the Hopfield core; the softmax-weighted lookup in the middle is unchanged.

Krotov & Hopfield (2016) had already worked out the dense associative memory generalisation that gives this form its exponential storage capacity. Vaswani 2017 reached the same equation by iterating on machine-translation benchmarks. Ramsauer 2020 noticed they were the same. The independent rediscovery is itself diagnostic: the structure isn't a design choice, it's a forced consequence of the requirements.

2. Why classical Hopfield breaks on MNIST (and why that's not a bug)

The original 1982 recall rule is

vᵢ ← sign(Σⱼ Wᵢⱼ · vⱼ)        # W = (1/N) Σₘ ξₘ ξₘᵀ,  Wᵢᵢ = 0   (index m runs over stored patterns)

This is the Hebb construction. Store ten MNIST digits, query each with 15 % pixel noise, observe what comes back.

Result: all ten queries collapse into the same end-state — an image that isn't visually any of the stored digits. Mean pairwise similarity between the ten "recalls": 0.99.

This is fully explained by the spectrum of W_Hebb. The eigenvalues are roughly

λ₁ ≈ 6.65,   λ₂ ≈ 0.65,   λ₃ ≈ 0.48,   ...

A factor-of-ten gap between λ₁ and the rest. The top eigenvector is essentially ξ̄ = (1/p) Σₘ ξₘ, the per-pixel mean — cosine 0.9999.

The Hebb rule is provably correct only under two conditions:

Pairwise orthogonality of stored patterns.
Zero-mean patterns.

MNIST digits violate both: pairwise inner products are 400–600 out of 784 (≈ two thirds of the pixels shared), and mean pixel values are −0.63 to −0.90 (much more "background" than "ink"). The failure is therefore not an implementation bug; it's the construction operating outside its range of validity. Centring the patterns kills the bias sink but reveals the next defect — the v → −v symmetry of E(v) = -½vᵀWv causes recalls to land on negations of stored patterns.

The didactic point: a learning rule is correct or incorrect relative to a data geometry. "Hebb is broken" is not a sentence. "Hebb is broken on MNIST" is.

3. The pseudoinverse fix and its capacity cliff

The Personnaz–Guyon–Dreyfus construction (1985) keeps the same recall machinery but builds W differently:

W_PI = X (XᵀX)⁻¹ Xᵀ

The factor (XᵀX)⁻¹ is exactly what's missing in Hebb — the inverse of the pattern-pattern Gram matrix. It removes correlations between stored patterns before the matrix becomes the energy landscape. For orthogonal patterns the two rules coincide; for correlated ones, only W_PI carries the algebraic guarantee

W_PI · ξₚ = ξₚ              # every stored pattern is a fixed point with eigenvalue 1

Empirical capacity on MNIST, p stored patterns, 10 % pixel noise, fraction of queries that recover the original:

p	Hebb	Pseudoinverse
10	0 %	100 %
100	0 %	100 %
150	0 %	97 %
200	0 %	32 %
250	0 %	1 %
300	0 %	0 %

A sharp phase transition between p ≈ 150 and p ≈ 250. Far below the algebraic ceiling p = N = 784, where the Gram matrix becomes singular. The identity W_PI ξₚ = ξₚ holds throughout — but the basin of attraction around each fixed point shrinks as the patterns crowd one another, and 10 % noise overshoots the basin once p exceeds ~150.

Side note for readers who came in via the Eigenvalues post: the operator X(XᵀX)⁻¹Xᵀ is exactly ridge regression with λ = 0 — the pseudoinverse hat matrix. The Hopfield update with this W is therefore a non-linear filter built on top of an ordinary projection onto the span of stored patterns. The capacity cliff is the cliff of unregularised projection at near-singular Gram.

4. Modern Hopfield = Attention (the move that fixes capacity)

Stop iterating sign(Wv). Replace it with the soft, input-dependent

v ← X · softmax(β · Xᵀv)

Three structural changes happen at once:

Component	Classical (1982/1985)	Modern (Ramsauer 2020)
Operator	fixed `W ∈ ℝ^(N×N)`	none — direct softmax-lookup on X
Update	linear in v + sign	non-linear (softmax in v)
Energy	quadratic `-½ vᵀWv`	log-sum-exp + `½‖v‖²` (Lyapunov)
Convergence	iterative, many sweeps	one step (for sufficiently large β)
Capacity	dynamically ≪ N	`Ω(exp(N))` — exponential in N

The exponential capacity is the practical reason this works for LLMs at all: with N = 768 (a typical embedding dim), you can store effectively-unbounded context. With N = 784 (MNIST), the classical pseudoinverse rule plateaus near p ≈ 150 on real data.

And the parameter β is interpretable. At small β, the softmax is near-uniform and the recall is a soft average of all stored patterns. At large β, it concentrates on the single best match — Modern Hopfield converges to 1-nearest-neighbour. Ramsauer's analysis of Transformer heads shows early layers running at low β (global averaging) and deeper layers running at high β (sharp lookup on a single token). The classical "attention is mysterious" complaint dissolves into a continuous interpolation between two known operations.

5. The surprise: same learning rule, different geometry, totally different outcome

The interesting finding from Negri, Tudisco, Lucibello et al. 2024 — Random Features Hopfield Networks generalize retrieval to previously unseen examples — is not "we made Hopfield better." It's the opposite:

The exact same learning rule that scores 65 % accuracy on MNIST (i.e., barely matches 1-NN, no real generalisation) achieves perfect generalisation — magnetisation 1.0 on unseen test patterns — when the data is built as a sparse mixture of a small set of random features.

Setup: let F ∈ {-1,+1}^(N×D) be a random feature matrix. Each pattern is ξ = sign(F · c) with c an L-sparse binary coefficient vector. Three sets share the same F:

Train (p stored patterns)
Features (the D feature columns of F — never stored)
Test (new patterns from the same distribution, never stored)

Sweep α = p/N and measure the magnetisation of each set. Three phases appear in order:

Storage (α small): only train patterns are stable attractors.
Learning (α medium): train magnetisation drops, features magnetisation rises — the network has begun to recognise the components it was implicitly trained on.
Generalisation (α large): test patterns become attractors too — without ever having been stored.

With the pseudoinverse rule this last transition is a hard jump to magnetisation 1.0, and the math explains why: once the trained patterns span enough of the feature mixtures, every feature mixture becomes an eigenvector of W_PI with eigenvalue 1 — by the same identity that made stored patterns fixed points.

The takeaway is not subtle: generalisation is a property of the data geometry, not of the learning rule. A textbook claim that "this learning rule generalises better" is well-typed only relative to a class of data. The reason language models generalise so well isn't that the attention mechanism has a special "ability" — it's that natural language already has the sparse compositional structure that makes Hopfield-style retrieval transfer beyond the training set. Words and constructions are a finite set of components; sentences are sparse mixtures. Hopfield-friendly by accident of biology.

6. What runs on this mathematics today

A non-exhaustive list, with the empirical claim each item is making:

Every Transformer attention layer. Modern Hopfield is what's there. With high probability the most-executed mathematical operation on global compute, by raw volume.
MHNfs (Klambauer/Hochreiter, Linz) — few-shot drug discovery, 100k+ context molecules as memory, SOTA on FS-Mol.
DeepRC (Widrich et al., NeurIPS 2020) — multi-instance learning over ~10⁶ immune-repertoire sequences, used for SARS-CoV-2 classification.
Memristor Hopfield chips (HP Labs Nature Electronics 2020; Peking U. Nature Comms 2024) — analogue MAX-CUT solvers, ~4 orders of magnitude energy advantage over digital. The Peking paper proves mathematical equivalence to a Hopfield attractor network, not just analogy.
PyTorch drop-in: ml-jku/hopfield-layers — Hopfield, HopfieldPooling, HopfieldLayer modules, swap-in replacements for LSTM / pooling / attention.

7. What this does not say

It does not claim that every modern ML algorithm is "secretly a Hopfield network." The identity is precise between Modern Hopfield and scaled dot-product attention. Diffusion models, state-space models, ConvNets have different mathematical structures.
The Chapter-6 generalisation result is for synthetic feature-mixture data, not for MNIST. To transfer it to real data you need to first extract a feature basis (PCA, dictionary learning, learned embeddings) — i.e. you have to engineer the right kind of sparse architecture, the data won't give it to you for free.
Classical Hopfield is not "back" as a general-purpose ML tool. ConvNets/diffusion/Transformers are better for almost every benchmark task. The Hopfield reading earns its keep when memory is the actual problem — few-shot, multi-instance, episodic recall, combinatorial optimisation on dedicated hardware.

Your Ridge Parameter Is PageRank's Damping Factor (and Why BLAS Beats GPU for Mid-Sized KRR)

Mathias Leonhardt — Tue, 21 Apr 2026 18:45:29 +0000

The ridge parameter λ in kernel regression and the damping factor d in PageRank are the same object. Both create a spectral gap. Both stabilize an otherwise-unstable iteration. Once you see it, four unrelated-looking algorithms collapse into one story.

This post is a condensed, math-first write-up of two longer pieces on ki-mathias.de:

Eigenvalues & AI — the unification story, with interactive eigendecomposition demos.
KRR Chat: Under the Hood — a 3 MB browser chatbot trained with kernel ridge regression, no neural network, ~50 k parameters.
Paper: Leonhardt 2026 — Kernel Ridge Regression and PageRank: A Unified Absorber-Stochastic Framework (Zenodo, v2, 10 pages).
Try the chatbot: Kalle · Source: pmmathias/krr-chat.

1. The identity that does the work

Every algorithm below is an eigenvalue problem with a regularizer. The regularizer is spelled differently each time, but it plays the same role: it pushes eigenvalues away from the unit circle, which makes power iteration converge.

Algorithm	Core equation	"Regularizer"
Ridge regression	`(K + λI) α = y`	λ
PageRank	`v = d · M · v + (1 − d) · u`	d (damping ≈ 0.85)
Early-stopped gradient descent	`α_{t+1} = α_t − η (K α_t − y)`, stop at iteration t	1/t (Theorem 1)
Schrödinger eigenproblem	`Ĥ ψ = E ψ`	mass/potential scaling

Theorem 1 in the paper makes the second-last row precise:

Running gradient descent on ½ ‖K α − y‖² for exactly t iterations and stopping approximates the ridge solution with regularizer λ ≈ 1/t.

Early stopping is spectral filtering. You never wrote λ into your training script, but by choosing a step count you chose one. That's why "no regularization + early stopping" and "ridge regression" often train to the same test loss. They are the same filter applied two different ways.

2. Why the spectrum matters — with one concrete example

A row-stochastic matrix M (like the link matrix of a hyperlink graph) has a leading eigenvalue of exactly 1, and the rest can hug that 1 arbitrarily closely. Power iteration will converge to the leading eigenvector, but slowly — the convergence rate is set by the gap 1 − |λ₂|. If the gap is 10⁻⁶, expect a million iterations.

PageRank fixes this by mixing in a teleport term:

v ← d · M · v + (1 − d) · u      # d = 0.85, u = uniform

That pins the spectral radius of the effective operator at d, not 1. Now every eigenvalue except the leading one is multiplied by 0.85 each iteration — the gap becomes ≥ 0.15, and the walk converges in tens of steps.

Ridge regression is the linear-algebra twin:

α = (K + λI)⁻¹ y

Same trick. The matrix K has eigenvalues that can be tiny (noise directions). Inverting K alone amplifies noise. Adding λI floors the spectrum: every eigenvalue of K + λI is at least λ, so the inverse is bounded by 1/λ. You trade bias for variance, and the λ knob is exactly PageRank's damping factor d.

3. Absorber-Stochasticization (the paper's §6 trick)

Here's the subtle bit. Writing PageRank as an eigenvalue problem requires M to be properly stochastic (rows sum to 1). Real link matrices have dangling nodes — pages with no outgoing links — so they're sub-stochastic. Standard treatments patch this with a uniform "teleport from dead ends" rule, but that mixes two different regularizers (λ and teleport).

The paper's construction is cleaner: add a single extra row and column — the absorber — that catches all missing mass. Now the matrix is exactly stochastic. The absorber state is transient; the stationary distribution lives on the original nodes.

The identical construction works for kernel ridge regression. Given (K + λI) α = y, build the augmented system

[ K + λI   r ]   [ α ]     [ y ]
[  sᵀ      σ ] · [ β ]  =  [ 0 ]

where the extra row/column carries the residual. The system is now block-solvable via a Krylov iteration whose spectrum is controlled by λ alone — no auxiliary regularizers, no hand-tuned dangling-node rule.

The practical payoff: you can train KRR at D ≈ 6k with a Block Preconditioned Conjugate Gradient and hit numerical convergence in ~11 iterations.

4. The benchmark that surprised me (BLAS > GPU)

Here is the v2 paper's Table 4 — training the Kalle chatbot (D = 6144, V = 2977):

Backend	Precision	Time (ms)	Iterations
Direct solve (NumPy/LAPACK)	Float64	2097	—
Block-PCG (NumPy)	Float64	6731	14
Block-PCG (PyTorch CPU)	Float32	1704	11
Block-PCG (PyTorch MPS / GPU)	Float32	1622	11

The surprise: PyTorch CPU is within 5% of GPU, and it beats NumPy's direct solve. Why?

BLAS is why. PyTorch CPU dispatches matmuls to MKL/oneDNN with AVX-512, multi-threaded. NumPy's linalg.solve calls LAPACK's direct factorization, which is O(D³) and doesn't parallelize as cleanly as a blocked matmul.
GPU transfer overhead eats the advantage. For D ≈ 6k, the operators fit in L2/L3 cache. Moving α and the preconditioner to GPU memory each iteration costs more than the matmul you save.
Float32 is enough. The preconditioner is well-conditioned after the absorber step, so Float32 converges in the same iteration count as Float64 with a 2× speed gain.

If your D is in the 10³–10⁴ range, don't reach for a GPU. Reach for PyTorch CPU with Float32. You'll ship on commodity hardware.

The inflection point is around D ~ 2·10⁴ on an M2 Pro — above that, GPU wins decisively. Your mileage varies with cache size and memory bandwidth; the takeaway is to measure, not to assume "GPU always wins."

5. Corpus scaling — N grows, solve stays 3–7% of total

The other surprise from the benchmarks: when I grew the training corpus from ~3 k to ~21 k sentences (V = 2977 → V = 21,433), the KRR solve stayed between 3% and 7% of the end-to-end pipeline. The bulk of time was spent in tokenization and RFF feature construction, not the linear solve.

This matters because the common intuition — "bigger corpus → quadratically harder training" — comes from full-kernel methods where you materialize the N×N gram matrix. With Random Fourier Features, the bottleneck is the D×D normal equations, and D is decoupled from N. Doubling your corpus doubles feature-extraction cost (linear in N) but leaves the solve essentially unchanged.

The scaling lever is D, not N. Budget accordingly.

6. Kalle — a 3 MB chatbot you can read

The paper's toy system is Kalle: a kernel-ridge-regression chatbot that I've deployed as a static web page at pmmathias.github.io/krr-chat. It has about 50 k parameters (three D×V matrices), fits in 3 MB, runs in-browser on your phone's CPU, and speaks German at a level somewhere between a broken Markov chain and a very confused graduate student. It won't replace ChatGPT. It will let you read every equation in the paper and see the matrices line up.

The point is pedagogical: at this scale, the mathematics is still visible. You can print out K + λI, look at its eigenvalues, and nothing is hidden behind a billion-parameter transformer. That's why I built it.

# Train step — the paper's Algorithm 1, 20 lines of Python
import torch

def train_krr_bpcg(K, y, lam, tol=1e-6, max_iter=50):
    D = K.shape[0]
    A = K + lam * torch.eye(D, dtype=K.dtype, device=K.device)
    M_inv = 1.0 / A.diagonal()  # Jacobi preconditioner
    alpha = torch.zeros_like(y)
    r = y - A @ alpha
    z = M_inv[:, None] * r
    p = z.clone()
    for it in range(max_iter):
        Ap = A @ p
        rz = (r * z).sum(dim=0)
        a = rz / (p * Ap).sum(dim=0)
        alpha = alpha + a * p
        r = r - a * Ap
        if r.norm() < tol:
            return alpha, it + 1
        z_new = M_inv[:, None] * r
        b = (r * z_new).sum(dim=0) / rz
        p = z_new + b * p
        z = z_new
    return alpha, max_iter

That's it. The absorber construction lives inside the preconditioner; the outer loop is textbook CG.

7. What this means if you're building

If you're writing a blog post that says "PageRank is an iterative algorithm and ridge regression is a closed-form algorithm" — they're the same eigenvalue problem, just with different stopping rules.
If you're choosing between GPU and CPU training for a moderate-D kernel method, measure first. For D < 10⁴, BLAS CPU is frequently the winner.
If you're looking for an early-stopping criterion, you can read it off the paper's Table 1: running for t iterations applies a filter approximately equivalent to λ ≈ 1/t. Choose t such that your effective λ matches the cross-validated ridge λ.
If you're trying to explain modern ML to someone else, the eigenvalue story is the shortest honest path. Every algorithm they'll encounter — PCA, PageRank, Schrödinger, ridge regression, neural network training — reduces to "find the eigenvectors of this operator, and regularize the spectrum so the iteration converges."

Why Quantum Computers Are Faster — the Answer Isn't Parallelism

Mathias Leonhardt — Mon, 20 Apr 2026 14:22:54 +0000

A qubit isn't a bit with randomness — it's a rotating arrow on a sphere. A quantum gate is a rotation of that arrow. A quantum algorithm is one global transformation that reshapes the whole solution landscape.

The headline everyone has read in a press release — "quantum computers try all possibilities in parallel" — is dangerously misleading. A measurement would just return one random answer anyway. So "parallel" on its own is useless. The actual trick is subtler, more beautiful, and the point of this post.

This is a condensed, tutorial-style version of a longer, interactive piece on ki-mathias.de — the full version has four interactive React demos (Bloch sphere, circuit builder, measurement simulator, Qiskit playground) and a 9-minute companion video. Everything here is runnable in Qiskit 1.x. The quantum-computing repo contains the notebooks.

1. A qubit, for real

At the hardware level, a qubit is always a two-level quantum system. There are several competing technologies in 2026 — superconducting circuits (Google, IBM), trapped ions (IonQ), neutral atoms (QuEra), photons (PsiQuantum) — but let's pick one and stay concrete: the superconducting qubit, because that's where most of today's qubits live.

A superconducting qubit is essentially a quantized electrical oscillator. You take an LC circuit — a coil and a capacitor, just like in an old radio — cool it to about 10 millikelvin (colder than intergalactic space), and add a special component called a Josephson junction. This junction is two superconductors separated by a thin insulating barrier. Brian Josephson predicted in 1962 that bound electron pairs (Cooper pairs) can quantum-mechanically tunnel through this barrier — Nobel Prize 1973.

The Josephson junction bends the energy ladder. Without it, you'd get a harmonic oscillator with equally spaced levels — a microwave pulse tuned to excite |0⟩→|1⟩ would also kick |1⟩→|2⟩. With it, the rungs get closer together as you go up, and you can address the lowest transition selectively. That selectivity is what turns a quantum oscillator into a usable qubit.

The two lowest levels, |0⟩ and |1⟩, form the qubit. Their energy gap corresponds to a microwave frequency around 5 GHz — that's the frequency of the pulses we'll use to manipulate the qubit.

2. Amplitudes, not probabilities

Every quantum state is a superposition

|ψ⟩ = α |0⟩ + β |1⟩

where α and β are complex numbers — amplitudes. The Born rule (Max Born, 1926, Nobel 1954) says the probability of measuring state |0⟩ is |α|² and |1⟩ is |β|², with |α|² + |β|² = 1.

Crucially, amplitudes can be negative or complex — unlike probabilities, they can cancel each other out. That's interference, and it's the whole point.

A Hadamard gate H turns |0⟩ into a perfectly balanced superposition:

H |0⟩ = (1/√2) (|0⟩ + |1⟩)

Here's the proof that amplitudes aren't classical probabilities: apply H twice to |0⟩. Each individual H puts you in a 50/50 superposition. A classical coin would still give 50/50 after a second flip. The qubit? Goes back to |0⟩. Deterministic.

Why? Because two H matrices multiply to the identity. Physically: the amplitude for |0⟩ coming from the |0⟩-branch adds constructively with the one from the |1⟩-branch; the amplitudes for |1⟩ cancel. Exactly the kind of destructive cancellation classical probabilities can't do.

3. The whole secret, one sentence

Before we look at an algorithm, the mental model has to shift.

Old picture: quantum computers try all possibilities in parallel. Wrong.

New picture: a quantum algorithm is one global linear transformation on a 2^n-amplitude vector, carefully designed so that wrong answers destructively cancel and the right one constructively amplifies.

Classical searches. Quantum shapes.

This isn't just poetry. Every quantum gate, expressed as a matrix, is 2^n × 2^n — it acts on the entire state vector at once. Ten qubits → a 1024 × 1024 matrix per gate. This is not parallel trial-and-error; it's a global transformation of a high-dimensional probability field.

If you've touched linear algebra before, there's an even tighter way to say this: a quantum algorithm is a chain of unitary operations whose eigenstates align with the answer we're looking for. What starts spread across all directions gets pushed onto the right axis with each iteration. Quantum computation is eigenstate amplification.

4. Grover, step by step

The cleanest place to see this in action is Grover's algorithm (1996), which finds a marked entry in an unsorted database of size N in O(√N) steps versus O(N) classically. Let's walk through N = 8 — three qubits.

Setup. Three Hadamards on three qubits create the equal superposition: eight basis states, each with amplitude 1/√8. Measurement here is just an 8-sided die. Useless.

Step 1 — The oracle. A unitary circuit that recognizes the marked state (say, |101⟩) and flips its sign. After the oracle, |101⟩ has amplitude -1/√8, the other seven stay at +1/√8. Measuring still gives equal probability — the squared magnitudes are the same. But the phase information is hiding in the sign.

Step 2 — Diffusion. Geometrically, this is a single operation: reflect every amplitude about the mean of all amplitudes. The mean is about 0.24 before reflection (seven positives dominate over one negative). After reflection, the marked state jumps to about +2.12 / √8 (amplified dramatically), the others drop to about 0.18 / √8 (shrunk).

Visually, in bar-chart form:

Start       | | | | | | | |    (all equal positive)
After oracle| | | | | |▽| |    (one flipped negative)
After diff. ▁ ▁ ▁ ▁ ▁ █ ▁ ▁    (one big, others small)
Round 2     . . . . . ██ . .   (solution dominates ~95%)

Step 3 — Repeat. One "Grover step" = oracle + diffusion. Optimal peak is around ⌊π/4 · √N⌋ iterations. For N = 8, that's 2 rounds. Then measure, and you get the marked state with > 94% probability.

Classical average: N/2 = 4 oracle queries. Grover: 2. The win grows: for N = 10⁹, classical needs ~500 million queries, Grover ~25,000.

Quadratic, not exponential — but enough to change everything about unsorted search. And the mechanism is exactly what we said: amplitudes being shaped, not possibilities being tried in parallel.

5. Your first quantum circuit: the Bell state in Qiskit

The "Hello World" of quantum computing. Two qubits, two gates, perfect correlation.

from qiskit import QuantumCircuit
from qiskit_aer import AerSimulator

qc = QuantumCircuit(2, 2)
qc.h(0)            # put q0 in superposition
qc.cx(0, 1)        # CNOT: flip q1 iff q0 is 1
qc.measure([0, 1], [0, 1])

sim = AerSimulator()
result = sim.run(qc, shots=1024).result()
counts = result.get_counts()
print(counts)
# {'00': ~512, '11': ~512}

The Hadamard puts qubit 0 into (|0⟩ + |1⟩)/√2. The CNOT flips qubit 1 iff qubit 0 is 1. Applied to the superposition, you end up with

|Φ⁺⟩ = (1/√2) (|00⟩ + |11⟩)

Measuring gives you |00⟩ 50% of the time, |11⟩ 50% — and never |01⟩ or |10⟩. That's entanglement. Measure one qubit, you instantly know the other — Einstein's "spooky action at a distance," experimentally confirmed (Aspect et al., Nobel 2022).

Running on real hardware is two more lines:

from qiskit_ibm_runtime import QiskitRuntimeService, SamplerV2
from qiskit import transpile

service = QiskitRuntimeService(channel="ibm_quantum_platform")
backend = service.least_busy(simulator=False, operational=True)

qc_t = transpile(qc, backend)
sampler = SamplerV2(mode=backend)
job = sampler.run([qc_t], shots=1024)
print(job.result()[0].data.meas.get_counts())

Real hardware gives you noisy results — maybe 490 / 508 / 20 "garbage". That's decoherence and gate errors. The IBM Quantum Platform has a free tier with 10 minutes of QPU time per month on 100+ qubit systems.

6. Shor in one paragraph

Shor's algorithm (1994) uses the same shaping principle for a totally different problem. The key insight: factoring N reduces to finding the period of the sequence a, a², a³, ... mod N. Period-finding classically is nearly as hard as factoring itself — but quantum mechanically, you put qubits into a superposition over all x, compute aˣ mod N into a second register, and apply the Quantum Fourier Transform to the first register. The QFT converts a periodic amplitude distribution into a sharply peaked one — amplitudes that don't match the true period cancel destructively, matching ones amplify constructively. A single measurement returns, with high probability, a value from which the period can be reconstructed classically.

Same move as Grover, different shaping tool.

7. What does NOT get faster (honest bit, 2026)

No Quantum Excel. No Quantum ChatGPT. No "10,000× AI training speedup."
Quantum computers are special-purpose processors for problems whose structure interference can exploit. Training neural networks on classical data? No speedup.
Current hardware (Google Willow 105 qubits, IBM Heron, IonQ Tempo) is NISQ-era — noisy, no full error correction. For Shor on 2048-bit RSA, we need a few million error-corrected qubits. Conservative estimate: 2030s.
But: the post-quantum crypto migration is already happening (NIST standards ML-KEM, ML-DSA since 2024), because "harvest now, decrypt later" is a real threat.

8. Further reading + code

Full blog post (interactive Bloch sphere, circuit builder, measurement simulator, Qiskit playground, SVG of Grover amplitudes): ki-mathias.de/en/quantum-computing.html
9-minute companion video with Manim animations: youtu.be/iMaqNfA0Gs0
Qiskit notebooks on GitHub (Bell state, Deutsch-Jozsa, Grover search, VQE for H₂ — all runnable on IBM free tier): github.com/pmmathias/quantum-computing

TL;DR

Classical computation is a path. Quantum computation is a field. The speedup comes not from running many computations in parallel, but from transforming the entire probability landscape in one shot — destructively cancelling wrong answers, amplifying the right one, before the measurement collapses everything to a single value.

Three concrete takeaways for any developer:

|ψ⟩ = α|0⟩ + β|1⟩ — amplitudes, not probabilities. They can cancel.
Every gate is a 2^n × 2^n unitary matrix acting on the whole state vector at once.
Grover amplifies a marked state by reflecting amplitudes about their mean. Shor amplifies the right Fourier peak. Same principle, different shaping tools.

If you've got Python and 15 minutes, fork the repo above, run 01_bell_state.ipynb, and see your first entangled pair of qubits. It's less intimidating than it sounds.

Deepfakes Explained — From Vectors to the Decoder Swap (Interactive)

Mathias Leonhardt — Sun, 19 Apr 2026 10:17:19 +0000

Most explanations of deepfakes start with "AI swaps faces." That's like explaining a car engine by saying "it goes vroom." I wanted to understand the math — so I built a blog post that walks through it from scratch.

The result: 13 chapters, 6 interactive demos, and a real autoencoder running in your browser — no TensorFlow.js, just 200KB of pure JavaScript matrix math.

The path from vectors to face-swapping

The post follows the arc of a tech talk I gave at my company. Each concept builds on the previous:

Vectors & Matrices — what does it mean to "transform" data?
Orthogonality — when are features truly independent?
Blind Source Separation — separating mixed audio signals with linear algebra
High-dimensional data — a 200-pixel image is a 200-dimensional vector
PCA — finding the axes that explain the most variance
Linear vs. nonlinear interpolation — why blending faces in pixel space gives you ghosts, not faces
The kernel trick — lifting data into higher dimensions where nonlinear becomes linear
Neural networks — the machine that does dimensionality reduction AND increase simultaneously
Autoencoders — compress → latent space → decompress
Latent space arithmetic — smiling woman − neutral woman + neutral man = smiling man
Deepfakes — swap the decoder. That's it. That's the trick.

The interactive parts

Every key concept has a visualization you can play with:

PCA Playground

Draw 2D points → watch the principal component axes update live. Eigenvalues, explained variance, the whole thing.

Matrix Transform

Drag sliders for rotation and scale → see how a matrix transforms a vector in real-time.

Kernel Trick

Toggle between 2D (not linearly separable) and 3D (linearly separable!) with a button. The separating plane appears when you lift the data.

Rotating Tesseract

A 4D hypercube projected to 2D, with three animated rotation planes. Includes a 3D↔4D slider that collapses the tesseract into a regular cube — showing what the 4th dimension "does."

MNIST Autoencoder

Draw a digit → the autoencoder encodes and reconstructs it live. The model is a tiny PyTorch-trained network (784→64→16→2→16→64→784) exported as float16 binary weights. Inference runs in pure JavaScript — no TensorFlow.js, no WebAssembly, just matrix multiplications and ReLU.

Total model size: 200KB.

Latent Space Explorer

Move your mouse across a 2D latent space → the decoder generates digits in real-time. Each color cluster = one digit class. You can see how the autoencoder organized the digits spatially.

The autoencoder — no TensorFlow.js needed

This was the most satisfying part to build. Instead of shipping a 3MB TensorFlow.js dependency, I:

Trained a tiny autoencoder in PyTorch (12 epochs on MNIST, ~30 seconds)
Exported weights as float16 binary (not JSON — 200KB instead of 2.2MB)
Wrote inference in ~60 lines of vanilla JavaScript:

function linear(x, W, b) {
  var out = new Float32Array(W.shape[0]);
  for (var i = 0; i < W.shape[0]; i++) {
    var sum = b.data[i];
    for (var j = 0; j < W.shape[1]; j++)
      sum += W.data[i * W.shape[1] + j] * x[j];
    out[i] = sum;
  }
  return out;
}

function encode(x, model) {
  var h1 = relu(linear(x, model.enc_0_weight, model.enc_0_bias));
  var h2 = relu(linear(h1, model.enc_2_weight, model.enc_2_bias));
  return linear(h2, model.enc_4_weight, model.enc_4_bias);
}

That's it. No framework, no build step, no WASM. Load the binary weights, do matrix math, draw pixels on a canvas.

The deepfake mechanism in one paragraph

Train two autoencoders with a shared encoder but different decoders. The encoder learns a universal face representation (latent space). Decoder A reconstructs face A, Decoder B reconstructs face B. Now swap: feed person A's image through the shared encoder, then through decoder B. Result: A's expression and pose, B's appearance. A deepfake.

Try it

Interactive blog post — all 6 demos, no login, no tracking
YouTube explainer — 6:34 min narrated version
Source code — static HTML, React components, self-hosted everything

The entire blog is built with Claude Code as an AI pair programmer. The autoencoder training, Manim animations, ElevenLabs voiceover, ffmpeg video assembly, and YouTube upload were all orchestrated from the terminal.

Questions welcome — especially if you spot a mistake in the math.

I built a bird flight simulator you control by flapping your arms (Three.js + MediaPipe)

Mathias Leonhardt — Sun, 19 Apr 2026 06:51:23 +0000

My 6-year-old son wanted a game where you "fly like a real bird." I'm a web developer, not a game dev — but I had Claude Code as a pair programmer.

The result: A 3D bird flight simulator running entirely in your browser. You control the bird by moving your arms in front of your webcam.

How it works

Arms up → glide. Arms down → dive. Flap → climb. No controller, no download.

The webcam feeds into MediaPipe's PoseLandmarker, which detects 33 body landmarks in real-time. I map shoulder-to-wrist vectors to flight controls:

Hands above shoulders → lift increases
Hands below shoulders → pitch down
Rapid up-down motion → flap detected → altitude boost

Also works on mobile with gravity-based gesture controls (tilt to steer).

The tech stack

Component	Tech
3D engine	Three.js + WebGL
Pose detection	MediaPipe PoseLandmarker
Terrain	Custom GLSL shader, 5 texture layers by altitude
Ocean	FFT-based (Tessendorf method) — same math JPEG uses
Trees	InstancedMesh × 100,000
Culling	Octree-based frustum culling
Physics	Custom aerodynamics (lift, drag, stall)

The interesting parts

FFT ocean waves

The ocean uses the same Fourier transform that powers JPEG compression — but in reverse. A Phillips spectrum describes wave energy per frequency, and an inverse FFT turns that into a height map that deforms the water mesh. 512×512 spectral points = 262,144 overlapping wave frequencies per frame.

Octree frustum culling

With 100k trees, you can't render everything. An octree partitions 3D space recursively, and frustum culling discards entire branches that fall outside the camera's view. Result: ~3,000–5,000 trees rendered per frame instead of 100k.

The AI pair programming workflow

Claude Code handled the boilerplate, shader code, and physics implementation. I focused on:

Product decisions ("the bird needs to turn faster")
Visual taste (sunset colors, water transparency)
Architecture ("use an octree, not a flat array")

The hard part wasn't the code — it was knowing what to build. My 6-year-old was surprisingly good at that. His requirements were non-negotiable: "there need to be sharks."

Try it yourself

Play in your browser (needs webcam or use keyboard: WASD + Space)
Source code (MIT license)
Blog post explaining the Fourier math in detail

Built with Three.js, MediaPipe, and a very opinionated 6-year-old. Questions welcome.