GANs (yesterday) make images with two networks fighting. Diffusion models — what actually powers Stable Diffusion, DALL·E, and Midjourney — do it differently and far more stably: they learn to turn noise into images, one denoising step at a time. Here's the process, visualized.
🌫️ Scrub the noise → image chain: https://dev48v.infy.uk/dl/day15-diffusion.html
Two processes
- Forward (fixed, no learning): take a real image and add a little Gaussian noise, over and over, for T steps — until it's pure static.
- Reverse (the learned part): train a network to undo one noise step. Run it backward from pure noise and an image emerges.
What the network actually predicts
Not the image — the noise that was added at each step. Subtract the predicted noise, repeat. The training loss is just MSE between predicted and actual noise. That simplicity is why diffusion trains so much more stably than GANs (no adversarial game, no mode collapse).
How you steer it
Text-to-image conditions the denoiser on a prompt (via cross-attention), and "latent diffusion" runs the whole thing in a compressed latent space for speed — that's the Stable Diffusion recipe.
GANs vs diffusion
GANs: fast to sample, tricky to train. Diffusion: stable to train, slower to sample (many steps) — though that gap keeps closing.
🔨 Full picture (forward noising → train to predict noise → sample by denoising) on the page: https://dev48v.infy.uk/dl/day15-diffusion.html
Part of DeepLearningFromZero. 🌐 https://dev48v.infy.uk
Top comments (0)