Diffusion: Noise and Denoise
Destroy a shape into pure noise, then rebuild it. Watch a point cloud dissolve under the forward process, then a score-based sampler walk it back to the data one small step at a time.
Diffusion: Noise and Denoise
A diffusion model learns to reverse a process that gradually destroys data. Here the data is a 2D ring of six clusters. The forward process dissolves it into pure noise; the reverse process rebuilds it. Because the target is a Gaussian mixture, its score is known in closed form, so you can watch the exact sampler walk noise back into structure.
alpha_bar_t is the fraction of the original signal that survives after t steps; it falls from 1 (all data) to nearly 0 (all noise). beta_t is how much fresh noise each step injects. The cosine schedule holds onto signal longer in the middle, which tends to give the reverse sampler more useful gradient to work with.
Dissolving structure into noise
Every point is scaled toward the origin by √(alpha_bar_t) and has Gaussian noise of scale √(1−alpha_bar_t) added. Early on the six clusters are still visible. As t grows the structure washes out until the cloud is an isotropic Gaussian blob that carries no memory of where it started. That endpoint is the same for any dataset, which is exactly why we can start sampling from plain noise.
Walking noise back to data
&epsilon̂ = −√(1−alpha_bar_t)·score, score = ∇ log q(x_t)
Each trajectory starts as one draw from pure noise and runs the update above from t = T−1 down to 0. The score points toward where data is denser; the network's job in a trained model is to estimate it, and equivalently to predict the noise &epsilon̂ that was added. We move only a little per step because a single giant jump would aim at the average of all six clusters and land in the empty middle. Small steps let each sample commit to one cluster, which is why the trails fan out and settle onto the ring.