Skip to content

Diffusion models basics

  • ML fundamentals
  • Specialist topic or differentiator
  • About 2 h to learn
  • 3 vetted resources
Build a plan that includes this

What this topic covers

Diffusion models power most image, video and audio generation, so generalist AI roles sometimes ask how they work, usually as “explain Stable Diffusion” or in a text-to-image system design. You aren’t expected to derive the ELBO. A strong answer explains the forward noising process and the learned reverse process, why the network predicts noise, how text conditioning enters through cross-attention, why latent diffusion with a VAE makes it affordable, what classifier-free guidance trades off, and why sampling is slow and how fewer-step samplers and distillation speed it up.

Builds on

You’re ready when you can

What interviewers check, from the resources below.

  • Name the three components of Stable Diffusion and what each does
  • Explain how training examples are made by adding noise
  • Explain why diffusing in latent space is cheaper
  • Explain how text conditions the U-Net through cross-attention
  • Write the closed-form noising step q(x_t | x_0) in code
  • Explain why the training loss is a simple MSE on predicted noise
  • Explain why sampling needs many sequential steps
  • Explain classifier-free guidance and what the guidance scale trades off
  • Name two ways to cut sampling steps
  • Explain why the simplified loss works in practice

What to study, in order

Each link opened and checked by hand; the date says when.

  1. Start here

    The Illustrated Stable Diffusion (opens in a new tab)

    Jay Alammar · Jay Alammar · 40 min · Free · checked 22 Oct 2026

    Why
    The easiest way to explain a text-to-image model at the level interviews want: the parts, how noise is removed step by step, and where the text comes in.
    What to read
    Read every section: “The Components of Stable Diffusion”, “What is Diffusion Anyway?”, “Speed Boost: Diffusion on Compressed (Latent) Data”, the CLIP text encoder part and “Feeding Text Information Into The Image Generation Process”.
    What to skip
    Skip the resources and citation sections.

  2. Free alternative

    The Annotated Diffusion Model (opens in a new tab)

    Hugging Face · Niels Rogge, Kashif Rasul · 75 min · Free · checked 22 Oct 2026

    Why
    A free, runnable walk-through of DDPM in PyTorch. Writing the training loss yourself is the quickest way to stop hand-waving about diffusion.
    What to read
    Read “What is a diffusion model?”, “Defining an objective function”, “Defining the forward diffusion process”, “Train the model” and “Sampling”. Run the notebook on Fashion-MNIST for a few epochs.
    What to skip
    Skim the network helper blocks (ResNet block, attention, group norm); you don’t need to rebuild the U-Net.

  3. Go deeper

    What are Diffusion Models? (opens in a new tab)

    Lil'Log · Lilian Weng · 60 min · Free · checked 22 Oct 2026

    Why
    The reference most engineers use when they need the maths behind diffusion, plus guidance and speed-ups, in one place.
    What to read
    Read “Forward diffusion process”, “Reverse diffusion process”, “Simplification” of the training loss, then “Conditioned Generation” (classifier-free guidance) and “Speed up Diffusion Models” (fewer steps, distillation, latent space).
    What to skip
    Skip the Langevin and score-network connections, the β_t and variance parameterisation sections, and the resolution scaling part.

Companies that test ML fundamentals most

Put Diffusion models basics in your plan

Tell us your interview date and hours a day; it lands on the right day, in the right order.

Build my plan