Creativity of Diffusion Models: A Mathematical Explanation from Google Research

Researchers from Google Research presented a study explaining the nature of creativity in diffusion models. They showed that a model's ability to generate new data, rather than merely copying training examples, is a mathematically predictable consequence of how neural networks are trained. During training, the neural network naturally smooths the score function—a vector field guiding the denoising process. This smoothing, called score smoothing, causes particles to stop at intermediate positions instead of exactly reproducing training points, creating new, plausible samples. In high-dimensional spaces, the effect appears only along tangent directions to the latent data manifold, without slowing convergence to the manifold itself. Thus, the model achieves a balance between quality and novelty: images are both realistic and original. The work was presented at ICLR 2026.