Google DiffusionGemma: Technical report explains the fast text diffusion model

Google DeepMind has published a technical report for DiffusionGemma, a model released in mid-June that generates text by refining blocks of 256 tokens in parallel, akin to image generation from noise. On Nvidia's H100 accelerator, it achieves about 1,500 tokens per second. Instead of training from scratch, the existing Gemma-4-26B-A4B model was converted into a diffusion model using less than 10% of the original training token budget. The training involved two stages: first, learning to reconstruct noisy text blocks, then a combined phase of reinforcement learning and sampler distillation, which Google calls SD·RL. The report states that this improves reasoning benchmarks by an average of ten points while nearly quadrupling tokens per step, and responses are about 50% shorter. DiffusionGemma's bidirectional processing allows self-correction before output, enabling it to solve Sudoku puzzles with 85% accuracy after fine-tuning, whereas the base model fails. It also retains the ability to generate word-by-word, letting users switch modes. However, absolute performance is below the autoregressive model due to factors like post-hoc conversion and a short training phase aimed at speed. The model occasionally falls into repetition loops and may fail to close reasoning sections, lowering benchmark scores. The speed advantage mainly applies in single-user scenarios, as autoregressive models catch up in throughput beyond 32 concurrent requests. Google positions DiffusionGemma as experimental, aiming to accelerate research and provide a basis for specialized adaptations. It is already used by startup Interfaze for multilingual speech recognition and in a project for interactive radiology reports. The model is available under Apache 2.0 on Hugging Face, and a precursor, Gemini Diffusion, was demoed in May 2025.