Text Diffusion — Brendon Dillon, Google DeepMind
Google DeepMind is advancing text diffusion as a forward-looking alternative to autoregressive next-token generation.
“we had a research demo release 1 year ago now called Gemini diffusion which was a variant of a Gemini model which did text diffusion instead of auto aggressive next token generation”
A DeepMind research scientist explained text diffusion, which iteratively denoises random tokens into clean text rather than generating left-to-right, and noted the Gemini Diffusion research preview shipped to ~100k people a year ago. It matters as an emerging non-autoregressive generation paradigm, but the talk is exploratory with hints of upcoming developments rather than a concrete new launch.