Methods, systems, and apparatus, including computer programs encoded on a computer storage medium. In one aspect, a method includes receiving a text prompt describing a scene; processing the text prompt using a text encoder neural network to generate a contextual embedding of the text prompt; and processing the contextual embedding using a sequence of generative neural networks to generate a final video depicting the scene.