/Method And Device For Generating Video Clip Based On Text Description And Sequence Of Key Points Synthesized By Diffusion Model
Abstract

A method of generating a video clip from a text description is provided. The method may include receiving the text description of the video clip to be generated. The method may include obtaining, based on the received text description, a vector representation of the text description. The method may include obtaining a vector representation of a sequence of key points of the video clip to be generated, by a diffusion motion model based on the obtained vector representation of the text description. The method may include mapping the vector representation of the sequence of key points to a series of key point images of the video clip being generated, wherein each key point image from the series corresponding to a respective frame of the video clip being generated. The method may include generating a frame sequence of the video clip.

Full Text

What is claimed is:

A method of generating a video clip from a text description is provided. The method may include receiving the text description of the video clip to be generated. The method may include obtaining, based on the received text description, a vector representation of the text description. The method may include obtaining a vector representation of a sequence of key points of the video clip to be generated, by a diffusion motion model based on the obtained vector representation of the text description. The method may include mapping the vector representation of the sequence of key points to a series of key point images of the video clip being generated, wherein each key point image from the series corresponding to a respective frame of the video clip being generated. The method may include generating a frame sequence of the video clip.
Timeline
Filed
04/27/2026
Published
09/03/2026
Granted
Not Available
IPC Codes(2)
G06T 11/00:Two-dimensional [2D] image generation
G06T 7/70:Determining position or orientation of objects or cameras (camera calibration G06T 7/80)