Systems, apparatus, articles of manufacture, and methods to implement attention-based noise seed selection for video diffusion models are disclosed. An example system selects an initial noise seed for a trained video diffusion model using attention data generated by the model. In some examples, the system generates multiple candidate noise seeds, runs the video diffusion model for a small number of iterations with each candidate, and extracts attention maps from one or more attention layers of the model. For each candidate seed, the system computes a similarity or difference score between the candidate's attention map(s) and those of the other candidates. The seed having the most consistent attention pattern is selected and then used to drive the remaining diffusion iterations to produce a synthetic video. The approach involves no model retraining, incurs low computational overhead, and can display interpretable attention heat maps for user inspection.
Full Text
What is claimed is: