👋 HyVideo

This project is a first step in integrating HunyuanVideo into Diffusers.

All credit go to Tencent for the original HunyuanVideo project.

Thank you to Huggingface for the Diffusers library. Special shout-out to @a-r-r-o-w for his work on integrating HunyuanVideo.

The License is inherted from HunyuanVideo.

This library is provided as-is and will be superseded by the official release of HunyuanVideo via Diffusers. Please help out if you can on the PR.

Installation

pip install git+https://github.com/ollanoinc/hyvideo.git

You will also need to install flash-attn for now.

Usage

Please note that you need at least 80GB VRAM to run this pipeline. CPU offloading is having issues at the moment (PRs welcome!).

import os
from hyvideo.diffusion.pipelines.pipeline_hunyuan_video import HunyuanVideoPipeline
from hyvideo.modules.models import HYVideoDiffusionTransformer
from hyvideo.vae.autoencoder_kl_causal_3d import AutoencoderKLCausal3D
import diffusers.pipelines

from types import ModuleType
def set_nested_attr(current, path, value):
    parts = path.split('.')
    for part in parts[:-1]:
        if not hasattr(current, part):
            setattr(current, part, ModuleType(part))
        current = getattr(current, part)
    setattr(current, parts[-1], value)

set_nested_attr(diffusers.pipelines, 'hyvideo.HunyuanVideoPipeline', HunyuanVideoPipeline)
set_nested_attr(diffusers.pipelines, 'hyvideo.HYVideoDiffusionTransformer', HYVideoDiffusionTransformer)
set_nested_attr(diffusers.pipelines, 'hyvideo.AutoencoderKLCausal3D', AutoencoderKLCausal3D)

os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = "1"

pipe = HunyuanVideoPipeline.from_pretrained(
    "magespace/hyvideo-diffusers",
    torch_dtype=torch.bfloat16
).to("cuda")
pipe.vae.enable_tiling()

Then running:

prompt = "Close-up, A little girl wearing a red hoodie in winter strikes a match. The sky is dark, there is a layer of snow on the ground, and it is still snowing lightly. The flame of the match flickers, illuminating the girl's face intermittently."

result = pipe(prompt)

Post-processing:

import PIL.Image
from diffusers.utils import export_to_video

output = result.videos[0].permute(1, 2, 3, 0).detach().cpu().numpy()
output = (output * 255).clip(0, 255).astype("uint8")
output = [PIL.Image.fromarray(x) for x in output]

export_to_video(output, "output.mp4", fps=24)

For faster generation, you can optimize the transformer with torch.compile. Additionally, increasing shift in the scheduler can allow for lower step values as shown in the original paper.

Generation time is quadratic with the number of pixels, so reducing the height and width and decreasing the number of frames will drastically speed up generation at the price of video quality.