Create README.md

README.md

@@ -0,0 +1,101 @@
+---
+datasets:
+- diffusers/tuxemon
+base_model:
+- tencent/HunyuanVideo
+library_name: diffusers
+widget:
+  - text: Style of snomexut, A fluffy, brown-haired tuxemon with a curious expression, sporting an orange and black striped harness and unique patterned tail, explores its surroundings.
+    output:
+      url: video_0.mp4
+  - text: Style of snomexut, A vibrant yellow Tuxemon, wielding a fiery blue staff, joyfully manipulates swirling flames around its cheerful form.
+    output:
+      url: video_1.mp4
+  - text: Style of snomexut, A vibrant yellow Tuxemon, wielding a fiery blue staff, joyfully manipulates swirling flames around its cheerful form.
+    output:
+      url: video_2.mp4
+  - text: Style of snomexut, A menacing blue tuxemon with sharp claws and fierce fangs roars against a cosmic backdrop, its body adorned with glowing orbs and sharp pink details.
+    output:
+      url: video_3.mp4
+tags:
+- text-to-video
+- diffusers-training
+- diffusers
+- lora
+- hunyuan_video
+- template:sd-lora
+---
+
+# LoRA Finetune
+
+<Gallery />
+
+## Model description
+
+This is a lora finetune of https://huggingface.co/hunyuanvideo-community/HunyuanVideo.
+
+The model was trained using [`finetrainers`](https://github.com/a-r-r-o-w/finetrainers). Training progress over time (4000 steps, 6000 steps, 8000 steps, 10000 steps):
+
+<video src="output.mp4"> Your browser does not support the video tag. </video>
+
+## Download model
+
+[Download LoRA](pytorch_lora_weights.safetensors) in the Files & Versions tab.
+
+## Usage
+
+Requires the [🧨 Diffusers library](https://github.com/huggingface/diffusers) installed.
+
+Trigger phrase: `"Style of snomexut,"`
+
+```python
+import torch
+from diffusers import HunyuanVideoPipeline, HunyuanVideoTransformer3DModel
+from diffusers.utils import export_to_video
+
+model_id = "hunyuanvideo-community/HunyuanVideo"
+transformer = HunyuanVideoTransformer3DModel.from_pretrained(
+    model_id, subfolder="transformer", torch_dtype=torch.bfloat16
+)
+pipe = HunyuanVideoPipeline.from_pretrained(model_id, transformer=transformer, torch_dtype=torch.float16)
+pipe.vae.enable_tiling()
+pipe.to("cuda")
+
+pipe.load_lora_weights("a-r-r-o-w/HunyuanVideo-tuxemons", adapter_name="hunyuanvideo-lora")
+pipe.set_adapters("hunyuanvideo-lora", 1.2)
+
+output = pipe(
+    prompt="Style of snomexut, a cat-like Tuxemon creature walks in alien-world grass, and observes its surroundings.",
+    height=768,
+    width=768,
+    num_frames=33,
+    num_inference_steps=30,
+    generator=torch.Generator().manual_seed(73),
+).frames[0]
+export_to_video(output, "output-tuxemon.mp4", fps=15)
+```
+
+<video src="output-tuxemon.mp4"> Your browser does not support the video tag. </video>
+
+For more details, including weighting, merging and fusing LoRAs, check the [documentation](https://huggingface.co/docs/diffusers/main/en/using-diffusers/loading_adapters) on loading LoRAs in diffusers.
+
+<details>
+<summary> Training params </summary>
+
+Dataset contained 250 images from `diffusers/tuxemon` and 150 samples generated from a Flux LoRA trained on the same dataset.
+
+```
+id_token: "Style of snomexut,"
+target_modules: to_q to_k to_v to_out.0 add_q_proj add_k_proj add_v_proj to_add_out
+lr: 1e-5
+training_steps: 10000
+video_resolution_buckets: 1x768x768 1x512x512
+rank: 128
+lora_alpha: 128
+optimizer: adamw
+weight_decay: 0.01
+flow_weighting_scheme: logit_normal
+flow_shift: 7.0
+validation_prompts: $ID_TOKEN A fluffy, brown-haired tuxemon with a curious expression, sporting an orange and black striped harness and unique patterned tail, explores its surroundings.@@@1x768x768:::$ID_TOKEN A curious yellow Tuxemon bird with large, expressive eyes and a distinctive golden beak, set against a starry black background, looks gently to its left.@@@1x768x768:::$ID_TOKEN A menacing Tuxemon with the appearance of a green and white aquatic shark, featuring sharp teeth and red eyes, emerges from the dark abyss, ready to unleash its fury.@@@1x768x768:::$ID_TOKEN A cheerful light blue tuxemon with four tentacle-like arms and sharp ears smiles playfully against a dark background.@@@1x768x768:::$ID_TOKEN A cheerful white Tuxemon with large ears and a playful expression joyfully floats in a starry outer space backdrop.@@@1x768x768:::$ID_TOKEN A finely detailed, silver-toned pin shaped like a futuristic double-barreled gun, adorned with orange textured panels, popularly associated with the Tuxemon universe.@@@1x768x768:::$ID_TOKEN A menacing blue tuxemon with sharp claws and fierce fangs roars against a cosmic backdrop, its body adorned with glowing orbs and sharp pink details.@@@1x768x768:::$ID_TOKEN A vibrant yellow Tuxemon, wielding a fiery blue staff, joyfully manipulates swirling flames around its cheerful form.@@@1x768x768:::$ID_TOKEN A fluffy, brown-haired tuxemon with a curious expression, sporting an orange and black striped harness and unique patterned tail, explores its surroundings.@@@49x768x768:::$ID_TOKEN A curious yellow Tuxemon bird with large, expressive eyes and a distinctive golden beak, set against a starry black background, looks gently to its left.@@@49x768x768:::$ID_TOKEN A menacing Tuxemon with the appearance of a green and white aquatic shark, featuring sharp teeth and red eyes, emerges from the dark abyss, ready to unleash its fury.@@@49x768x768:::$ID_TOKEN A cheerful light blue tuxemon with four tentacle-like arms and sharp ears smiles playfully against a dark background.@@@49x768x768:::$ID_TOKEN A finely detailed, silver-toned pin shaped like a futuristic double-barreled gun, adorned with orange textured panels, popularly associated with the Tuxemon universe.@@@49x768x768:::$ID_TOKEN A menacing blue tuxemon with sharp claws and fierce fangs roars against a cosmic backdrop, its body adorned with glowing orbs and sharp pink details.@@@49x768x768:::$ID_TOKEN A vibrant yellow Tuxemon, wielding a fiery blue staff, joyfully manipulates swirling flames around its cheerful form.@@@49x768x768
+```
+</details>