README_WIP.md

title: Video Model Studio emoji: 🎥 colorFrom: gray colorTo: gray sdk: gradio sdk_version: 5.15.0 app_file: app.py pinned: true license: apache-2.0 short_description: All-in-one tool for AI video training

🎥 Video Model Studio (VMS)

Presentation

VMS is an all-in-one tool to train LoRA models for various open-source AI video models:

Data collection from various sources
Splitting videos into short single camera shots
Automatic captioning
Training HunyuanVideo or LTX-Video

Similar projects

I wasn't aware of it when I started this project, but there is also this: https://github.com/alisson-anjos/diffusion-pipe-ui

Installation

VMS is built on top of Finetrainers and Gradio, and designed to run as a Hugging Face Space (but you can deploy it elsewhere if you want to).

Full installation at Hugging Face

Easy peasy: create a Space (make sure to use the Gradio type/template), and push the repo. No Docker needed!

Dev mode on Hugging Face

Enable dev mode in the space, then open VSCode in local or remote and run:

pip install -r requirements.txt

As this is not automatic, then click on "Restart" in the space dev mode UI widget.

Full installation somewhere else

I haven't tested it, but you can try to provided Dockerfile

Full installation in local

the full installation requires:

Linux
CUDA 12
Python 3.10

This is because of flash attention, which is defined in the requirements.txt using an URL to download a prebuilt wheel (python bindings for a native library)

./setup.sh

Degraded installation in local

If you cannot meet the requirements, you can:

solution 1: fix requirements.txt to use another prebuilt wheel
solution 2: manually build/install flash attention
solution 3: don't use clip captioning

Here is how to do solution 3:

./setup_no_captions.sh

Run

Running the Gradio app

Note: please make sure you properly define the environment variables for STORAGE_PATH (eg. /data/) and HF_HOME (eg. /data/huggingface/)

python app.py

Running locally

See above remarks about the environment variable.

By default run.sh will store stuff in .data/ (located inside the current working directory):

./run.sh