Spaces:
Running
Running
Commit
·
11ef19f
1
Parent(s):
e020146
temporary readme
Browse files- README.md +1 -81
- README_WIP.md +97 -0
README.md
CHANGED
@@ -13,84 +13,4 @@ short_description: All-in-one tool for AI video training
|
|
13 |
|
14 |
# 🎥 Video Model Studio (VMS)
|
15 |
|
16 |
-
|
17 |
-
|
18 |
-
VMS is an all-in-one tool to train LoRA models for various open-source AI video models:
|
19 |
-
|
20 |
-
- Data collection from various sources
|
21 |
-
- Splitting videos into short single camera shots
|
22 |
-
- Automatic captioning
|
23 |
-
- Training HunyuanVideo or LTX-Video
|
24 |
-
|
25 |
-
## Similar projects
|
26 |
-
|
27 |
-
I wasn't aware of it when I started this project,
|
28 |
-
but there is also this: https://github.com/alisson-anjos/diffusion-pipe-ui
|
29 |
-
|
30 |
-
## Installation
|
31 |
-
|
32 |
-
VMS is built on top of Finetrainers and Gradio, and designed to run as a Hugging Face Space (but you can deploy it elsewhere if you want to).
|
33 |
-
|
34 |
-
### Full installation at Hugging Face
|
35 |
-
|
36 |
-
Easy peasy: create a Space (make sure to use the `Gradio` type/template), and push the repo. No Docker needed!
|
37 |
-
|
38 |
-
### Dev mode on Hugging Face
|
39 |
-
|
40 |
-
Enable dev mode in the space, then open VSCode in local or remote and run:
|
41 |
-
|
42 |
-
```
|
43 |
-
pip install -r requirements.txt
|
44 |
-
```
|
45 |
-
|
46 |
-
As this is not automatic, then click on "Restart" in the space dev mode UI widget.
|
47 |
-
|
48 |
-
### Full installation somewhere else
|
49 |
-
|
50 |
-
I haven't tested it, but you can try to provided Dockerfile
|
51 |
-
|
52 |
-
### Full installation in local
|
53 |
-
|
54 |
-
the full installation requires:
|
55 |
-
- Linux
|
56 |
-
- CUDA 12
|
57 |
-
- Python 3.10
|
58 |
-
|
59 |
-
This is because of flash attention, which is defined in the `requirements.txt` using an URL to download a prebuilt wheel (python bindings for a native library)
|
60 |
-
|
61 |
-
```bash
|
62 |
-
./setup.sh
|
63 |
-
```
|
64 |
-
|
65 |
-
### Degraded installation in local
|
66 |
-
|
67 |
-
If you cannot meet the requirements, you can:
|
68 |
-
|
69 |
-
- solution 1: fix requirements.txt to use another prebuilt wheel
|
70 |
-
- solution 2: manually build/install flash attention
|
71 |
-
- solution 3: don't use clip captioning
|
72 |
-
|
73 |
-
Here is how to do solution 3:
|
74 |
-
```bash
|
75 |
-
./setup_no_captions.sh
|
76 |
-
```
|
77 |
-
|
78 |
-
## Run
|
79 |
-
|
80 |
-
### Running the Gradio app
|
81 |
-
|
82 |
-
Note: please make sure you properly define the environment variables for `STORAGE_PATH` (eg. `/data/`) and `HF_HOME` (eg. `/data/huggingface/`)
|
83 |
-
|
84 |
-
```bash
|
85 |
-
python app.py
|
86 |
-
```
|
87 |
-
|
88 |
-
### Running locally
|
89 |
-
|
90 |
-
See above remarks about the environment variable.
|
91 |
-
|
92 |
-
By default `run.sh` will store stuff in `.data/` (located inside the current working directory):
|
93 |
-
|
94 |
-
```bash
|
95 |
-
./run.sh
|
96 |
-
```
|
|
|
13 |
|
14 |
# 🎥 Video Model Studio (VMS)
|
15 |
|
16 |
+
This project is a work in progress, not all features are working yet (there are some issue with the automatic captioning).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
README_WIP.md
ADDED
@@ -0,0 +1,97 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
README_WIP.md
|
2 |
+
---
|
3 |
+
title: Video Model Studio
|
4 |
+
emoji: 🎥
|
5 |
+
colorFrom: gray
|
6 |
+
colorTo: gray
|
7 |
+
sdk: gradio
|
8 |
+
sdk_version: 5.15.0
|
9 |
+
app_file: app.py
|
10 |
+
pinned: true
|
11 |
+
license: apache-2.0
|
12 |
+
short_description: All-in-one tool for AI video training
|
13 |
+
---
|
14 |
+
|
15 |
+
# 🎥 Video Model Studio (VMS)
|
16 |
+
|
17 |
+
## Presentation
|
18 |
+
|
19 |
+
VMS is an all-in-one tool to train LoRA models for various open-source AI video models:
|
20 |
+
|
21 |
+
- Data collection from various sources
|
22 |
+
- Splitting videos into short single camera shots
|
23 |
+
- Automatic captioning
|
24 |
+
- Training HunyuanVideo or LTX-Video
|
25 |
+
|
26 |
+
## Similar projects
|
27 |
+
|
28 |
+
I wasn't aware of it when I started this project,
|
29 |
+
but there is also this: https://github.com/alisson-anjos/diffusion-pipe-ui
|
30 |
+
|
31 |
+
## Installation
|
32 |
+
|
33 |
+
VMS is built on top of Finetrainers and Gradio, and designed to run as a Hugging Face Space (but you can deploy it elsewhere if you want to).
|
34 |
+
|
35 |
+
### Full installation at Hugging Face
|
36 |
+
|
37 |
+
Easy peasy: create a Space (make sure to use the `Gradio` type/template), and push the repo. No Docker needed!
|
38 |
+
|
39 |
+
### Dev mode on Hugging Face
|
40 |
+
|
41 |
+
Enable dev mode in the space, then open VSCode in local or remote and run:
|
42 |
+
|
43 |
+
```
|
44 |
+
pip install -r requirements.txt
|
45 |
+
```
|
46 |
+
|
47 |
+
As this is not automatic, then click on "Restart" in the space dev mode UI widget.
|
48 |
+
|
49 |
+
### Full installation somewhere else
|
50 |
+
|
51 |
+
I haven't tested it, but you can try to provided Dockerfile
|
52 |
+
|
53 |
+
### Full installation in local
|
54 |
+
|
55 |
+
the full installation requires:
|
56 |
+
- Linux
|
57 |
+
- CUDA 12
|
58 |
+
- Python 3.10
|
59 |
+
|
60 |
+
This is because of flash attention, which is defined in the `requirements.txt` using an URL to download a prebuilt wheel (python bindings for a native library)
|
61 |
+
|
62 |
+
```bash
|
63 |
+
./setup.sh
|
64 |
+
```
|
65 |
+
|
66 |
+
### Degraded installation in local
|
67 |
+
|
68 |
+
If you cannot meet the requirements, you can:
|
69 |
+
|
70 |
+
- solution 1: fix requirements.txt to use another prebuilt wheel
|
71 |
+
- solution 2: manually build/install flash attention
|
72 |
+
- solution 3: don't use clip captioning
|
73 |
+
|
74 |
+
Here is how to do solution 3:
|
75 |
+
```bash
|
76 |
+
./setup_no_captions.sh
|
77 |
+
```
|
78 |
+
|
79 |
+
## Run
|
80 |
+
|
81 |
+
### Running the Gradio app
|
82 |
+
|
83 |
+
Note: please make sure you properly define the environment variables for `STORAGE_PATH` (eg. `/data/`) and `HF_HOME` (eg. `/data/huggingface/`)
|
84 |
+
|
85 |
+
```bash
|
86 |
+
python app.py
|
87 |
+
```
|
88 |
+
|
89 |
+
### Running locally
|
90 |
+
|
91 |
+
See above remarks about the environment variable.
|
92 |
+
|
93 |
+
By default `run.sh` will store stuff in `.data/` (located inside the current working directory):
|
94 |
+
|
95 |
+
```bash
|
96 |
+
./run.sh
|
97 |
+
```
|