|
--- |
|
license: apache-2.0 |
|
datasets: |
|
- TempoFunk/webvid-10M |
|
language: |
|
- en |
|
tags: |
|
- text-to-video |
|
base_model: |
|
- ali-vilab/text-to-video-ms-1.7b |
|
--- |
|
# caT text to video |
|
|
|
Conditionally augmented text-to-video model. Uses pre-trained weights from modelscope text-to-video model, augmented with temporal conditioning transformers to extend generated clips and create a smooth transition between them. |
|
Supports prompt interpolation as well to change scenes during clip extensions. |
|
|
|
This model was trained at home as a hobby. |
|
|
|
Do not expect high quality samples. |
|
|
|
## Installation |
|
|
|
### Clone the Repository |
|
|
|
```bash |
|
git clone https://github.com/motexture/caT-text-to-video.git |
|
cd caT-text-to-video |
|
python3 -m venv venv |
|
source venv/bin/activate # On Windows use `venv\Scripts\activate` |
|
pip install -r requirements.txt |
|
python3 run.py |
|
``` |
|
|
|
Visit the provided URL in your browser to interact with the interface and start generating videos. |
|
|
|
Note: Ensure that you are on the latest commit, as the positional encodings have been updated compared to the initial models. |
|
|
|
<video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/64a86f7d03835e13f95c3687/qr-NXxvmkquF_mMlx_5P-.mp4"></video> |
|
<video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/64a86f7d03835e13f95c3687/32B1RPHAmieomeXWp2XvC.mp4"></video> |
|
<video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/64a86f7d03835e13f95c3687/40KrBvzMf8DmPO8VvATfC.mp4"></video> |
|
<video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/64a86f7d03835e13f95c3687/SEtFOILcwwNT4M8mXMNWt.mp4"></video> |
|
|
