|
--- |
|
license: creativeml-openrail-m |
|
library_name: diffusers |
|
inference: true |
|
pipeline_tag: text-to-video |
|
tags: |
|
- text-to-video |
|
- text-to-image |
|
--- |
|
# Text2Video-Zero Model Card - ControlNet Canny GTA-5 Style |
|
|
|
[Text2Video-Zero](https://arxiv.org/abs/2303.13439) is a zero-shot text to video generator. It can perform `zero-shot text-to-video generation`, `Video Instruct Pix2Pix` (instruction-guided video editing), |
|
`text and pose conditional video generation`, `text and canny-edge conditional video generation`, and |
|
`text, canny-edge and dreambooth conditional video generation`. For more information about this work, |
|
please have a look at our [paper](https://arxiv.org/abs/2303.13439) and our demo: [](https://huggingface.co/spaces/PAIR/Text2Video-Zero) |
|
Our [code](https://github.com/Picsart-AI-Research/Text2Video-Zero) works with any StableDiffusion base model. |
|
|
|
This model provides [DreamBooth](https://arxiv.org/abs/2208.12242) weights for the `GTA-5 style` to be used with edge guidance (using [ControlNet](https://arxiv.org/abs/2302.05543)) in text2video zero. |
|
|
|
|
|
## Weights for Text2Video-Zero |
|
We converted the original weights into diffusers and made them usable for [ControlNet](https://arxiv.org/abs/2302.05543) with edge guidance using: https://github.com/lllyasviel/ControlNet/discussions/12. |
|
|
|
|
|
### Model Details |
|
- **Developed by:** Levon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan and Humphrey Shi |
|
- **Model type:** Dreambooth text-to-image and text-to-video generation model with edge control for text2video zero |
|
- **Language(s):** English |
|
- **License:** [The CreativeML OpenRAIL M license](https://huggingface.co/spaces/CompVis/stable-diffusion-license). |
|
- **Model Description:** This is a model for [text2video zero](https://github.com/Picsart-AI-Research/Text2Video-Zero) with edge guidance and gta-5 style. |
|
It can be used also with ControlNet in a text-to-image setup with edge guidance. |
|
- **DreamBoth Keyword:** gtav style |
|
- **Resources for more information:** [GitHub](https://github.com/Picsart-AI-Research/Text2Video-Zero), [Paper](https://arxiv.org/abs/2303.13439), [CIVITAI](https://civitai.com/models/1309/gta5-artwork-diffusion). |
|
- **Cite as:** |
|
|
|
@article{text2video-zero, |
|
title={Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators}, |
|
author={Khachatryan, Levon and Movsisyan, Andranik and Tadevosyan, Vahram and Henschel, Roberto and Wang, Zhangyang and Navasardyan, Shant and Shi, Humphrey}, |
|
journal={arXiv preprint arXiv:2303.13439}, |
|
year={2023} |
|
} |
|
|
|
|
|
|
|
|
|
## Original Weights |
|
The Dreambooth weights for the GTA-5 style were taken from [CIVITAI](https://civitai.com/models/1309/gta5-artwork-diffusion). |
|
|
|
### Model Details |
|
- **Developed by:** Quiet_Joker (Username listed on CIVITAI) |
|
- **Model type:** Dreambooth text-to-image generation model |
|
- **Language(s):** English |
|
- **License:** [The CreativeML OpenRAIL M license](https://huggingface.co/spaces/CompVis/stable-diffusion-license). |
|
- **Model Description:** This is a model that was created using [DreamBooth](https://arxiv.org/abs/2208.12242) to generate images with GTA-5 style, based on text prompts. |
|
- **DreamBoth Keyword:** gtav style |
|
- **Resources for more information:** [CIVITAI](https://civitai.com/models/1309/gta5-artwork-diffusion). |
|
|
|
|
|
## Biases content acknowledgement: |
|
Beware that Text2Video-Zero may output content that reinforces or exacerbates societal biases, as well as realistic faces, pornography, and violence. Text2Video-Zero in this demo is meant only for research purposes. |
|
|
|
|
|
# Citation |
|
@article{text2video-zero, |
|
title={Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators}, |
|
author={Khachatryan, Levon and Movsisyan, Andranik and Tadevosyan, Vahram and Henschel, Roberto and Wang, Zhangyang and Navasardyan, Shant and Shi, Humphrey}, |
|
journal={arXiv preprint arXiv:2303.13439}, |
|
year={2023} |
|
} |
