CogView3-Plus-3B

📄 Read in English | 🤗 Hugging Face Space | 🌐 Github | 📜 arxiv

📍 前往清言和 API平台体验更大规模的商业版视频生成模型。

推理要求和模型介绍

该模型是 CogView3 的 DiT 版本图像生成模型，支持从 512 到 2048 范围内的图像生成。

分辨率: 长宽均需满足 512px - 2048px 之间，均需被32整除。
推理速度: 1s / step (在 A100 进行测试)
精度: BF16 / FP32 (不支持FP16，会出现溢出导致纯黑图片)

显存消耗

我们在A100设备上对几个常见分辨率的显存消耗进行了测试，batchsize=1, BF16, 如下表所示：

分辨率	enable_model_cpu_offload OFF	enable_model_cpu_offload ON
512 * 512	19GB	11GB
720 * 480	20GB	11GB
1024 * 1024	23GB	11GB
1280 * 720	24GB	11GB
2048 * 2048	25GB	11GB

快速开始

首先，确保从源代码安装diffusers库。

pip install git+https://github.com/huggingface/diffusers.git

接着，运行以下代码：

from diffusers import CogView3PlusPipeline
import torch

pipe = CogView3PlusPipeline.from_pretrained("THUDM/CogView3-Plus-3B", torch_dtype=torch.float16).to("cuda")

# Open it for reduce GPU memory usage
pipe.enable_model_cpu_offload()
pipe.vae.enable_slicing()
pipe.vae.enable_tiling()

prompt = "A vibrant cherry red sports car sits proudly under the gleaming sun, its polished exterior smooth and flawless, casting a mirror-like reflection. The car features a low, aerodynamic body, angular headlights that gaze forward like predatory eyes, and a set of black, high-gloss racing rims that contrast starkly with the red. A subtle hint of chrome embellishes the grille and exhaust, while the tinted windows suggest a luxurious and private interior. The scene conveys a sense of speed and elegance, the car appearing as if it's about to burst into a sprint along a coastal road, with the ocean's azure waves crashing in the background."
image = pipe(
    prompt=prompt,
    guidance_scale=7.0,
    num_images_per_prompt=1,
    num_inference_steps=50,
    width=1024,
    height=1024,
).images[0]

image.save("cogview3.png")

更多内容以及下载 SAT 原始权重，请前往我们的 github。

引用

🌟 如果您发现我们的工作有所帮助，欢迎引用我们的文章，留下宝贵的stars

@article{zheng2024cogview3,
  title={Cogview3: Finer and faster text-to-image generation via relay diffusion},
  author={Zheng, Wendi and Teng, Jiayan and Yang, Zhuoyi and Wang, Weihan and Chen, Jidong and Gu, Xiaotao and Dong, Yuxiao and Ding, Ming and Tang, Jie},
  journal={arXiv preprint arXiv:2403.05121},
  year={2024}
}

模型协议

该模型基于 Apache 2.0 License 协议发布。