sschoenmeyer
commited on
Commit
•
bd6180e
1
Parent(s):
8cb21b8
Add more info to README.md
Browse filesAdd additional info from https://huggingface.co/tlwu/sdxl-turbo-onnxruntime/blob/main/README.md
README.md
CHANGED
@@ -1,5 +1,88 @@
|
|
1 |
---
|
|
|
2 |
license: other
|
3 |
license_name: sai-nc-community
|
4 |
-
license_link: LICENSE
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
5 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
+
pipeline_tag: text-to-image
|
3 |
license: other
|
4 |
license_name: sai-nc-community
|
5 |
+
license_link: https://huggingface.co/stabilityai/sdxl-turbo/blob/main/LICENSE.TXT
|
6 |
+
base_model: stabilityai/sdxl-turbo
|
7 |
+
language:
|
8 |
+
- en
|
9 |
+
tags:
|
10 |
+
- stable-diffusion
|
11 |
+
- sdxl
|
12 |
+
- onnxruntime
|
13 |
+
- onnx
|
14 |
+
- text-to-image
|
15 |
---
|
16 |
+
# Stable Diffusion XL Turbo for ONNX Runtime CUDA
|
17 |
+
|
18 |
+
## Introduction
|
19 |
+
|
20 |
+
This repository hosts the optimized onnx models of **SDXL Turbo** to accelerate inference with ONNX Runtime CUDA execution provider for Nvidia GPUs. It cannot run in other providers like CPU or DirectML.
|
21 |
+
|
22 |
+
The models are generated by [Olive](https://github.com/microsoft/Olive/tree/main/examples/stable_diffusion) with command like the following:
|
23 |
+
```
|
24 |
+
python stable_diffusion_xl.py --provider cuda --model_id stabilityai/sdxl-turbo --optimize --use_fp16_fixed_vae
|
25 |
+
```
|
26 |
+
|
27 |
+
See the [usage instructions](#usage-example) for how to run the SDXL pipeline with the ONNX files hosted in this repository.
|
28 |
+
|
29 |
+
## Model Description
|
30 |
+
|
31 |
+
- **Developed by:** Stability AI
|
32 |
+
- **Model type:** Diffusion-based text-to-image generative model
|
33 |
+
- **License:** [STABILITY AI NON-COMMERCIAL RESEARCH COMMUNITY LICENSE](https://huggingface.co/stabilityai/sd-turbo/blob/main/LICENSE)
|
34 |
+
- **Model Description:** This is a conversion of the [SDXL-Turbo](https://huggingface.co/stabilityai/sdxl-turbo) model for [ONNX Runtime](https://github.com/microsoft/onnxruntime) inference with CUDA execution provider.
|
35 |
+
|
36 |
+
The VAE decoder is converted from [sdxl-vae-fp16-fix](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix). There are slight discrepancies between its output and that of the original VAE, but the decoded images should be [close enough for most purposes](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/discussions/7#64c5c0f8e2e5c94bd04eaa80).
|
37 |
+
|
38 |
+
## Usage Example
|
39 |
+
|
40 |
+
Following the [demo instructions](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/models/stable_diffusion/README.md#run-demo-with-docker). Example steps:
|
41 |
+
|
42 |
+
0. Install nvidia-docker using these [instructions](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).
|
43 |
+
|
44 |
+
1. Clone onnxruntime repository.
|
45 |
+
```shell
|
46 |
+
git clone https://github.com/microsoft/onnxruntime
|
47 |
+
cd onnxruntime
|
48 |
+
```
|
49 |
+
|
50 |
+
2. Download the SDXL ONNX files from this repo
|
51 |
+
```shell
|
52 |
+
git lfs install
|
53 |
+
git clone https://huggingface.co/tlwu/sdxl-turbo-onnxruntime
|
54 |
+
```
|
55 |
+
|
56 |
+
3. Launch the docker
|
57 |
+
```shell
|
58 |
+
docker run --rm -it --gpus all -v $PWD:/workspace nvcr.io/nvidia/pytorch:23.10-py3 /bin/bash
|
59 |
+
```
|
60 |
+
|
61 |
+
4. Build ONNX Runtime from source
|
62 |
+
```shell
|
63 |
+
export CUDACXX=/usr/local/cuda-12.2/bin/nvcc
|
64 |
+
git config --global --add safe.directory '*'
|
65 |
+
sh build.sh --config Release --build_shared_lib --parallel --use_cuda --cuda_version 12.2 \
|
66 |
+
--cuda_home /usr/local/cuda-12.2 --cudnn_home /usr/lib/x86_64-linux-gnu/ --build_wheel --skip_tests \
|
67 |
+
--use_tensorrt --tensorrt_home /usr/src/tensorrt \
|
68 |
+
--cmake_extra_defines onnxruntime_BUILD_UNIT_TESTS=OFF \
|
69 |
+
--cmake_extra_defines CMAKE_CUDA_ARCHITECTURES=80 \
|
70 |
+
--allow_running_as_root
|
71 |
+
python3 -m pip install build/Linux/Release/dist/onnxruntime_gpu-*-cp310-cp310-linux_x86_64.whl --force-reinstall
|
72 |
+
```
|
73 |
+
|
74 |
+
If the GPU is not A100, change CMAKE_CUDA_ARCHITECTURES=80 in the command line according to the GPU compute capacity (like 89 for RTX 4090, or 86 for RTX 3090). If your machine has less than 64GB memory, replace --parallel by --parallel 4 --nvcc_threads 1 to avoid out of memory.
|
75 |
+
5. Install libraries and requirements
|
76 |
+
```shell
|
77 |
+
python3 -m pip install --upgrade pip
|
78 |
+
cd /workspace/onnxruntime/python/tools/transformers/models/stable_diffusion
|
79 |
+
python3 -m pip install -r requirements-cuda12.txt
|
80 |
+
python3 -m pip install --upgrade polygraphy onnx-graphsurgeon --extra-index-url https://pypi.ngc.nvidia.com
|
81 |
+
```
|
82 |
+
6. Perform ONNX Runtime optimized inference
|
83 |
+
```shell
|
84 |
+
python3 demo_txt2img_xl.py \
|
85 |
+
"starry night over Golden Gate Bridge by van gogh" \
|
86 |
+
--version xl-turbo \
|
87 |
+
--engine-dir /workspace/sdxl-turbo-onnxruntime
|
88 |
+
```
|