sschoenmeyer commited on
Commit
bd6180e
1 Parent(s): 8cb21b8

Add more info to README.md

Browse files

Add additional info from https://huggingface.co/tlwu/sdxl-turbo-onnxruntime/blob/main/README.md

Files changed (1) hide show
  1. README.md +84 -1
README.md CHANGED
@@ -1,5 +1,88 @@
1
  ---
 
2
  license: other
3
  license_name: sai-nc-community
4
- license_link: LICENSE
 
 
 
 
 
 
 
 
 
5
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ pipeline_tag: text-to-image
3
  license: other
4
  license_name: sai-nc-community
5
+ license_link: https://huggingface.co/stabilityai/sdxl-turbo/blob/main/LICENSE.TXT
6
+ base_model: stabilityai/sdxl-turbo
7
+ language:
8
+ - en
9
+ tags:
10
+ - stable-diffusion
11
+ - sdxl
12
+ - onnxruntime
13
+ - onnx
14
+ - text-to-image
15
  ---
16
+ # Stable Diffusion XL Turbo for ONNX Runtime CUDA
17
+
18
+ ## Introduction
19
+
20
+ This repository hosts the optimized onnx models of **SDXL Turbo** to accelerate inference with ONNX Runtime CUDA execution provider for Nvidia GPUs. It cannot run in other providers like CPU or DirectML.
21
+
22
+ The models are generated by [Olive](https://github.com/microsoft/Olive/tree/main/examples/stable_diffusion) with command like the following:
23
+ ```
24
+ python stable_diffusion_xl.py --provider cuda --model_id stabilityai/sdxl-turbo --optimize --use_fp16_fixed_vae
25
+ ```
26
+
27
+ See the [usage instructions](#usage-example) for how to run the SDXL pipeline with the ONNX files hosted in this repository.
28
+
29
+ ## Model Description
30
+
31
+ - **Developed by:** Stability AI
32
+ - **Model type:** Diffusion-based text-to-image generative model
33
+ - **License:** [STABILITY AI NON-COMMERCIAL RESEARCH COMMUNITY LICENSE](https://huggingface.co/stabilityai/sd-turbo/blob/main/LICENSE)
34
+ - **Model Description:** This is a conversion of the [SDXL-Turbo](https://huggingface.co/stabilityai/sdxl-turbo) model for [ONNX Runtime](https://github.com/microsoft/onnxruntime) inference with CUDA execution provider.
35
+
36
+ The VAE decoder is converted from [sdxl-vae-fp16-fix](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix). There are slight discrepancies between its output and that of the original VAE, but the decoded images should be [close enough for most purposes](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/discussions/7#64c5c0f8e2e5c94bd04eaa80).
37
+
38
+ ## Usage Example
39
+
40
+ Following the [demo instructions](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/models/stable_diffusion/README.md#run-demo-with-docker). Example steps:
41
+
42
+ 0. Install nvidia-docker using these [instructions](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).
43
+
44
+ 1. Clone onnxruntime repository.
45
+ ```shell
46
+ git clone https://github.com/microsoft/onnxruntime
47
+ cd onnxruntime
48
+ ```
49
+
50
+ 2. Download the SDXL ONNX files from this repo
51
+ ```shell
52
+ git lfs install
53
+ git clone https://huggingface.co/tlwu/sdxl-turbo-onnxruntime
54
+ ```
55
+
56
+ 3. Launch the docker
57
+ ```shell
58
+ docker run --rm -it --gpus all -v $PWD:/workspace nvcr.io/nvidia/pytorch:23.10-py3 /bin/bash
59
+ ```
60
+
61
+ 4. Build ONNX Runtime from source
62
+ ```shell
63
+ export CUDACXX=/usr/local/cuda-12.2/bin/nvcc
64
+ git config --global --add safe.directory '*'
65
+ sh build.sh --config Release --build_shared_lib --parallel --use_cuda --cuda_version 12.2 \
66
+ --cuda_home /usr/local/cuda-12.2 --cudnn_home /usr/lib/x86_64-linux-gnu/ --build_wheel --skip_tests \
67
+ --use_tensorrt --tensorrt_home /usr/src/tensorrt \
68
+ --cmake_extra_defines onnxruntime_BUILD_UNIT_TESTS=OFF \
69
+ --cmake_extra_defines CMAKE_CUDA_ARCHITECTURES=80 \
70
+ --allow_running_as_root
71
+ python3 -m pip install build/Linux/Release/dist/onnxruntime_gpu-*-cp310-cp310-linux_x86_64.whl --force-reinstall
72
+ ```
73
+
74
+ If the GPU is not A100, change CMAKE_CUDA_ARCHITECTURES=80 in the command line according to the GPU compute capacity (like 89 for RTX 4090, or 86 for RTX 3090). If your machine has less than 64GB memory, replace --parallel by --parallel 4 --nvcc_threads 1 to avoid out of memory.
75
+ 5. Install libraries and requirements
76
+ ```shell
77
+ python3 -m pip install --upgrade pip
78
+ cd /workspace/onnxruntime/python/tools/transformers/models/stable_diffusion
79
+ python3 -m pip install -r requirements-cuda12.txt
80
+ python3 -m pip install --upgrade polygraphy onnx-graphsurgeon --extra-index-url https://pypi.ngc.nvidia.com
81
+ ```
82
+ 6. Perform ONNX Runtime optimized inference
83
+ ```shell
84
+ python3 demo_txt2img_xl.py \
85
+ "starry night over Golden Gate Bridge by van gogh" \
86
+ --version xl-turbo \
87
+ --engine-dir /workspace/sdxl-turbo-onnxruntime
88
+ ```