Update README.md
Browse files
README.md
CHANGED
@@ -26,6 +26,58 @@ prompt: a ballerina, romantic sunset, 4k photo
|
|
26 |
|
27 |
License: refers to the OpenPose's one.
|
28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
29 |
|
30 |
### Training
|
31 |
|
|
|
26 |
|
27 |
License: refers to the OpenPose's one.
|
28 |
|
29 |
+
### Using in 🧨 diffusers
|
30 |
+
|
31 |
+
First, install all the libraries:
|
32 |
+
|
33 |
+
```bash
|
34 |
+
pip install -q controlnet_aux transformers accelerate
|
35 |
+
pip install -q git+https://github.com/huggingface/diffusers
|
36 |
+
```
|
37 |
+
|
38 |
+
Now, we're ready to make Darth Vader dance:
|
39 |
+
|
40 |
+
```python
|
41 |
+
from diffusers import AutoencoderKL, StableDiffusionXLControlNetPipeline, ControlNetModel, UniPCMultistepScheduler
|
42 |
+
import torch
|
43 |
+
from controlnet_aux import OpenposeDetector
|
44 |
+
from diffusers.utils import load_image
|
45 |
+
|
46 |
+
|
47 |
+
# Compute openpose conditioning image.
|
48 |
+
openpose = OpenposeDetector.from_pretrained("lllyasviel/ControlNet")
|
49 |
+
|
50 |
+
image = load_image(
|
51 |
+
"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/person.png"
|
52 |
+
)
|
53 |
+
openpose_image = openpose(image)
|
54 |
+
|
55 |
+
# Initialize ControlNet pipeline.
|
56 |
+
controlnet = ControlNetModel.from_pretrained("thibaud/controlnet-openpose-sdxl-1.0", torch_dtype=torch.float16)
|
57 |
+
pipe = StableDiffusionXLControlNetPipeline.from_pretrained(
|
58 |
+
"stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, torch_dtype=torch.float16
|
59 |
+
)
|
60 |
+
pipe.enable_model_cpu_offload()
|
61 |
+
|
62 |
+
|
63 |
+
# Infer.
|
64 |
+
prompt = "Darth vader dancing in a desert, high quality"
|
65 |
+
negative_prompt = "low quality, bad quality"
|
66 |
+
images = pipe(
|
67 |
+
prompt,
|
68 |
+
negative_prompt=negative_prompt,
|
69 |
+
num_inference_steps=25,
|
70 |
+
num_images_per_prompt=4,
|
71 |
+
image=openpose_image.resize((1024, 1024)),
|
72 |
+
generator=torch.manual_seed(97),
|
73 |
+
).images
|
74 |
+
images[0]
|
75 |
+
```
|
76 |
+
|
77 |
+
Here are some gemerated examples:
|
78 |
+
|
79 |
+
![]()
|
80 |
+
|
81 |
|
82 |
### Training
|
83 |
|