Update README.md to include a diffusers example

#2
by sayakpaul HF staff - opened
Files changed (3) hide show
  1. .gitattributes +1 -0
  2. README.md +52 -0
  3. darth_vader_grid.png +3 -0
.gitattributes CHANGED
@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  out_ballerina.png filter=lfs diff=lfs merge=lfs -text
 
 
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  out_ballerina.png filter=lfs diff=lfs merge=lfs -text
37
+ darth_vader_grid.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -26,6 +26,58 @@ prompt: a ballerina, romantic sunset, 4k photo
26
 
27
  License: refers to the OpenPose's one.
28
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
29
 
30
  ### Training
31
 
 
26
 
27
  License: refers to the OpenPose's one.
28
 
29
+ ### Using in 🧨 diffusers
30
+
31
+ First, install all the libraries:
32
+
33
+ ```bash
34
+ pip install -q controlnet_aux transformers accelerate
35
+ pip install -q git+https://github.com/huggingface/diffusers
36
+ ```
37
+
38
+ Now, we're ready to make Darth Vader dance:
39
+
40
+ ```python
41
+ from diffusers import AutoencoderKL, StableDiffusionXLControlNetPipeline, ControlNetModel, UniPCMultistepScheduler
42
+ import torch
43
+ from controlnet_aux import OpenposeDetector
44
+ from diffusers.utils import load_image
45
+
46
+
47
+ # Compute openpose conditioning image.
48
+ openpose = OpenposeDetector.from_pretrained("lllyasviel/ControlNet")
49
+
50
+ image = load_image(
51
+ "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/person.png"
52
+ )
53
+ openpose_image = openpose(image)
54
+
55
+ # Initialize ControlNet pipeline.
56
+ controlnet = ControlNetModel.from_pretrained("thibaud/controlnet-openpose-sdxl-1.0", torch_dtype=torch.float16)
57
+ pipe = StableDiffusionXLControlNetPipeline.from_pretrained(
58
+ "stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, torch_dtype=torch.float16
59
+ )
60
+ pipe.enable_model_cpu_offload()
61
+
62
+
63
+ # Infer.
64
+ prompt = "Darth vader dancing in a desert, high quality"
65
+ negative_prompt = "low quality, bad quality"
66
+ images = pipe(
67
+ prompt,
68
+ negative_prompt=negative_prompt,
69
+ num_inference_steps=25,
70
+ num_images_per_prompt=4,
71
+ image=openpose_image.resize((1024, 1024)),
72
+ generator=torch.manual_seed(97),
73
+ ).images
74
+ images[0]
75
+ ```
76
+
77
+ Here are some gemerated examples:
78
+
79
+ ![](./darth_vader_grid.png)
80
+
81
 
82
  ### Training
83
 
darth_vader_grid.png ADDED

Git LFS Details

  • SHA256: eb3337cb5fa7b8ad9a09e95fb02e6af8966170737a855297f678e10265038994
  • Pointer size: 132 Bytes
  • Size of remote file: 6.62 MB