multimodalart HF staff valhalla commited on
Commit
4104f86
1 Parent(s): e6ef24f

Add diffusers example (#3)

Browse files

- Add diffusers example (590d71ae3343479be99b4e18f4d4eb02fa07de04)


Co-authored-by: Suraj Patil <[email protected]>

Files changed (1) hide show
  1. README.md +37 -1
README.md CHANGED
@@ -14,7 +14,7 @@ This `stable-diffusion-2-inpainting` model is resumed from [stable-diffusion-2-b
14
  ![image](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting/resolve/main/merged-leopards.png)
15
 
16
  - Use it with the [`stablediffusion`](https://github.com/Stability-AI/stablediffusion) repository: download the `512-inpainting-ema.ckpt` [here](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting/resolve/main/512-inpainting-ema.ckpt).
17
- - Use it with 🧨 diffusers (_coming soon_)
18
 
19
  ## Model Details
20
  - **Developed by:** Robin Rombach, Patrick Esser
@@ -34,6 +34,42 @@ This `stable-diffusion-2-inpainting` model is resumed from [stable-diffusion-2-b
34
  pages = {10684-10695}
35
  }
36
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
  # Uses
38
 
39
  ## Direct Use
 
14
  ![image](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting/resolve/main/merged-leopards.png)
15
 
16
  - Use it with the [`stablediffusion`](https://github.com/Stability-AI/stablediffusion) repository: download the `512-inpainting-ema.ckpt` [here](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting/resolve/main/512-inpainting-ema.ckpt).
17
+ - Use it with 🧨 diffusers (https://huggingface.co/stabilityai/stable-diffusion-2-inpainting#Examples)
18
 
19
  ## Model Details
20
  - **Developed by:** Robin Rombach, Patrick Esser
 
34
  pages = {10684-10695}
35
  }
36
 
37
+ ## Examples
38
+
39
+ Using the [🤗's Diffusers library](https://github.com/huggingface/diffusers) to run Stable Diffusion 2 inpainting in a simple and efficient manner.
40
+
41
+ ```bash
42
+ pip install --upgrade git+https://github.com/huggingface/diffusers.git transformers accelerate scipy
43
+ ```
44
+
45
+ ```python
46
+ from diffusers import StableDiffusionInpaintPipeline
47
+ pipe = StableDiffusionInpaintPipeline.from_pretrained(
48
+ "stable-diffusion-2-inpainting",
49
+ revision="fp16",
50
+ torch_dtype=torch.float16,
51
+ )
52
+ prompt = "Face of a yellow cat, high resolution, sitting on a park bench"
53
+ #image and mask_image should be PIL images.
54
+ #The mask structure is white for inpainting and black for keeping as is
55
+ image = pipe(prompt=prompt, image=image, mask_image=mask_image).images[0]
56
+ image.save("./yellow_cat_on_park_bench.png")
57
+ ```
58
+
59
+ **Notes**:
60
+ - Despite not being a dependency, we highly recommend you to install [xformers](https://github.com/facebookresearch/xformers) for memory efficient attention (better performance)
61
+ - If you have low GPU RAM available, make sure to add a `pipe.enable_attention_slicing()` after sending it to `cuda` for less VRAM usage (to the cost of speed)
62
+
63
+ **How it works:**
64
+ `image` | `mask_image`
65
+ :-------------------------:|:-------------------------:|
66
+ <img src="https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png" alt="drawing" width="300"/> | <img src="https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png" alt="drawing" width="300"/>
67
+
68
+
69
+ `prompt` | `Output`
70
+ :-------------------------:|:-------------------------:|
71
+ <span style="position: relative;bottom: 150px;">Face of a yellow cat, high resolution, sitting on a park bench</span> | <img src="https://huggingface.co/datasets/patrickvonplaten/images/resolve/main/test.png" alt="drawing" width="300"/>
72
+
73
  # Uses
74
 
75
  ## Direct Use