Commit
•
4104f86
1
Parent(s):
e6ef24f
Add diffusers example (#3)
Browse files- Add diffusers example (590d71ae3343479be99b4e18f4d4eb02fa07de04)
Co-authored-by: Suraj Patil <[email protected]>
README.md
CHANGED
@@ -14,7 +14,7 @@ This `stable-diffusion-2-inpainting` model is resumed from [stable-diffusion-2-b
|
|
14 |
![image](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting/resolve/main/merged-leopards.png)
|
15 |
|
16 |
- Use it with the [`stablediffusion`](https://github.com/Stability-AI/stablediffusion) repository: download the `512-inpainting-ema.ckpt` [here](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting/resolve/main/512-inpainting-ema.ckpt).
|
17 |
-
- Use it with 🧨 diffusers (
|
18 |
|
19 |
## Model Details
|
20 |
- **Developed by:** Robin Rombach, Patrick Esser
|
@@ -34,6 +34,42 @@ This `stable-diffusion-2-inpainting` model is resumed from [stable-diffusion-2-b
|
|
34 |
pages = {10684-10695}
|
35 |
}
|
36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
37 |
# Uses
|
38 |
|
39 |
## Direct Use
|
|
|
14 |
![image](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting/resolve/main/merged-leopards.png)
|
15 |
|
16 |
- Use it with the [`stablediffusion`](https://github.com/Stability-AI/stablediffusion) repository: download the `512-inpainting-ema.ckpt` [here](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting/resolve/main/512-inpainting-ema.ckpt).
|
17 |
+
- Use it with 🧨 diffusers (https://huggingface.co/stabilityai/stable-diffusion-2-inpainting#Examples)
|
18 |
|
19 |
## Model Details
|
20 |
- **Developed by:** Robin Rombach, Patrick Esser
|
|
|
34 |
pages = {10684-10695}
|
35 |
}
|
36 |
|
37 |
+
## Examples
|
38 |
+
|
39 |
+
Using the [🤗's Diffusers library](https://github.com/huggingface/diffusers) to run Stable Diffusion 2 inpainting in a simple and efficient manner.
|
40 |
+
|
41 |
+
```bash
|
42 |
+
pip install --upgrade git+https://github.com/huggingface/diffusers.git transformers accelerate scipy
|
43 |
+
```
|
44 |
+
|
45 |
+
```python
|
46 |
+
from diffusers import StableDiffusionInpaintPipeline
|
47 |
+
pipe = StableDiffusionInpaintPipeline.from_pretrained(
|
48 |
+
"stable-diffusion-2-inpainting",
|
49 |
+
revision="fp16",
|
50 |
+
torch_dtype=torch.float16,
|
51 |
+
)
|
52 |
+
prompt = "Face of a yellow cat, high resolution, sitting on a park bench"
|
53 |
+
#image and mask_image should be PIL images.
|
54 |
+
#The mask structure is white for inpainting and black for keeping as is
|
55 |
+
image = pipe(prompt=prompt, image=image, mask_image=mask_image).images[0]
|
56 |
+
image.save("./yellow_cat_on_park_bench.png")
|
57 |
+
```
|
58 |
+
|
59 |
+
**Notes**:
|
60 |
+
- Despite not being a dependency, we highly recommend you to install [xformers](https://github.com/facebookresearch/xformers) for memory efficient attention (better performance)
|
61 |
+
- If you have low GPU RAM available, make sure to add a `pipe.enable_attention_slicing()` after sending it to `cuda` for less VRAM usage (to the cost of speed)
|
62 |
+
|
63 |
+
**How it works:**
|
64 |
+
`image` | `mask_image`
|
65 |
+
:-------------------------:|:-------------------------:|
|
66 |
+
<img src="https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png" alt="drawing" width="300"/> | <img src="https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png" alt="drawing" width="300"/>
|
67 |
+
|
68 |
+
|
69 |
+
`prompt` | `Output`
|
70 |
+
:-------------------------:|:-------------------------:|
|
71 |
+
<span style="position: relative;bottom: 150px;">Face of a yellow cat, high resolution, sitting on a park bench</span> | <img src="https://huggingface.co/datasets/patrickvonplaten/images/resolve/main/test.png" alt="drawing" width="300"/>
|
72 |
+
|
73 |
# Uses
|
74 |
|
75 |
## Direct Use
|