DionTimmer radames commited on
Commit
bb98e5e
1 Parent(s): 8dd37d8

add how to diffusers (#6)

Browse files

- add how to diffusers (e8f3a76443118248a52e7d2210734ff2f016ed78)


Co-authored-by: Radamés Ajna <[email protected]>

Files changed (1) hide show
  1. README.md +65 -0
README.md CHANGED
@@ -19,6 +19,71 @@ Separate repos for usage in diffusers can be found here:<br>
19
  1.5: https://huggingface.co/DionTimmer/controlnet_qrcode-control_v11p_sd21<br>
20
  2.1: https://huggingface.co/DionTimmer/controlnet_qrcode-control_v1p_sd15<br>
21
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
  ## Performance and Limitations
23
 
24
  These models perform quite well in most cases, but please note that they are not 100% accurate. In some instances, the QR code shape might not come through as expected. You can increase the ControlNet weight to emphasize the QR code shape. However, be cautious as this might negatively impact the style of your output.**To optimize for scanning, please generate your QR codes with correction mode 'H' (30%).**
 
19
  1.5: https://huggingface.co/DionTimmer/controlnet_qrcode-control_v11p_sd21<br>
20
  2.1: https://huggingface.co/DionTimmer/controlnet_qrcode-control_v1p_sd15<br>
21
 
22
+ ## How to use with Diffusers
23
+
24
+
25
+ ```bash
26
+ pip -q install diffusers transformers accelerate torch xformers
27
+ ```
28
+
29
+ ```python
30
+ import torch
31
+ from PIL import Image
32
+ from diffusers import StableDiffusionControlNetImg2ImgPipeline, ControlNetModel, DDIMScheduler
33
+ from diffusers.utils import load_image
34
+
35
+ controlnet = ControlNetModel.from_pretrained("DionTimmer/controlnet_qrcode-control_v1p_sd15",
36
+ torch_dtype=torch.float16)
37
+
38
+ pipe = StableDiffusionControlNetImg2ImgPipeline.from_pretrained(
39
+ "runwayml/stable-diffusion-v1-5",
40
+ controlnet=controlnet,
41
+ safety_checker=None,
42
+ torch_dtype=torch.float16
43
+ )
44
+
45
+ pipe.enable_xformers_memory_efficient_attention()
46
+ pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config)
47
+ pipe.enable_model_cpu_offload()
48
+
49
+ def resize_for_condition_image(input_image: Image, resolution: int):
50
+ input_image = input_image.convert("RGB")
51
+ W, H = input_image.size
52
+ k = float(resolution) / min(H, W)
53
+ H *= k
54
+ W *= k
55
+ H = int(round(H / 64.0)) * 64
56
+ W = int(round(W / 64.0)) * 64
57
+ img = input_image.resize((W, H), resample=Image.LANCZOS)
58
+ return img
59
+
60
+
61
+ # play with guidance_scale, controlnet_conditioning_scale and strength to make a valid QR Code Image
62
+
63
+ # qr code image
64
+ source_image = load_image("https://s3.amazonaws.com/moonup/production/uploads/6064e095abd8d3692e3e2ed6/A_RqHaAM6YHBodPLwqtjn.png")
65
+ # initial image, anything
66
+ init_image = load_image("https://s3.amazonaws.com/moonup/production/uploads/noauth/KfMBABpOwIuNolv1pe3qX.jpeg")
67
+ condition_image = resize_for_condition_image(source_image, 768)
68
+ init_image = resize_for_condition_image(init_image, 768)
69
+ generator = torch.manual_seed(123121231)
70
+ image = pipe(prompt="a bilboard in NYC with a qrcode",
71
+ negative_prompt="ugly, disfigured, low quality, blurry, nsfw",
72
+ image=init_image,
73
+ control_image=condition_image,
74
+ width=768,
75
+ height=768,
76
+ guidance_scale=20,
77
+ controlnet_conditioning_scale=1.5,
78
+ generator=generator,
79
+ strength=0.9,
80
+ num_inference_steps=150,
81
+ )
82
+
83
+ image.images[0]
84
+
85
+ ```
86
+
87
  ## Performance and Limitations
88
 
89
  These models perform quite well in most cases, but please note that they are not 100% accurate. In some instances, the QR code shape might not come through as expected. You can increase the ControlNet weight to emphasize the QR code shape. However, be cautious as this might negatively impact the style of your output.**To optimize for scanning, please generate your QR codes with correction mode 'H' (30%).**