Commit
·
46b6a5b
1
Parent(s):
1099b93
Update README.md
Browse files
README.md
CHANGED
@@ -1,12 +1,46 @@
|
|
1 |
---
|
2 |
license: openrail
|
3 |
datasets:
|
4 |
-
-
|
5 |
language:
|
6 |
- en
|
7 |
---
|
8 |
|
9 |
-
Based on https://github.com/lllyasviel/ControlNet/discussions/318
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
10 |
|
11 |
```
|
12 |
accelerate launch train_controlnet.py ^
|
@@ -25,6 +59,25 @@ accelerate launch train_controlnet.py ^
|
|
25 |
--seed=0
|
26 |
```
|
27 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
28 |
Images converted with https://github.com/shaojunluo/EDLinePython
|
29 |
|
30 |
Default settings are:
|
@@ -40,14 +93,16 @@ Default settings are:
|
|
40 |
}
|
41 |
```
|
42 |
|
43 |
-
|
44 |
-
|
45 |
-
additional argument: `--proportion_empty_prompts=0.5`
|
46 |
Trained for 40000 steps with default settings
|
47 |
-
=>
|
48 |
|
49 |
-
**control-edgedrawing-default-
|
50 |
|
51 |
-
|
|
|
|
|
|
|
|
|
52 |
Trained for 40000 steps with default settings
|
53 |
-
=>
|
|
|
1 |
---
|
2 |
license: openrail
|
3 |
datasets:
|
4 |
+
- ChristophSchuhmann/improved_aesthetics_6.5plus
|
5 |
language:
|
6 |
- en
|
7 |
---
|
8 |
|
9 |
+
Based on my GitHub monologs at [Edge Drawing - a Canny alternative](https://github.com/lllyasviel/ControlNet/discussions/318)
|
10 |
+
|
11 |
+
Controls image generation by edge maps generated with [EdgeDrawing Parameter-Free](https://github.com/CihanTopal/ED_Lib).
|
12 |
+
|
13 |
+
For usage see the model page on [Civitai.com]().
|
14 |
+
|
15 |
+
# Image dataset
|
16 |
+
|
17 |
+
* [laion2B-en aesthetics>=6.5 dataset](https://huggingface.co/datasets/ChristophSchuhmann/improved_aesthetics_6.5plus)
|
18 |
+
* `--min_image_size 512 --max_aspect_ratio 2 --resize_mode="center_crop" --image_size 512`
|
19 |
+
* resulting in 180k images
|
20 |
+
|
21 |
+
Original
|
22 |
+
|
23 |
+
|
24 |
+

|
25 |
+
|
26 |
+
EdgeDrawing Parameter-Free
|
27 |
+
|
28 |
+
|
29 |
+

|
30 |
+
|
31 |
+
Canndy Edge Detection (default in Automatic1111)
|
32 |
+
|
33 |
+
|
34 |
+

|
35 |
+
|
36 |
+
Example
|
37 |
+
sampler=UniPC steps=20 cfg=7.5 seed=0 batch=9 model: v1-5-pruned-emaonly.safetensors cherry-picked: 1/9
|
38 |
+
prompt: _a detailed high-quality professional photo of swedish woman standing in front of a mirror, dark brown hair, white hat with purple feather_
|
39 |
+
|
40 |
+
|
41 |
+

|
42 |
+
|
43 |
+
# Training
|
44 |
|
45 |
```
|
46 |
accelerate launch train_controlnet.py ^
|
|
|
59 |
--seed=0
|
60 |
```
|
61 |
|
62 |
+
# Versions
|
63 |
+
|
64 |
+
**v3 - control-edgedrawing-cv480edpf-drop0-fp16-checkpoint-45000**
|
65 |
+
|
66 |
+
Conditioning images generated with [edpf.py](https://gitlab.com/-/snippets/3601881) using [opencv-contrib-python::ximgproc::EdgeDrawing](https://docs.opencv.org/4.8.0/d1/d1c/classcv_1_1ximgproc_1_1EdgeDrawing.html).
|
67 |
+
|
68 |
+
```
|
69 |
+
ed = cv2.ximgproc.createEdgeDrawing()
|
70 |
+
params = cv2.ximgproc.EdgeDrawing.Params()
|
71 |
+
params.PFmode = True
|
72 |
+
ed.setParams(params)
|
73 |
+
edges = ed.detectEdges(image)
|
74 |
+
edge_map = ed.getEdgeImage(edges)
|
75 |
+
```
|
76 |
+
|
77 |
+
45000 steps
|
78 |
+
|
79 |
+
**v2 - control-edgedrawing-default-noisy-drop0-fp16-checkpoint-40000**
|
80 |
+
|
81 |
Images converted with https://github.com/shaojunluo/EDLinePython
|
82 |
|
83 |
Default settings are:
|
|
|
93 |
}
|
94 |
```
|
95 |
|
96 |
+
`smoothed=True`, but no empty prompts
|
|
|
|
|
97 |
Trained for 40000 steps with default settings
|
98 |
+
=> conditioning images are too noisy
|
99 |
|
100 |
+
**v1 - control-edgedrawing-default-drop50-fp16-checkpoint-40000**
|
101 |
|
102 |
+
Same as v1
|
103 |
+
|
104 |
+
Update: bug in algorithm produces too sparse images on default, see https://github.com/shaojunluo/EDLinePython/issues/4
|
105 |
+
|
106 |
+
additional arguments: `--proportion_empty_prompts=0.5`
|
107 |
Trained for 40000 steps with default settings
|
108 |
+
=> empty prompts were probably too excessive
|