Commit
·
f99d377
1
Parent(s):
e5022ef
Update readme
Browse files
README.md
CHANGED
@@ -18,13 +18,15 @@ The model works best with the "Balanced" ControlNet setting. Try using a "Contro
|
|
18 |
|
19 |
Smaller checker grids tend to perform worse (e.g. 5x5 vs a 32x32)
|
20 |
|
|
|
|
|
21 |
### Usage:
|
22 |
|
23 |
-
To install, copy the `.safetensors` and `.yaml` files to your Automatic1111 ControlNet extension's model directory (e.g. `stable-diffusion-webui
|
24 |
|
25 |
There is no preprocessor. Instead, supply a black and white checkerboard image as the control input. Examples are in the `example-control-images` directory of this repository. (https://huggingface.co/thomaseding/pixelnet/tree/main/example-control-images)
|
26 |
|
27 |
-
The script `gen_checker.py` can be used to generate checkerboard images of arbitrary sizes. (https://huggingface.co/thomaseding/pixelnet/blob/main/gen_checker.py) Example: `python gen_checker.py --upscale-dims 512x512 --output-file
|
28 |
|
29 |

|
30 |
|
@@ -40,9 +42,16 @@ Q: Should I use this model with a post-processor?
|
|
40 |
|
41 |
A: Yes, I still recommend you do post-processing to clean up the image. This model is not perfect and will still have artifacts. Note that none of the sample output images are post-processed; they are raw outputs from the model. Consider sampling the image based on the location of the control grid checker faces. I will provide a custom script specialized for this in the near future.
|
42 |
|
|
|
|
|
|
|
43 |
Q: Will there be a better trained model of this in the future?
|
44 |
|
45 |
-
A: I hope so. I will need to curate a much larger and higher-quality dataset, which might take me a long time. Regardless, I plan on making the control more faithful to the control image
|
|
|
|
|
|
|
|
|
46 |
|
47 |
### Sample Outputs:
|
48 |
|
|
|
18 |
|
19 |
Smaller checker grids tend to perform worse (e.g. 5x5 vs a 32x32)
|
20 |
|
21 |
+
Too low or too high of a "Steps" value breaks the model. Try something like 15-30, depending on an assortment of factors. Feel free to experiment with the built-in A1111 "X/Y/Z Plot" script.
|
22 |
+
|
23 |
### Usage:
|
24 |
|
25 |
+
To install, copy the `.safetensors` and `.yaml` files to your Automatic1111 ControlNet extension's model directory (e.g. `stable-diffusion-webui/extensions/sd-webui-controlnet/models`). Completely restart the Automatic1111 server after doing this and then refresh the web page.
|
26 |
|
27 |
There is no preprocessor. Instead, supply a black and white checkerboard image as the control input. Examples are in the `example-control-images` directory of this repository. (https://huggingface.co/thomaseding/pixelnet/tree/main/example-control-images)
|
28 |
|
29 |
+
The script `gen_checker.py` can be used to generate checkerboard images of arbitrary sizes. (https://huggingface.co/thomaseding/pixelnet/blob/main/gen_checker.py) Example: `python gen_checker.py --upscale-dims 512x512 --dims 70x70 --output-file control.png` to generate a 70x70 checkerboard image upscaled to 512x512 pixels.
|
30 |
|
31 |

|
32 |
|
|
|
42 |
|
43 |
A: Yes, I still recommend you do post-processing to clean up the image. This model is not perfect and will still have artifacts. Note that none of the sample output images are post-processed; they are raw outputs from the model. Consider sampling the image based on the location of the control grid checker faces. I will provide a custom script specialized for this in the near future.
|
44 |
|
45 |
+
Q: Does the model support non-square grids?
|
46 |
+
A: Kind of. I trained it with some non-perfect square grids (when pre-upscaled checkerboards are not a factor of the upscaled image size), so in that sense it should work fine. I also trained it with some checkerboard images with genuine non-square rectangular faces (e.g. double-wide pixels).
|
47 |
+
|
48 |
Q: Will there be a better trained model of this in the future?
|
49 |
|
50 |
+
A: I hope so. I will need to curate a much larger and higher-quality dataset, which might take me a long time. Regardless, I plan on making the control effect more faithful to the control image. I may decide to try to generalize this beyond rectangular grids, but that is not a priority. I think including non-square rectangular faces in some of the training data was perhaps harmful to the model's performance. Likewise for grids smaller than 8x8. Perhaps it is better to train separate models for very small grids (but at that point, you might as well make the images by hand) and for non-square rectangular grids.
|
51 |
+
|
52 |
+
Q: What about color quantization?
|
53 |
+
|
54 |
+
A: Coming soon, "PaletteNet".
|
55 |
|
56 |
### Sample Outputs:
|
57 |
|