Update README.md
Browse files
README.md
CHANGED
@@ -13,11 +13,11 @@ The SD models are all "Original" (not "Split-Einsum") and built for CPU and GPU.
|
|
13 |
|
14 |
The Stable Diffusion v1.5 model and the other SD 1.5 type models contain both the standard Unet and the ControlledUnet used for a ControlNet pipeline. The correct one will be used automatically based on whether a ControlNet is enabled or not.
|
15 |
|
16 |
-
They have VAEEncoder.mlmodelc bundles that allow Image2Image to operate correctly at the noted resolutions, when used with a current Swift CLI pipeline or a current GUI built with ml-stable-diffusion 0.4.0, such as Mochi Diffusion 3.2, 4.0, or later.
|
17 |
|
18 |
All of the ControlNet models are "Original" ones, built for CPU and GPU compute units (cpuAndGPU) and for SD-1.5 type models. They will not work with SD-2.1 type models. The zip files each have a set of models at 4 resolutions. The 512x512 builds appear to also work with "Split-Einsum" models, using CPU and GPU (cpuAmdGPU), but from my tests, they will not work with "Split-Einsum" models when using the Neural Engine (NE).
|
19 |
|
20 |
-
All of the models in this repo work with Swift and the
|
21 |
|
22 |
The full SD models are in the "SD" folder of this repo. They are in subfolders by model name and individually zipped for a particular resolution. They need to be unzipped for use after downloading.
|
23 |
|
@@ -25,11 +25,11 @@ The ControlNet model files are in the "CN" folder of this repo. They are zipped
|
|
25 |
|
26 |
There is also a "MISC" folder that has text files with some notes and a screencap of my directory structure. These are provided for those who want to convert models themselves and/or run the models with a SwiftCLI. The notes are not perfect, and may be out of date if any of the Python or CoreML packages referenced have been updated recently. You can open a Discussion here if you need help with any of the "MISC" items.
|
27 |
|
28 |
-
For command line use, the "MISC" notes cover setting up a miniconda3 environment. If you are using the command line, please read the notes concerning naming and placement of your ControlNet model folder.
|
29 |
|
30 |
If you are using a GUI like Mochi Diffusion 4.0, the app will most likely guide you to the correct location/arrangement for your ConrolNet model folder.
|
31 |
|
32 |
-
Please note that when you unzip the ControlNet files (for example Canny.zip) from this repo, they will unzip into a folder, with the actual model files inside that folder. This folder is just for the zipping process. **What you want to move into your ControlNet model folder in Mochi Diffusion will be the individual files, not the folder they unzip into.** This is different from base models where you do want to copy the folder itself. See the images [**here**](https://huggingface.co/jrrjrr/CoreML-Models-For-ControlNet/blob/main/CN/-Settings.jpg) and [**here**](https://huggingface.co/jrrjrr/CoreML-Models-For-ControlNet/blob/main/CN/-Folders.jpg) for an example of how my folders are set up.
|
33 |
|
34 |
The sizes noted for all model type inputs/outputs are WIDTH x HEIGHT. A 512x768 is "portrait" orientation and a 768x512 is "landscape" orientation.
|
35 |
|
|
|
13 |
|
14 |
The Stable Diffusion v1.5 model and the other SD 1.5 type models contain both the standard Unet and the ControlledUnet used for a ControlNet pipeline. The correct one will be used automatically based on whether a ControlNet is enabled or not.
|
15 |
|
16 |
+
They have VAEEncoder.mlmodelc bundles that allow Image2Image to operate correctly at the noted resolutions, when used with a current Swift CLI pipeline or a current GUI built with ml-stable-diffusion 0.4.0 or ml-stable-diffusion 1.0.0, such as Mochi Diffusion 3.2, 4.0, or later.
|
17 |
|
18 |
All of the ControlNet models are "Original" ones, built for CPU and GPU compute units (cpuAndGPU) and for SD-1.5 type models. They will not work with SD-2.1 type models. The zip files each have a set of models at 4 resolutions. The 512x512 builds appear to also work with "Split-Einsum" models, using CPU and GPU (cpuAmdGPU), but from my tests, they will not work with "Split-Einsum" models when using the Neural Engine (NE).
|
19 |
|
20 |
+
All of the models in this repo work with Swift and the apple/ml-stable-diffusion pipeline (release 0.4.0 or 1.0.0). They were not built for, and will not work with, a Python Diffusers pipeline. They need [**ml-stable-diffusion**](https://github.com/apple/ml-stable-diffusion) for command line use, or a Swift app that supports ControlNet, such as the new (June 2023) [**Mochi Diffusion**](https://github.com/godly-devotion/MochiDiffusion) 4.0 version.
|
21 |
|
22 |
The full SD models are in the "SD" folder of this repo. They are in subfolders by model name and individually zipped for a particular resolution. They need to be unzipped for use after downloading.
|
23 |
|
|
|
25 |
|
26 |
There is also a "MISC" folder that has text files with some notes and a screencap of my directory structure. These are provided for those who want to convert models themselves and/or run the models with a SwiftCLI. The notes are not perfect, and may be out of date if any of the Python or CoreML packages referenced have been updated recently. You can open a Discussion here if you need help with any of the "MISC" items.
|
27 |
|
28 |
+
For command line use, the "MISC" notes cover setting up a miniconda3 environment. If you are using the command line, please read the notes concerning naming and placement of your ControlNet model folder. Briefly, they will need to go inside a "controlnet" folder that you placed inside your base model folder. You'll need a "controlnet" folder inside each base model folder, or a symlink named "controlnet" pointing to a central folder with all your ControlNet models inside it.
|
29 |
|
30 |
If you are using a GUI like Mochi Diffusion 4.0, the app will most likely guide you to the correct location/arrangement for your ConrolNet model folder.
|
31 |
|
32 |
+
Please note that when you unzip the ControlNet files (for example Canny.zip) from this repo, they will unzip into a folder, with the actual model files inside that folder. This folder is just for the zipping process. **What you want to move into your ControlNet model folder in Mochi Diffusion will be the individual files, not the folder they unzip into.** This is different from base models where you do want to copy the folder itself. See the images [**here**](https://huggingface.co/jrrjrr/CoreML-Models-For-ControlNet/blob/main/CN/-Settings.jpg) and [**here**](https://huggingface.co/jrrjrr/CoreML-Models-For-ControlNet/blob/main/CN/-Folders.jpg) for an example of how my folders are set up for Mochi Diffusion.
|
33 |
|
34 |
The sizes noted for all model type inputs/outputs are WIDTH x HEIGHT. A 512x768 is "portrait" orientation and a 768x512 is "landscape" orientation.
|
35 |
|