RunDiffusion
commited on
Testing linking to 500 steps image
Browse files
README.md
CHANGED
@@ -116,24 +116,24 @@ A vintage comic book cover of Wonderman. On the cover, there are three main char
|
|
116 |
Wonderman, a male superhero character. He is wearing a green and red costume with a large 'W' emblem on the chest. Wonderman has a muscular physique, brown hair, and is wearing a black mask covering his eyes. He stands confidently with his hands by his sides. photo
|
117 |
![Standing Wonderman](https://huggingface.co/RunDiffusion/Wonderman-Flux-POC/resolve/main/Cleaned%20and%20Captioned%20Data/00002.png)
|
118 |
|
119 |
-
|
120 |
-
-
|
121 |
-
|
122 |
-
-
|
123 |
-
-
|
124 |
-
-
|
125 |
-
|
126 |
-
-
|
127 |
-
|
128 |
-
|
129 |
-
|
130 |
-
|
131 |
-
|
132 |
-
|
133 |
-
|
134 |
-
|
135 |
-
|
136 |
-
|
137 |
|
138 |
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
139 |
|
@@ -296,4 +296,11 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
|
|
296 |
|
297 |
## Model Card Contact
|
298 |
|
299 |
-
[More Information Needed]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
116 |
Wonderman, a male superhero character. He is wearing a green and red costume with a large 'W' emblem on the chest. Wonderman has a muscular physique, brown hair, and is wearing a black mask covering his eyes. He stands confidently with his hands by his sides. photo
|
117 |
![Standing Wonderman](https://huggingface.co/RunDiffusion/Wonderman-Flux-POC/resolve/main/Cleaned%20and%20Captioned%20Data/00002.png)
|
118 |
|
119 |
+
### Train the Data
|
120 |
+
All tasks were performed on a local workstation equipped with an RTX 4090, i7 processor, and 64GB RAM. Note that 32GB RAM will not suffice, as you may encounter out-of-memory (OOM) errors when caching latents. We did use RunDiffusion.com for testing the LoRAs created, enabling us to launch five servers with five checkpoints to determine the best one that converged
|
121 |
+
We're not going to dive into the rank and learning rate and stuff because this really depends on your goals and what you're trying to accomplish. But the rules below are good ones to follow.
|
122 |
+
- We used Ostris's ai-toolkit available here: https://github.com/ostris/ai-toolkit/tree/main
|
123 |
+
- Default config with LR: 4e-4 at Rank 16
|
124 |
+
- 2200 - 2600 steps saw good convergence. Even some checkpoints into the 4k step range turned out pretty good.
|
125 |
+
If targeting finer details, you may want to adjust the rank up to 32 and lower the learning rate. You will also need to run more steps if you do this.
|
126 |
+
**Training a style:** Using simple captions with clear examples to maintain a coherent style is crucial. Although caption-less LoRAs can sometimes work for styles, this was not within the scope of our goals, so we cannot provide specific insights.
|
127 |
+
**Training a concept:** You can choose either descriptive captions to avoid interfering with existing tokens or general captions that might interfere, depending on your intention. This choice should be intentional.
|
128 |
+
|
129 |
+
Captioning has never been more critical. Flux "gives you what you ask for" - and that's a good thing. You can train a LoRA on a single cartoon concept and still generate photo realistic people. You can even caption a cartoon in the foreground and a realistic scene in the background! This capability is BY DESIGN - so do not resist it - embrace it! (Spoiler alert next!)
|
130 |
+
![prompt different backgrounds]()
|
131 |
+
You'll see in the next page of examples where the captioning really helps or hurts you. Depending on your goals again you will need to choose the path that fits what you're trying to accomplish.
|
132 |
+
Total time for the LoRA was about 2 to 2.5 hours. $1 to $2 on RunPod, Vast, or local electricity will be even cheaper.
|
133 |
+
Now for the results! (This next file is big to preserve the quality)
|
134 |
+
|
135 |
+
## 500 Steps
|
136 |
+
![500 steps](Huggingface-assets/500-steps.jpg)
|
137 |
|
138 |
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
139 |
|
|
|
296 |
|
297 |
## Model Card Contact
|
298 |
|
299 |
+
[More Information Needed]
|
300 |
+
|
301 |
+
|
302 |
+
- **Developed by:** Darin Holbrook - RunDiffusion co-founder and Chief Technology Officer
|
303 |
+
- **Funded by:** RunDiffusion.com / RunPod.io
|
304 |
+
- **Model type:** Flux [dev] LoRA
|
305 |
+
- **License:** flux1dev https://huggingface.co/black-forest-labs/FLUX.1-dev
|
306 |
+
- **Finetuned from model:** https://huggingface.co/black-forest-labs/FLUX.1-dev
|