|
--- |
|
license: other |
|
license_name: flux-1-dev-non-commercial-license |
|
license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md |
|
language: |
|
- en |
|
tags: |
|
- flux |
|
- diffusers |
|
- lora |
|
- replicate |
|
base_model: "black-forest-labs/FLUX.1-dev" |
|
pipeline_tag: text-to-image |
|
widget: |
|
- text: 'screenprint tshirt design, a happy cat holding a sign that says "I LOVE VE REPLICATE", LNTP illustration style' |
|
output: |
|
url: "images/1.webp" |
|
|
|
- text: "a t-shirt, LNTP illustration style" |
|
output: |
|
url: "images/2.webp" |
|
|
|
- text: "a young girl playing piano, yellow background, LNTP illustration style" |
|
output: |
|
url: "images/3.webp" |
|
|
|
- text: "a book with the words 'Don't Panic!' written on cover, an homage to the hitchhikers guide to the galaxy, LNTP cartoon style" |
|
output: |
|
url: "images/4.webp" |
|
|
|
- text: "a robot, blue background, LNTP illustration style" |
|
output: |
|
url: "images/5.webp" |
|
|
|
- text: "girl, orange background, LNTP illustration style" |
|
output: |
|
url: "images/6.webp" |
|
|
|
instance_prompt: LNTP |
|
--- |
|
|
|
# Flux latentpop |
|
|
|
flux-latentpop features vibrant backgrounds with grungy limited screenprinting color goodness. |
|
|
|
It does great with t-shirt designs, general illustrations, and character portraits. |
|
|
|
<Gallery /> |
|
|
|
It was trained on Replicate, here: https://replicate.com/ostris/flux-dev-lora-trainer/train |
|
|
|
The training set is comprised of 23 images generated on MidJourney using the `--sref 3102110963` and `--personalize 3xdy3qw` flags. You can find the entire training set here in this repo: `./2024-08-24-latentpop.zip` |
|
|
|
Below are the training parameters I used, which seem to work fairly well for illustration/cartoony Flux LoRAs: |
|
|
|
``` |
|
{ |
|
"steps": 1300, |
|
"lora_rank": 24, |
|
"optimizer": "adamw8bit", |
|
"batch_size": 4, |
|
"resolution": "512,768,1024", |
|
"autocaption": true, |
|
"input_images": "https://replicate.delivery/pbxt/Lg3C1KUPfrRZZvJFaaSTmQ9qtAyXSonLvLSuTuj4Nop9vcSu/2024-08-24-latentpop.zip", |
|
"trigger_word": "LNTP", |
|
"learning_rate": 0.0002, |
|
"autocaption_suffix": "LNTP style", |
|
"caption_dropout_rate": 0.05, |
|
} |
|
``` |
|
|
|
Shoutout to @ciguleva on x who originally shared this sref on x: https://x.com/ciguleva/status/1827398343779098720 |
|
|
|
## Usage |
|
|
|
You should use `LNTP` to trigger the image generation. The output images look more stylistically interesting with a `guidance_scale` of ~`2.5`. |
|
|
|
## Use it with the [🧨 diffusers library](https://github.com/huggingface/diffusers) |
|
|
|
```py |
|
from diffusers import AutoPipelineForText2Image |
|
import torch |
|
|
|
pipeline = AutoPipelineForText2Image.from_pretrained('black-forest-labs/FLUX.1-dev', torch_dtype=torch.float16).to('cuda') |
|
pipeline.load_lora_weights('jakedahn/flux-latentpop', weight_name='lora.safetensors') |
|
image = pipeline('your prompt').images[0] |
|
``` |
|
|
|
For more details, including weighting, merging and fusing LoRAs, check the [documentation on loading LoRAs in diffusers](https://huggingface.co/docs/diffusers/main/en/using-diffusers/loading_adapters) |
|
|