--- license: other base_model: "black-forest-labs/FLUX.1-schnell" tags: - flux - flux-diffusers - text-to-image - diffusers - simpletuner - not-for-all-audiences - lora - template:sd-lora - lycoris inference: true widget: - text: 'unconditional (blank prompt)' parameters: negative_prompt: '''' output: url: ./assets/image_0_0.png - text: 'A woman dons a yellow outfit with checkered flag pattern accents, basking in the sunlight with people nearby. Emblazoned on the left side of her shirt is the word "PIRELLI", while the right side reads "FACTORY".' parameters: negative_prompt: '''' output: url: ./assets/image_1_0.png - text: 'The image depicts a dark green, textured trash can adorned with a bright sticker on a cobblestone street. The sticker reads "DRUNK DRIVER TARGET," suggesting the trash can is intended for use by inebriated individuals to prevent littering or accidents.' parameters: negative_prompt: '''' output: url: ./assets/image_2_0.png - text: 'A moment during a track relay race where one runner is passing the baton to his teammate; spectators can be seen in the stands, cheering.' parameters: negative_prompt: '''' output: url: ./assets/image_3_0.png - text: 'An anime character is racing down the street while wearing cat ears, a red dress, black shoes, and wearing aviator style sunglasses.' parameters: negative_prompt: '''' output: url: ./assets/image_4_0.png - text: 'A figurine of a character with green hair, wearing a white shirt, a black vest, and a gray cap, sitting with one hand on their knee and the other hand making a peace sign. The character is wearing a blue pendant and has a gold bracelet. In the background, there are green plants and a tree branch.' parameters: negative_prompt: '''' output: url: ./assets/image_5_0.png --- # flux-training This is a LyCORIS adapter derived from [black-forest-labs/FLUX.1-schnell](https://huggingface.co/black-forest-labs/FLUX.1-schnell). The main validation prompt used during training was: ``` A figurine of a character with green hair, wearing a white shirt, a black vest, and a gray cap, sitting with one hand on their knee and the other hand making a peace sign. The character is wearing a blue pendant and has a gold bracelet. In the background, there are green plants and a tree branch. ``` ## Validation settings - CFG: `3.0` - CFG Rescale: `0.0` - Steps: `20` - Sampler: `None` - Seed: `42` - Resolution: `1024x1024` Note: The validation settings are not necessarily the same as the [training settings](#training-settings). You can find some example images in the following gallery: The text encoder **was not** trained. You may reuse the base model text encoder for inference. ## Training settings - Training epochs: 0 - Training steps: 158000 - Learning rate: 5e-06 - Effective batch size: 5 - Micro-batch size: 1 - Gradient accumulation steps: 1 - Number of GPUs: 5 - Prediction type: flow-matching - Rescaled betas zero SNR: False - Optimizer: adamw_bf16 - Precision: Pure BF16 - Quantised: Yes: int8-quanto - Xformers: Not used - LyCORIS Config: ```json { "algo": "lokr", "multiplier": 1.0, "linear_dim": 1000000, "linear_alpha": 1, "factor": 2, "full_matrix": true, "apply_preset": { "name_algo_map": { "transformer_blocks.[0-7]*": { "algo": "lokr", "factor": 4, "linear_dim": 1000000, "linear_alpha": 1, "full_matrix": true }, "transformer_blocks.[8-15]*": { "algo": "lokr", "factor": 5, "linear_dim": 1000000, "linear_alpha": 1, "full_matrix": true }, "transformer_blocks.[16-18]*": { "algo": "lokr", "factor": 10, "linear_dim": 1000000, "linear_alpha": 1, "full_matrix": true }, "single_transformer_blocks.[0-15]*": { "algo": "lokr", "factor": 8, "linear_dim": 1000000, "linear_alpha": 1, "full_matrix": true }, "single_transformer_blocks.[16-23]*": { "algo": "lokr", "factor": 5, "linear_dim": 1000000, "linear_alpha": 1, "full_matrix": true }, "single_transformer_blocks.[24-37]*": { "algo": "lokr", "factor": 4, "linear_dim": 1000000, "linear_alpha": 1, "use_scalar": true, "full_matrix": true } }, "use_fnmatch": true } } ``` ## Datasets ### default_dataset_arb - Repeats: 0 - Total number of images: ~39950 - Total number of aspect buckets: 6 - Resolution: 1.048576 megapixels - Cropped: True - Crop style: random - Crop aspect: closest ### default_dataset_arb2 - Repeats: 0 - Total number of images: ~40005 - Total number of aspect buckets: 6 - Resolution: 1.048576 megapixels - Cropped: True - Crop style: random - Crop aspect: closest ### default_dataset_arb3 - Repeats: 0 - Total number of images: ~64080 - Total number of aspect buckets: 6 - Resolution: 1.048576 megapixels - Cropped: True - Crop style: random - Crop aspect: closest ### default_dataset_arb4 - Repeats: 0 - Total number of images: ~70340 - Total number of aspect buckets: 6 - Resolution: 1.048576 megapixels - Cropped: True - Crop style: random - Crop aspect: closest ### default_dataset_arb5 - Repeats: 0 - Total number of images: ~31790 - Total number of aspect buckets: 6 - Resolution: 1.048576 megapixels - Cropped: True - Crop style: random - Crop aspect: closest ### default_dataset_arb6 - Repeats: 0 - Total number of images: ~22035 - Total number of aspect buckets: 5 - Resolution: 1.048576 megapixels - Cropped: True - Crop style: random - Crop aspect: closest ### default_dataset_arb7 - Repeats: 0 - Total number of images: ~39390 - Total number of aspect buckets: 6 - Resolution: 1.048576 megapixels - Cropped: True - Crop style: random - Crop aspect: closest ### default_dataset - Repeats: 0 - Total number of images: ~121070 - Total number of aspect buckets: 1 - Resolution: 1.048576 megapixels - Cropped: True - Crop style: random - Crop aspect: square ### default_dataset2 - Repeats: 0 - Total number of images: ~123100 - Total number of aspect buckets: 1 - Resolution: 1.048576 megapixels - Cropped: True - Crop style: random - Crop aspect: square ### default_dataset3 - Repeats: 0 - Total number of images: ~64355 - Total number of aspect buckets: 1 - Resolution: 1.048576 megapixels - Cropped: True - Crop style: random - Crop aspect: square ### default_dataset4 - Repeats: 0 - Total number of images: ~69970 - Total number of aspect buckets: 1 - Resolution: 1.048576 megapixels - Cropped: True - Crop style: random - Crop aspect: square ### default_dataset5 - Repeats: 0 - Total number of images: ~32300 - Total number of aspect buckets: 1 - Resolution: 1.048576 megapixels - Cropped: True - Crop style: random - Crop aspect: square ### default_dataset6 - Repeats: 0 - Total number of images: ~22190 - Total number of aspect buckets: 1 - Resolution: 1.048576 megapixels - Cropped: True - Crop style: random - Crop aspect: square ### default_dataset7 - Repeats: 0 - Total number of images: ~39560 - Total number of aspect buckets: 1 - Resolution: 1.048576 megapixels - Cropped: True - Crop style: random - Crop aspect: square ## Inference ```python import torch from diffusers import DiffusionPipeline from lycoris import create_lycoris_from_weights model_id = 'black-forest-labs/FLUX.1-schnell' adapter_id = 'pytorch_lora_weights.safetensors' # you will have to download this manually lora_scale = 1.0 wrapper, _ = create_lycoris_from_weights(lora_scale, adapter_id, pipeline.transformer) wrapper.merge_to() prompt = "A figurine of a character with green hair, wearing a white shirt, a black vest, and a gray cap, sitting with one hand on their knee and the other hand making a peace sign. The character is wearing a blue pendant and has a gold bracelet. In the background, there are green plants and a tree branch." pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') image = pipeline( prompt=prompt, num_inference_steps=20, generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(1641421826), width=1024, height=1024, guidance_scale=3.0, ).images[0] image.save("output.png", format="PNG") ```