anime-chars-simpletuner-lora-1

This is a LyCORIS adapter derived from black-forest-labs/FLUX.1-dev.

No validation prompt was used during training.

None

Validation settings

• CFG: 3.5
• CFG Rescale: 0.0
• Steps: 20
• Sampler: None
• Seed: 42
• Resolution: 1536x864

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt

unconditional (blank prompt)

Negative Prompt

blurry, cropped, ugly

Prompt

A scene from One Piece. Nami stands at the edge of a cliff, wind blowing through her hair, with the ocean far below. She's wearing a purple tank top and white skirt, her eyes full of determination. Sanji Vinsmoke is behind her, looking away, leaning on a tree with a cigarette in hand. He's dressed in his usual black suit with a blue dress shirt and black tie. The setting sun casts a golden glow over the rocky terrain and crashing waves below.

Negative Prompt

blurry, cropped, ugly

Prompt

A scene from Jujutsu Kaisen. Maki Zenin is standing on a beach during sunset holding a sign that says 'I love prompts'.

Negative Prompt

blurry, cropped, ugly

Prompt

A scene from Demon Slayer. Muzan Kibutsuji is standing in front of a large chalkboard filled with cryptic equations and diagrams. He's wearing a white fedora, a white suit with a purple shirt underneath, and white gloves. He holds a piece of chalk in one hand and a test tube in the other, his eyes gleaming with focus. The warm light from nearby candles flickers against the shelves of mysterious bottles and ingredients in his lab, giving the scene an eerie ambiance.

Negative Prompt

blurry, cropped, ugly

Prompt

A scene from Attack on Titan. Levi Ackerman is sitting under a tree, holding the piece of paper close to his face as he reads intently

Negative Prompt

blurry, cropped, ugly

Prompt

A scene from My Hero Academia. Shoto Todoroki stands on a cliff overlooking a vast desert, his hand outstretched and half of the cliffside covered in ice. He's wearing his hero costume: a dark blue jacket with elbow-length sleeves, matching pants, a silver-colored combat vest, and white boots. His outfit contrasts against the bright, sandy terrain. The harsh sunlight creates long shadows across the rocky landscape, adding to the intensity of the moment.

Negative Prompt

blurry, cropped, ugly

Prompt

A scene from Demon Slayer. Tanjiro Kamado stands beside him, holding his sword at the ready. He's wearing his Demon Slayer uniform: a dark jacket with gold trim over a white shirt, along with his signature green and black checkered haori fluttering in the wind. Shoto Todoroki stands in a dense forest, with a serious expression. He's wearing his hero costume: a dark blue jacket with elbow-length sleeves, matching pants, a silver-colored combat vest, and white boots. His ice powers cover part of the forest floor.

Negative Prompt

blurry, cropped, ugly

Prompt

A scene from Attack on Titan. Mikasa Ackerman watches in disbelief. She's wearing her Survey Corps uniform: a short, light brown jacket with the Wings of Freedom insignia, white pants, and dark brown boots. Her military uniform flutters in the wind as she stands ready with her ODM gear. Monkey D. Luffy is climbing the massive walls of Paradis Island, laughing as he pulls himself higher. He's wearing his classic outfit: a red vest, blue shorts, and sandals, with his straw hat secured around his neck. The sky is clear, with just a few clouds drifting above, and the vast landscape stretches out below them.

Negative Prompt

blurry, cropped, ugly

Prompt

A scene from My Hero Academia. Makima stands in the middle of a quiet street, holding a test tube filled with a strange glowing liquid. She's wearing her usual attire: a white button-up shirt, black tie, and dark pants, with her distinctive red hair tied back in a braid. Shoto Todoroki stands next to her, his hand raised as he is waving. He's in his hero costume: a dark blue jacket with elbow-length sleeves, matching pants, a silver-colored combat vest, and white boots. The street is eerily still, with the remains of a destroyed building in the background, and the sun is setting, casting long shadows across the scene.

Negative Prompt

blurry, cropped, ugly

Prompt

A scene from One Piece. Levi Ackerman is on the deck of the Thousand Sunny, methodically cleaning his gear with a stern expression. He's wearing his Survey Corps uniform: a short, light brown jacket with the Wings of Freedom insignia, white pants, and dark brown boots. Nami stands nearby, glancing over at him with a mix of curiosity and amusement, her arms crossed. She's wearing a blue and white striped bikini top, low-rise jeans, and high-heeled sandals. The ship sails through calm waters under a bright blue sky, with a distant island just visible on the horizon.

Negative Prompt

blurry, cropped, ugly

Prompt

A scene from My Hero Academia. Shoto Todoroki stands in the middle of a training field, facing off against an opponent. Shoto Todoroki is in his hero costume: a dark blue jacket with elbow-length sleeves, matching pants, a silver-colored combat vest, and white boots. His Opponent, Ichigo Kurosaki stands opposite him, he is wearing his Shinigami robes: a black kimono and hakama, with a white sash around his waist. He holds his sword with a confident expression. The sky above is cloudy, with faint rays of sunlight piercing through.

Negative Prompt

blurry, cropped, ugly

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

• Training epochs: 382
• Training steps: 25200
• Learning rate: 0.001
• Effective batch size: 160
  • Micro-batch size: 40
  • Gradient accumulation steps: 1
  • Number of GPUs: 4
• Prediction type: flow-matching
• Rescaled betas zero SNR: False
• Optimizer: adamw_bf16
• Precision: Pure BF16
• Quantised: Yes: int8-quanto
• Xformers: Not used
• LyCORIS Config:

{
    "algo": "lokr",
    "multiplier": 1.0,
    "linear_dim": 10000,
    "linear_alpha": 1,
    "factor": 12,
    "apply_preset": {
        "target_module": [
            "Attention",
            "FeedForward"
        ],
        "module_algo_map": {
            "Attention": {
                "factor": 12
            },
            "FeedForward": {
                "factor": 6
            }
        }
    }
}

Datasets

anime_char-512

• Repeats: 2
• Total number of images: ~3476
• Total number of aspect buckets: 1
• Resolution: 0.262144 megapixels
• Cropped: False
• Crop style: None
• Crop aspect: None

Inference

import torch
from diffusers import DiffusionPipeline
from lycoris import create_lycoris_from_weights

model_id = 'black-forest-labs/FLUX.1-dev'
adapter_id = 'pytorch_lora_weights.safetensors' # you will have to download this manually
lora_scale = 1.0
wrapper, _ = create_lycoris_from_weights(lora_scale, adapter_id, pipeline.transformer)
wrapper.merge_to()

prompt = "An astronaut is riding a horse through the jungles of Thailand."

pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu')
image = pipeline(
    prompt=prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(1641421826),
    width=1536,
    height=864,
    guidance_scale=3.5,
).images[0]
image.save("output.png", format="PNG")