minpeter/LoRA-FLUX-anime-style-v1

See ostris/ai-toolkit config

ostris/ai-toolkit commit hash: fa187b1

job: "extension"
config:
  name: "FAS"
  process:
    - type: "ui_trainer"
      training_folder: "/data/minpeter/github.com/flux/ai-toolkit/output"
      sqlite_db_path: "./aitk_db.db"
      device: "cuda"
      trigger_word: "anime style"
      performance_log_every: 10
      network:
        type: "lora"
        linear: 32
        linear_alpha: 32
        lokr_full_rank: true
        lokr_factor: -1
      save:
        dtype: "bf16"
        save_every: 250
        max_step_saves_to_keep: 4
        save_format: "diffusers"
        push_to_hub: false
      datasets:
        - folder_path: "/data/minpeter/github.com/flux/ai-toolkit/datasets/fas"
          mask_path: null
          mask_min_value: 0.1
          default_caption: "anime style, "
          caption_ext: "txt"
          caption_dropout_rate: 0.05
          cache_latents_to_disk: false
          is_reg: false
          network_weight: 1
          resolution:
            - 512
            - 768
            - 1024
            - 1280
            - 1536
            - 256
        - folder_path: "/data/minpeter/github.com/flux/ai-toolkit/datasets/f"
          mask_path: null
          mask_min_value: 0.1
          default_caption: ""
          caption_ext: "txt"
          caption_dropout_rate: 0.05
          cache_latents_to_disk: false
          is_reg: false
          network_weight: 1
          resolution:
            - 512
            - 768
            - 1024
            - 1280
            - 1536
            - 256
      train:
        batch_size: 1
        bypass_guidance_embedding: false
        steps: 2500
        gradient_accumulation: 1
        train_unet: true
        train_text_encoder: false
        gradient_checkpointing: true
        noise_scheduler: "flowmatch"
        optimizer: "adamw8bit"
        timestep_type: "sigmoid"
        content_or_style: "balanced"
        optimizer_params:
          weight_decay: 0.0001
        unload_text_encoder: false
        lr: 0.0001
        ema_config:
          use_ema: true
          ema_decay: 0.99
        dtype: "bf16"
        diff_output_preservation: false
        diff_output_preservation_multiplier: 1
        diff_output_preservation_class: "person"
      model:
        name_or_path: "black-forest-labs/FLUX.1-dev"
        quantize: true
        quantize_te: true
        arch: "flux"
        low_vram: false
      sample:
        sampler: "flowmatch"
        sample_every: 250
        width: 1024
        height: 1024
        prompts:
          - "anime style, woman with red hair, playing chess at the park, bomb going off in the background"
          - "anime style, a man showing off his cool new t shirt at the beach, a shark is jumping out of the water in the background"
          - "anime style, a bear building a log cabin in the snow covered mountains"
        neg: ""
        seed: 42
        walk_seed: true
        guidance_scale: 4
        sample_steps: 25
        num_frames: 1
        fps: 1
meta:
  name: "[name]"
  version: "1.0"

Fantasy Anime Style

A Flux LoRA trained on a local computer with ostris/ai-toolkit.

Prompt

anime style, woman playing the guitar, on stage, singing a song, laser lights, punk rocker.

Prompt

anime style, thick, glossy steak that looks incredibly delicious is placed on a wooden cutting board, garnished with rosemary, whole peppercorns, and coarse salt. On the warm-toned wooden table, a glass of red wine and silver cutlery are set, with soft natural light illuminating the steak, casting appetizing shadows. Captured in a close-up angle to vividly depict the marbling and crispy exterior of the steak, as well as its juicy texture.

Prompt

anime style, An animated image of an anime girl with long black hair and a bow on her head. She is wearing a white dress with a blue belt around her waist. The girls eyes are blue and she has a serious expression on her face. The background is a vibrant blue with white letters that spell out the word 'FAS v1'.

Trigger words

You should use anime style to trigger the image generation.

Creating images using a diffusers

import torch
from diffusers import FluxPipeline

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16).to("cuda")

from huggingface_hub import hf_hub_download

pipe.load_lora_weights(
    hf_hub_download("minpeter/FAS-v1", "adapter.safetensors"), adapter_name="fas-v1"
)

pipe.set_adapters(["fas-v1"], adapter_weights=[1])

prompt = "anime style, A vibrant fantasy city with sky-high spires and unique architectural buildings stacked upon each other, featuring bright and colorful tones with intricate details – anime style background."

image = pipe(
    prompt=prompt,
    guidance_scale=3.5,
    height=1024,
    width=1024,
    num_inference_steps=20,
).images[0]

image.save("s4.png")