metadata

license: other
base_model: stabilityai/stable-diffusion-3.5-medium
tags:
  - sd3
  - sd3-diffusers
  - text-to-image
  - diffusers
  - simpletuner
  - not-for-all-audiences
  - lora
  - template:sd-lora
  - standard
inference: true
widget:
  - text: unconditional (blank prompt)
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_0_0.png
  - text: >-
      A simplistic, hand-drawn illustration of an elephant. the elephant is
      depicted in a walking pose, with its trunk raised slightly. the drawing is
      done in black ink on a white background. the elephant's posture and the
      positioning of its legs suggest movement. the style is minimalistic, with
      clean lines and a lack of intricate details. the lighting appears to be
      coming from the top left, casting a shadow on the right side of the
      elephant.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_1_0.png

simpletuner-lora

This is a standard PEFT LoRA derived from stabilityai/stable-diffusion-3.5-medium.

The main validation prompt used during training was:

A simplistic, hand-drawn illustration of an elephant. the elephant is depicted in a walking pose, with its trunk raised slightly. the drawing is done in black ink on a white background. the elephant's posture and the positioning of its legs suggest movement. the style is minimalistic, with clean lines and a lack of intricate details. the lighting appears to be coming from the top left, casting a shadow on the right side of the elephant.

Validation settings

CFG: 7.5
CFG Rescale: 0.0
Steps: 35
Sampler: FlowMatchEulerDiscreteScheduler
Seed: 42
Resolution: 512x512
Skip-layer guidance:

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt
unconditional (blank prompt)

Negative Prompt
blurry, cropped, ugly

Prompt
A simplistic, hand-drawn illustration of an elephant. the elephant is depicted in a walking pose, with its trunk raised slightly. the drawing is done in black ink on a white background. the elephant's posture and the positioning of its legs suggest movement. the style is minimalistic, with clean lines and a lack of intricate details. the lighting appears to be coming from the top left, casting a shadow on the right side of the elephant.

Negative Prompt
blurry, cropped, ugly

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

Training epochs: 6
Training steps: 4000
Learning rate: 0.0001
- Learning rate schedule: cosine
- Warmup steps: 100
Max grad norm: 0.01
Effective batch size: 16
- Micro-batch size: 4
- Gradient accumulation steps: 4
- Number of GPUs: 1
Gradient checkpointing: True
Prediction type: flow-matching (extra parameters=['shift=3'])
Optimizer: adamw_bf16
Trainable parameter precision: Pure BF16
Caption dropout probability: 10.0%
LoRA Rank: 128
LoRA Alpha: None
LoRA Dropout: 0.1
LoRA initialisation style: default

Datasets

pacs

Repeats: 0
Total number of images: 9980
Total number of aspect buckets: 1
Resolution: 1.0 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

Inference

import torch
from diffusers import DiffusionPipeline

model_id = 'stabilityai/stable-diffusion-3.5-medium'
adapter_id = 'Cha-Imaa/simpletuner-lora'
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
pipeline.load_lora_weights(adapter_id)

prompt = "A simplistic, hand-drawn illustration of an elephant. the elephant is depicted in a walking pose, with its trunk raised slightly. the drawing is done in black ink on a white background. the elephant's posture and the positioning of its legs suggest movement. the style is minimalistic, with clean lines and a lack of intricate details. the lighting appears to be coming from the top left, casting a shadow on the right side of the elephant."
negative_prompt = 'blurry, cropped, ugly'

## Optional: quantise the model to save on vram.
## Note: The model was quantised during training, and so it is recommended to do the same during inference time.
from optimum.quanto import quantize, freeze, qint8
quantize(pipeline.transformer, weights=qint8)
freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
image = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=35,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=512,
    height=512,
    guidance_scale=7.5,
).images[0]
image.save("output.png", format="PNG")