metadata
license: openrail
datasets:
- laion/laion2B-en-aesthetic
language:
- en
Based on https://github.com/lllyasviel/ControlNet/discussions/318
accelerate launch train_controlnet.py ^
--pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5" ^
--output_dir="control-edgedrawing-[version]-fp16/" ^
--dataset_name="mydataset" ^
--mixed_precision="fp16" ^
--resolution=512 ^
--learning_rate=1e-5 ^
--train_batch_size=1 ^
--gradient_accumulation_steps=4 ^
--gradient_checkpointing ^
--use_8bit_adam ^
--enable_xformers_memory_efficient_attention ^
--set_grads_to_none ^
--seed=0
Images converted with https://github.com/shaojunluo/EDLinePython
Default settings are:
smoothed=False
{ 'ksize' : 5
, 'sigma' : 1.0
, 'gradientThreshold': 36
, 'anchorThreshold' : 8
, 'scanIntervals' : 1
}
control-edgedrawing-default-drop50-fp16-checkpoint-40000
additional argument: --proportion_empty_prompts=0.5
Trained for 40000 steps with default settings
=> empty prompts were probably too excessive
control-edgedrawing-default-noisy-drop0-fp16-checkpoint-40000
smoothed=True, but no empty prompts
Trained for 40000 steps with default settings
=> conditioning images are too noisy