ご利用の際は下記のライセンス内容を十分にご確認ください。
HonyararaMixはアニメ調の人物を出力しやすいように調整されたStable Diffusion 2.1 解像度768対応版モデルです。
ベースモデルのIlluminati Diffusion 1.0へWaifu Diffusion 1.5 Beta 、Waifu Diffusion 1.5 Beta2、Replicant-V1.0、Stable Diffusion 2.1を用いマージしたものをベースに
DeDeDeDatasetによる学習などを行い調整したものです。
旧HonyararaMixにはライセンス上の問題があったため、それらを解決したモデルになります。
Prompt/Bad Promptについては、ベースモデルとなった二種のモデルが用意した文書の内容、およびTI Embeddingがこのモデルでも作用します。
WDについてはこちらを参照。Illuminati DiffusionについてはNfixerが割とよく効く印象です。
例
best quality, masterpiece, great nature and secret garden with anime 1girl with twintail green hair and green eye and beautiful iris and (((from behind)))
Negative prompt: nfixer,3d, flat shading, flat shadow, bad anatomy, bad hands, missing fingers, extra digit, fewer digits looking at viewer
Steps: 28, Sampler: DDIM, CFG scale: 9, Seed: 43662843, Size: 1024x768, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B
newest, masterpiece, exceptional, anime game cg of cute cute cute 1girl sitting on a bench in harajuku park with detailed wavy pink hair pink and detailed yellow hair
Negative prompt: nfixer, 3d, flat shading, flat shadow, inaccurate limb, bad hands, missing fingers, extra digit, fewer digits
Steps: 28, Sampler: DPM++ SDE Karras, CFG scale: 7.5, Seed: 4179369813, Size: 768x1152, Model hash: 4dbf24eef6, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B
newest, masterpiece, exceptional, anime game cg of kawaii samurai 1girl, fighting form and (((intricate hands))), fist and dynamic angle with detailed pink hair and detailed yellow eye and beautiful iris in japanese background
Negative prompt: nfixer, 3d, flat shading, flat shadow, ((bad hands, missing fingers, extra digit, fewer digits))
Steps: 16, Sampler: DPM++ SDE Karras, CFG scale: 7.5, Seed: 2246223688, Size: 768x1152, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B
((masterpiece, best quality)),best aesthetic, 1girl, solo, black skirt, blue eyes, guitar, headphones, instrument, long hair, music, one side up, pink hair, playing guiter, pleated skirt, black shirt, indoors
Negative prompt: nfixer,missing finger, extra digits, fewer digits,((mutated hands and fingers)), 3d, flat shadow
Steps: 16, Sampler: DPM++ SDE Karras, CFG scale: 7, Seed: 3689716537, Size: 768x1152, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B
newest, masterpiece, exceptional, anime game cg of teenage 1boy wizard bust shot casting fire magic spell in New York City, detailed eye and beautiful iris,picturesque, golden hour, dynamic pose
Negative prompt: nfixer, 3d, flat shading, flat shadow, inaccurate limb, bad hands, missing fingers, extra digit, fewer digits
Steps: 25, Sampler: Euler a, CFG scale: 7.5, Seed: 1414427875, Size: 768x1152, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B
newest, masterpiece, exceptional, anime game cg of samurai (((((old man))))), fighting pose, dynamic angle, fist with detailed pink hair and detailed yellow eye and beautiful iris in japanese background
nfixer, 3d, flat shading, flat shadow, inaccurate limb, bad hands, missing fingers, extra digit, fewer digits
Steps: 25, Sampler: DDIM, CFG scale: 9, Seed: 3970412380, Size: 768x1152, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B
マージ・学習手順について
Illuminati Diffusion 1.0にWaifu Diffusion 1.5 Beta1を差分マージする
Interpolation Method Primary Model Secondary Model Tertiary Model Merge Name Add Difference @ 0.6 Illuminati Diffusion 1.0 Waifu Diffusion 1.5 Beta1 Stable Diffusion v2-1_768-ema-pruned HonyararaMix_pre1 HonyararaMix_pre1に対し、自前で用意した他Diffusion Modelの出力からなる素材画像にて学習させる
用意の際に利用したサービス/モデルは、SD2.1、Novel AI、WD1.3/1.4、CoolJapan Diffusion 2.1、Dreamlike Photoreal 2.0。
使用したソフトはkohya版Fine Tuning。
総数は30000程、flipしたものと合わせてlearning rateは5e-6、以下の設定で学習させ、
試行錯誤の後にepoch 2相当のものをHonyararaMix_pre2とした。
accelerate launch --num_cpu_threads_per_process 1 fine_tune.py
--pretrained_model_name_or_path=Honyarara_pre1.safetensors
--in_json meta_lat.json --train_data_dir=train_data
--output_dir=fine_tuned
--shuffle_caption
--train_batch_size=4
--learning_rate=5e-6
--max_train_steps=60000
--v2
--v_parameterization
--use_8bit_adam
--xformers
--mixed_precision=bf16
--save_every_n_epochs=2
--save_precision=float
--max_token_length=150
--seed=42
Honyarara_pre1にHonyarara_pre2を加重平均でマージする
Interpolation Method Primary Model Secondary Model Merge Name Weighted Sum @ 0.5 HonyararaMix_pre1 HonyararaMix_pre2 HonyararaMix_pre3 HonyararaMix_pre3にIlluminati Diffusion 1.0を階層マージする
使用マージ用スクリプトはSuper Merger。Model: A Model: B Weight Merge Name HonyararaMix_pre3 Illuminati Diffusion 1.0 1.0,0.45,0.0,0.0,0.0,0.0,0.0,0.1,0.3,0.5,0.7,0.9,1.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1,0.3,0.5,0.7,0.9,1.0 HonyararaMix_pre4 HonyararaMix_pre4にWaifu Diffusion 1.5 Beta2 Aestheticを差分マージする
Interpolation Method Primary Model Secondary Model Tertiary Model Merge Name Add Difference @ 0.5 HonyararaMix_pre4 WD 1.5 Beta 2 Aesthetic WD 1.5 Beta 2 HonyararaMix_pre5 HonyararaMix_pre5にReplicant-V1.0を加重平均でマージする
Interpolation Method Primary Model Secondary Model Merge Name Weighted Sum @ 0.5 HonyararaMix_pre5 Replicant-V1.0 HonyararaMix A HonyararaMix AにHonyararaMix_pre4を加重平均でマージする
Interpolation Method Primary Model Secondary Model Merge Name Weighted Sum @ 0.5 HonyararaMix A HonyararaMix_pre4 HonyararaMix B
ライセンスについて
Fair AI Public License 1.0-SD (https://freedevproject.org/faipl-1.0-sd/)