Commit
·
597e0e4
1
Parent(s):
a63f45e
debug
Browse files- .gitattributes +0 -39
- .gitignore +0 -1
- GirlsFrontline_SDXL_LoRA.safetensors +0 -3
- README.md +0 -128
- assets/Image_20240210144837_out.png +0 -3
- assets/Image_20240210150121_out.png +0 -3
- assets/Image_20240210150741_out.png +0 -3
- assets/events.out.tfevents.1707400528.autodl-container-76db429c66-9866c56d.2317.0 +0 -3
- assets/events.out.tfevents.1707463908.autodl-container-76db429c66-9866c56d.1301.0 +0 -3
- assets/example.png +0 -3
- assets/loss_average.svg +0 -1
- assets/loss_epoch.svg +0 -1
- assets/lr_d_lr.svg +0 -1
- output/GirlsFrontline-000005.safetensors.0 +0 -3
- output/GirlsFrontline-000007.safetensors.0 +0 -3
- output/GirlsFrontline-00006.safetensors.0 +0 -3
.gitattributes
DELETED
@@ -1,39 +0,0 @@
|
|
1 |
-
*.7z filter=lfs diff=lfs merge=lfs -text
|
2 |
-
*.arrow filter=lfs diff=lfs merge=lfs -text
|
3 |
-
*.bin filter=lfs diff=lfs merge=lfs -text
|
4 |
-
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
5 |
-
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
6 |
-
*.ftz filter=lfs diff=lfs merge=lfs -text
|
7 |
-
*.gz filter=lfs diff=lfs merge=lfs -text
|
8 |
-
*.h5 filter=lfs diff=lfs merge=lfs -text
|
9 |
-
*.joblib filter=lfs diff=lfs merge=lfs -text
|
10 |
-
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
11 |
-
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
12 |
-
*.model filter=lfs diff=lfs merge=lfs -text
|
13 |
-
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
14 |
-
*.npy filter=lfs diff=lfs merge=lfs -text
|
15 |
-
*.npz filter=lfs diff=lfs merge=lfs -text
|
16 |
-
*.onnx filter=lfs diff=lfs merge=lfs -text
|
17 |
-
*.ot filter=lfs diff=lfs merge=lfs -text
|
18 |
-
*.parquet filter=lfs diff=lfs merge=lfs -text
|
19 |
-
*.pb filter=lfs diff=lfs merge=lfs -text
|
20 |
-
*.pickle filter=lfs diff=lfs merge=lfs -text
|
21 |
-
*.pkl filter=lfs diff=lfs merge=lfs -text
|
22 |
-
*.pt filter=lfs diff=lfs merge=lfs -text
|
23 |
-
*.pth filter=lfs diff=lfs merge=lfs -text
|
24 |
-
*.rar filter=lfs diff=lfs merge=lfs -text
|
25 |
-
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
26 |
-
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
27 |
-
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
28 |
-
*.tar filter=lfs diff=lfs merge=lfs -text
|
29 |
-
*.tflite filter=lfs diff=lfs merge=lfs -text
|
30 |
-
*.tgz filter=lfs diff=lfs merge=lfs -text
|
31 |
-
*.wasm filter=lfs diff=lfs merge=lfs -text
|
32 |
-
*.xz filter=lfs diff=lfs merge=lfs -text
|
33 |
-
*.zip filter=lfs diff=lfs merge=lfs -text
|
34 |
-
*.zst filter=lfs diff=lfs merge=lfs -text
|
35 |
-
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
36 |
-
*.png filter=lfs diff=lfs merge=lfs -text
|
37 |
-
output/GirlsFrontline-000005.safetensors.0 filter=lfs diff=lfs merge=lfs -text
|
38 |
-
output/GirlsFrontline-000007.safetensors.0 filter=lfs diff=lfs merge=lfs -text
|
39 |
-
output/GirlsFrontline-00006.safetensors.0 filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.gitignore
DELETED
@@ -1 +0,0 @@
|
|
1 |
-
datasets/
|
|
|
|
GirlsFrontline_SDXL_LoRA.safetensors
DELETED
@@ -1,3 +0,0 @@
|
|
1 |
-
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:f168686ba31428c672400c5a20e07c8c8b66addf2ce85c8315407eb52316dd1a
|
3 |
-
size 228487908
|
|
|
|
|
|
|
|
README.md
DELETED
@@ -1,128 +0,0 @@
|
|
1 |
-
---
|
2 |
-
language: en
|
3 |
-
thumbnail: ./assets/example.png
|
4 |
-
tags:
|
5 |
-
- text-to-image
|
6 |
-
- diffusion models
|
7 |
-
- LoRA fine-tuning
|
8 |
-
- animagine-xl-3.0
|
9 |
-
- stable_diffusion_xl
|
10 |
-
- kohya_ss
|
11 |
-
- waifu2x
|
12 |
-
license: apache-2.0
|
13 |
-
model:
|
14 |
-
name: SDXL-LoRA-Text2Image
|
15 |
-
description: A model based on SDXL with LoRA fine-tuning for Text-To-Image generation.
|
16 |
-
pipeline_tag: image-generation
|
17 |
-
repo: https://huggingface.co/TfiyuenLau/GirlsFrontline_SDXL_LoRA
|
18 |
-
library: huggingface
|
19 |
-
framework: pytorch
|
20 |
-
version: 1.0.0
|
21 |
-
pretrained_model: stable_diffusion_xl
|
22 |
-
base_model: animagine-xl-3.0
|
23 |
-
fine_tuner: kohya_ss
|
24 |
-
data_augmentation: waifu2x
|
25 |
-
task: text-to-image
|
26 |
-
---
|
27 |
-
|
28 |
-
# 基于SDXL模型LoRA微调实现Text-To-Image文生图
|
29 |
-
|
30 |
-
![example](./assets/example.png)
|
31 |
-
|
32 |
-
## 一、Model Library
|
33 |
-
1. 微调数据集:[基于SDXL模型的《少女前线2:追放》LoRA微调数据集](https://www.kaggle.com/datasets/yukikonata/sdxl2lora)
|
34 |
-
2. 预训练模型:[stable_diffusion_xl](https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/stable_diffusion_xl)
|
35 |
-
3. 底模:[animagine-xl-3.0](https://huggingface.co/cagliostrolab/animagine-xl-3.0)
|
36 |
-
4. SDXL LoRA微调训练器:[kohya_ss](https://github.com/bmaltais/kohya_ss)
|
37 |
-
5. 数据集画质增强:[waifu2x](https://github.com/nagadomi/waifu2x)
|
38 |
-
|
39 |
-
## 二、Prompt Dict
|
40 |
-
1. 少前2追放角色5
|
41 |
-
* 佩里缇亚: PKPSP
|
42 |
-
* 塞布丽娜: SPAS12
|
43 |
-
* 托洛洛: AKAlfa
|
44 |
-
* 桑朵莱希: G36
|
45 |
-
* 琼玖: QBZ191
|
46 |
-
* 维普雷: Vepr12
|
47 |
-
* 莫辛纳甘: MosinNagant
|
48 |
-
* 黛烟: QBZ95
|
49 |
-
* 克罗丽科: Kroliko
|
50 |
-
* 夏克里: XCRL
|
51 |
-
* 奇塔: MP7
|
52 |
-
* 寇尔芙: TaurusCurve
|
53 |
-
* 科谢尼娅: APS
|
54 |
-
* 纳甘: Nagant1895
|
55 |
-
* 纳美西丝: OM50
|
56 |
-
* 莉塔拉: GalilARM
|
57 |
-
* 闪电: OTs14
|
58 |
-
|
59 |
-
2. Pixiv画师风格
|
60 |
-
* おにねこ(鬼猫): Onineko26
|
61 |
-
* 麻生: AsouAsabu
|
62 |
-
* mignon: Mignon
|
63 |
-
* migolu: Migolu
|
64 |
-
|
65 |
-
## 三、使用方式
|
66 |
-
1. 安装部分环境(默认已安装pytorch等必要环境)
|
67 |
-
~~~sh
|
68 |
-
pip install diffusers --upgrade
|
69 |
-
pip install transformers accelerate safetensors
|
70 |
-
~~~
|
71 |
-
|
72 |
-
2. 使用Hugging Face下载并使用底模(animagine-xl-3.0)和LoRA模型
|
73 |
-
~~~python
|
74 |
-
import torch
|
75 |
-
import datetime
|
76 |
-
from PIL import Image
|
77 |
-
import matplotlib.pyplot as plt
|
78 |
-
from diffusers import (
|
79 |
-
StableDiffusionXLPipeline,
|
80 |
-
EulerAncestralDiscreteScheduler,
|
81 |
-
AutoencoderKL
|
82 |
-
)
|
83 |
-
|
84 |
-
# LoRA Hugging Face ID
|
85 |
-
lora_id = "TfiyuenLau/GirlsFrontline_SDXL_LoRA"
|
86 |
-
|
87 |
-
# Load VAE component
|
88 |
-
vae = AutoencoderKL.from_pretrained(
|
89 |
-
"madebyollin/sdxl-vae-fp16-fix",
|
90 |
-
torch_dtype=torch.float16
|
91 |
-
)
|
92 |
-
|
93 |
-
# Configure the pipeline
|
94 |
-
pipe = StableDiffusionXLPipeline.from_pretrained(
|
95 |
-
"cagliostrolab/animagine-xl-3.0",
|
96 |
-
vae=vae,
|
97 |
-
torch_dtype=torch.float16,
|
98 |
-
use_safetensors=True,
|
99 |
-
)
|
100 |
-
pipe.load_lora_weights(lora_id)
|
101 |
-
pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)
|
102 |
-
pipe.to('cuda')
|
103 |
-
~~~
|
104 |
-
|
105 |
-
3. 生成图像
|
106 |
-
~~~python
|
107 |
-
# Define Prompt
|
108 |
-
output = "./output.png"
|
109 |
-
prompt = "1girl, OTs14, gloves, looking at viewer, smile, food, holding, solo, closed mouth, sitting, yellow eyes, black gloves, masterpiece, best quality"
|
110 |
-
negative_prompt = "nsfw, lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, artist name"
|
111 |
-
|
112 |
-
# Generate Image
|
113 |
-
image = pipe(
|
114 |
-
prompt,
|
115 |
-
negative_prompt=negative_prompt,
|
116 |
-
width=1024,
|
117 |
-
height=1024,
|
118 |
-
guidance_scale=7,
|
119 |
-
num_inference_steps=28
|
120 |
-
).images[0]
|
121 |
-
|
122 |
-
# Save & Show
|
123 |
-
image.save(output)
|
124 |
-
image = Image.open(output)
|
125 |
-
plt.axis('off')
|
126 |
-
plt.imshow(image)
|
127 |
-
image.close()
|
128 |
-
~~~
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
assets/Image_20240210144837_out.png
DELETED
Git LFS Details
|
assets/Image_20240210150121_out.png
DELETED
Git LFS Details
|
assets/Image_20240210150741_out.png
DELETED
Git LFS Details
|
assets/events.out.tfevents.1707400528.autodl-container-76db429c66-9866c56d.2317.0
DELETED
@@ -1,3 +0,0 @@
|
|
1 |
-
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:53e90d8de21a92470deb6bcdb3e5c92d84eb64c1357e2417e65ebe20dc47e30f
|
3 |
-
size 8434946
|
|
|
|
|
|
|
|
assets/events.out.tfevents.1707463908.autodl-container-76db429c66-9866c56d.1301.0
DELETED
@@ -1,3 +0,0 @@
|
|
1 |
-
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:0f83386197ebd96624993d1d09b27a18aa5ff0dfacb4937c6f8b3e3c4c2a2e62
|
3 |
-
size 20561120
|
|
|
|
|
|
|
|
assets/example.png
DELETED
Git LFS Details
|
assets/loss_average.svg
DELETED
assets/loss_epoch.svg
DELETED
assets/lr_d_lr.svg
DELETED
output/GirlsFrontline-000005.safetensors.0
DELETED
@@ -1,3 +0,0 @@
|
|
1 |
-
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:c5cf838604d00be38f882299b3c6d5b5dbe35d3ce25ab6f50e811117c0cf52a5
|
3 |
-
size 228487908
|
|
|
|
|
|
|
|
output/GirlsFrontline-000007.safetensors.0
DELETED
@@ -1,3 +0,0 @@
|
|
1 |
-
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:f168686ba31428c672400c5a20e07c8c8b66addf2ce85c8315407eb52316dd1a
|
3 |
-
size 228487908
|
|
|
|
|
|
|
|
output/GirlsFrontline-00006.safetensors.0
DELETED
@@ -1,3 +0,0 @@
|
|
1 |
-
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:10467e4525d162084aecedb433f38951f4a5d964f142a3287762bb8ca05f851c
|
3 |
-
size 228487908
|
|
|
|
|
|
|
|