Update README.md
Browse files
README.md
CHANGED
@@ -12,70 +12,4 @@ library_name: diffusers
|
|
12 |
For detailed information, code, and documentation, please visit our GitHub repository:
|
13 |
[Anonymize-Anyone](https://github.com/fh2c1/Anonymize-Anyone)
|
14 |
|
15 |
-
|
16 |
-
|
17 |
-

|
18 |
-
|
19 |
-
## Model
|
20 |
-
|
21 |
-

|
22 |
-
|
23 |
-
**Anonymize Anyone** presents a novel approach to text-to-face synthesis using a Diffusion Model that considers Race Fairness. Our method uses facial segmentation masks to edit specific facial regions, and employs a Stable Diffusion v2 Inpainting model trained on a curated Asian dataset. We introduce two key losses: **βπΉπΉπΈ** (Focused Feature Enhancement Loss) to enhance performance with limited data, and **βπ«π°ππ** (Difference Loss) to address catastrophic forgetting. Finally, we apply **Simple Preference Optimization** (SimPO) for refined and enhanced image generation.
|
24 |
-
|
25 |
-
## Model Checkpoints
|
26 |
-
|
27 |
-
- [Anonymize-Anyone (Inpainting model with **βπΉπΉπΈ** and **βπ«π°ππ**)](https://huggingface.co/fh2c1/Anonymize-Anyone)
|
28 |
-
- [SimPO-LoRA (Diffusion model with **Simple Preference Optimization**)](https://huggingface.co/fh2c1/SimPO-LoRA)
|
29 |
-
|
30 |
-
### Using with Diffusersπ§¨
|
31 |
-
|
32 |
-
You can use this model directly with the `diffusers` library:
|
33 |
-
|
34 |
-
|
35 |
-
```python
|
36 |
-
import torch
|
37 |
-
from PIL import Image
|
38 |
-
from diffusers import StableDiffusionInpaintPipeline
|
39 |
-
|
40 |
-
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
|
41 |
-
sd_pipe = StableDiffusionInpaintPipeline.from_pretrained(
|
42 |
-
"fh2c1/Anonymize-Anyone",
|
43 |
-
torch_dtype=torch.float16,
|
44 |
-
safety_checker=None,
|
45 |
-
).to(device)
|
46 |
-
sd_pipe.load_lora_weights("fh2c1/SimPO-LoRA", adapter_name="SimPO")
|
47 |
-
sd_pipe.set_adapters(["SimPO"], adapter_weights=[0.5])
|
48 |
-
|
49 |
-
def generate_image(image_path, mask_path, prompt, negative_prompt, pipe, seed):
|
50 |
-
try:
|
51 |
-
in_image = Image.open(image_path)
|
52 |
-
in_mask = Image.open(mask_path)
|
53 |
-
except IOError as e:
|
54 |
-
print(f"Loading error: {e}")
|
55 |
-
return None
|
56 |
-
generator = torch.Generator(device).manual_seed(seed)
|
57 |
-
result = pipe(image=in_image, mask_image=in_mask, prompt=prompt,
|
58 |
-
negative_prompt=negative_prompt, generator=generator)
|
59 |
-
return result.images[0]
|
60 |
-
|
61 |
-
image = '/content/Anonymize-Anyone/data/2.png'
|
62 |
-
mask = "/content/Anonymize-Anyone/data/2_mask.png"
|
63 |
-
prompt = "he is an asian man."
|
64 |
-
seed = 38189219984105
|
65 |
-
negative_prompt = "low resolution, ugly, disfigured, ugly, bad, immature, cartoon, anime, 3d, painting, b&w, deformed eyes, low quailty, noise"
|
66 |
-
|
67 |
-
try:
|
68 |
-
generated_image = generate_image(image_path=image, mask_path=mask, prompt=prompt,
|
69 |
-
negative_prompt=negative_prompt, pipe=sd_pipe, seed=seed)
|
70 |
-
except TypeError as e:
|
71 |
-
print(f"TypeError : {e}")
|
72 |
-
|
73 |
-
generated_image
|
74 |
-
```
|
75 |
-

|
76 |
-
|
77 |
-
For more detailed usage instructions, including how to prepare segmentation masks and run inference, please refer to our [GitHub repository](https://github.com/fh2c1/Anonymize-Anyone).
|
78 |
-
|
79 |
-
## Training
|
80 |
-
|
81 |
-
For information on how to train the model, including the use of **βπΉπΉπΈ** (Focused Feature Enhancement Loss) and **βπ«π°ππ** (Difference Loss), please see our GitHub repository's [training section](https://github.com/fh2c1/Anonymize-Anyone#running_man-train).
|
|
|
12 |
For detailed information, code, and documentation, please visit our GitHub repository:
|
13 |
[Anonymize-Anyone](https://github.com/fh2c1/Anonymize-Anyone)
|
14 |
|
15 |
+
# We updated our model to [Anonymize Anyone v1.2](https://huggingface.co/fh2c1/SimPO-LoRA-1.2), improving race fairness in text-to-face synthesis using Simple Preference Optimization in diffusion models.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|