fh2c1 commited on
Commit
082a57a
Β·
verified Β·
1 Parent(s): 1022fa1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -67
README.md CHANGED
@@ -12,70 +12,4 @@ library_name: diffusers
12
  For detailed information, code, and documentation, please visit our GitHub repository:
13
  [Anonymize-Anyone](https://github.com/fh2c1/Anonymize-Anyone)
14
 
15
- ## Anonymize Anyone
16
-
17
- ![anonymiza-anyone demo images](./assets/Fig1.png)
18
-
19
- ## Model
20
-
21
- ![overall_structure](./assets/Fig2.png)
22
-
23
- **Anonymize Anyone** presents a novel approach to text-to-face synthesis using a Diffusion Model that considers Race Fairness. Our method uses facial segmentation masks to edit specific facial regions, and employs a Stable Diffusion v2 Inpainting model trained on a curated Asian dataset. We introduce two key losses: **ℒ𝐹𝐹𝐸** (Focused Feature Enhancement Loss) to enhance performance with limited data, and **ℒ𝑫𝑰𝑭𝑭** (Difference Loss) to address catastrophic forgetting. Finally, we apply **Simple Preference Optimization** (SimPO) for refined and enhanced image generation.
24
-
25
- ## Model Checkpoints
26
-
27
- - [Anonymize-Anyone (Inpainting model with **ℒ𝐹𝐹𝐸** and **ℒ𝑫𝑰𝑭𝑭**)](https://huggingface.co/fh2c1/Anonymize-Anyone)
28
- - [SimPO-LoRA (Diffusion model with **Simple Preference Optimization**)](https://huggingface.co/fh2c1/SimPO-LoRA)
29
-
30
- ### Using with Diffusers🧨
31
-
32
- You can use this model directly with the `diffusers` library:
33
-
34
-
35
- ```python
36
- import torch
37
- from PIL import Image
38
- from diffusers import StableDiffusionInpaintPipeline
39
-
40
- device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
41
- sd_pipe = StableDiffusionInpaintPipeline.from_pretrained(
42
- "fh2c1/Anonymize-Anyone",
43
- torch_dtype=torch.float16,
44
- safety_checker=None,
45
- ).to(device)
46
- sd_pipe.load_lora_weights("fh2c1/SimPO-LoRA", adapter_name="SimPO")
47
- sd_pipe.set_adapters(["SimPO"], adapter_weights=[0.5])
48
-
49
- def generate_image(image_path, mask_path, prompt, negative_prompt, pipe, seed):
50
- try:
51
- in_image = Image.open(image_path)
52
- in_mask = Image.open(mask_path)
53
- except IOError as e:
54
- print(f"Loading error: {e}")
55
- return None
56
- generator = torch.Generator(device).manual_seed(seed)
57
- result = pipe(image=in_image, mask_image=in_mask, prompt=prompt,
58
- negative_prompt=negative_prompt, generator=generator)
59
- return result.images[0]
60
-
61
- image = '/content/Anonymize-Anyone/data/2.png'
62
- mask = "/content/Anonymize-Anyone/data/2_mask.png"
63
- prompt = "he is an asian man."
64
- seed = 38189219984105
65
- negative_prompt = "low resolution, ugly, disfigured, ugly, bad, immature, cartoon, anime, 3d, painting, b&w, deformed eyes, low quailty, noise"
66
-
67
- try:
68
- generated_image = generate_image(image_path=image, mask_path=mask, prompt=prompt,
69
- negative_prompt=negative_prompt, pipe=sd_pipe, seed=seed)
70
- except TypeError as e:
71
- print(f"TypeError : {e}")
72
-
73
- generated_image
74
- ```
75
- ![result](./assets/Fig3.png)
76
-
77
- For more detailed usage instructions, including how to prepare segmentation masks and run inference, please refer to our [GitHub repository](https://github.com/fh2c1/Anonymize-Anyone).
78
-
79
- ## Training
80
-
81
- For information on how to train the model, including the use of **ℒ𝐹𝐹𝐸** (Focused Feature Enhancement Loss) and **ℒ𝑫𝑰𝑭𝑭** (Difference Loss), please see our GitHub repository's [training section](https://github.com/fh2c1/Anonymize-Anyone#running_man-train).
 
12
  For detailed information, code, and documentation, please visit our GitHub repository:
13
  [Anonymize-Anyone](https://github.com/fh2c1/Anonymize-Anyone)
14
 
15
+ # We updated our model to [Anonymize Anyone v1.2](https://huggingface.co/fh2c1/SimPO-LoRA-1.2), improving race fairness in text-to-face synthesis using Simple Preference Optimization in diffusion models.