Create README.md

README.md

@@ -0,0 +1,72 @@
+# Anonymize Anyone: Toward Race Fairness in Text-to-Face Synthesis using Simple Preference Optimization in Diffusion Model
+
+For detailed information, code, and documentation, please visit our GitHub repository:
+[Anonymize-Anyone](https://github.com/fh2c1/Anonymize-Anyone)
+
+## Anonymize Anyone
+
+![anonymiza-anyone demo images](./assets/Fig1.png)
+
+## Model
+
+![overall_structure](./assets/Fig2.png)
+
+**Anonymize Anyone** presents a novel approach to text-to-face synthesis using a Diffusion Model that considers Race Fairness. Our method uses facial segmentation masks to edit specific facial regions, and employs a Stable Diffusion v2 Inpainting model trained on a curated Asian dataset. We introduce two key losses: **ℒ𝐹𝐹𝐸** (Focused Feature Enhancement Loss) to enhance performance with limited data, and **ℒ𝑫𝑰𝑭𝑭** (Difference Loss) to address catastrophic forgetting. Finally, we apply **Simple Preference Optimization** (SimPO) for refined and enhanced image generation.
+
+## Model Checkpoints
+
+- [Anonymize-Anyone (Inpainting model with FFEL and DIFF losses)](https://huggingface.co/fh2c1/Anonymize-Anyone)
+- [SimPO-LoRA (Diffusion model with Simple Preference Optimization)](https://huggingface.co/fh2c1/SimPO-LoRA)
+
+### Using with Diffusers🧨
+
+You can use this model directly with the `diffusers` library:
+
+
+```python
+import torch
+from PIL import Image
+from diffusers import StableDiffusionInpaintPipeline
+
+device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+sd_pipe = StableDiffusionInpaintPipeline.from_pretrained(
+    "fh2c1/Anonymize-Anyone",
+    torch_dtype=torch.float16,
+    safety_checker=None,
+).to(device)
+sd_pipe.load_lora_weights("fh2c1/SimPO-LoRA", adapter_name="SimPO")
+sd_pipe.set_adapters(["SimPO"], adapter_weights=[0.5])
+
+def generate_image(image_path, mask_path, prompt, negative_prompt, pipe, seed):
+    try:
+        in_image = Image.open(image_path)
+        in_mask = Image.open(mask_path)
+    except IOError as e:
+        print(f"Loading error: {e}")
+        return None
+    generator = torch.Generator(device).manual_seed(seed)
+    result = pipe(image=in_image, mask_image=in_mask, prompt=prompt,
+                  negative_prompt=negative_prompt, generator=generator)
+    return result.images[0]
+
+image = '/content/Anonymize-Anyone/data/2.png'
+mask = "/content/Anonymize-Anyone/data/2_mask.png"
+prompt = "he is an asian man."
+seed = 38189219984105
+negative_prompt = "low resolution, ugly, disfigured, ugly, bad, immature, cartoon, anime, 3d, painting, b&w, deformed eyes, low quailty, noise"
+
+try:
+    generated_image = generate_image(image_path=image, mask_path=mask, prompt=prompt,
+                                     negative_prompt=negative_prompt, pipe=sd_pipe, seed=seed)
+except TypeError as e:
+    print(f"TypeError : {e}")
+
+generated_image
+```
+![result](./assets/Fig3.png)
+
+For more detailed usage instructions, including how to prepare segmentation masks and run inference, please refer to our [GitHub repository](https://github.com/fh2c1/Anonymize-Anyone).
+
+## Training
+
+For information on how to train the model, including the use of **ℒ𝐹𝐹𝐸** (Focused Feature Enhancement Loss) and **ℒ𝑫𝑰𝑭𝑭** (Difference Loss), please see our GitHub repository's [training section](https://github.com/fh2c1/Anonymize-Anyone#running_man-train).