Update README.md

README.md

@@ -6,10 +6,73 @@ base_model:
 - guozinan/PuLID
 pipeline_tag: image-to-image
 ---
-## :wrench: Dependencies and Installation
+# PuLID (NeurIPS 2024)
+
+### :open_book: PuLID: Pure and Lightning ID Customization via Contrastive Alignment
+> [![arXiv](https://img.shields.io/badge/arXiv-Paper-<COLOR>.svg)](https://arxiv.org/abs/2404.16022) [![xl](https://img.shields.io/badge/🤗-HuggingFaceDemo-orange)](https://huggingface.co/spaces/yanze/PuLID) [![flux](https://img.shields.io/badge/🤗-PuLID_FLUX_demo-orange)](https://huggingface.co/spaces/yanze/PuLID-FLUX) [![Replicate](https://img.shields.io/badge/Replicate-Demo_for_PuLID-blue)](https://replicate.com/zsxkib/pulid) [![Replicate](https://img.shields.io/badge/Replicate-PuLID_FLUX-blue)](https://replicate.com/zsxkib/flux-pulid)<br>
+> Zinan Guo*, Yanze Wu*✝, Zhuowei Chen, Lang Chen, Peng Zhang, Qian He <br>
+> (*Equal Contribution, ✝Corresponding Author) <br>
+> ByteDance Inc <br>
+
+### :triangular_flag_on_post: Updates
+* **2024.10.31**: 🔥 We are happy to release our latest [models](https://huggingface.co/guozinan/PuLID), **PuLID-v1.1** and **PuLID-FLUX-v0.9.1**. See more in [Model Zoo](#european_castle-model-zoo) and [pulid v1.1 model](docs/pulid_v1.1.md). We also update a new revision for the [arXiv paper](https://arxiv.org/abs/2404.16022), which includes more results, details, and analysis, please check it out.
+* **2024.09.26**: 🎉 PuLID accepted by NeurIPS 2024
+* **2024.09.12**: We're thrilled to announce the release of the **PuLID-FLUX-v0.9.0 model**. Enjoy exploring its capabilities! 😊 [Learn more about this model](docs/pulid_for_flux.md)
+* **2024.05.23**: share the [preview of our upcoming v1.1 model](docs/v1.1_preview.md), please stay tuned
+* **2024.05.01**: release v1 codes&models, also the [🤗HuggingFace Demo](https://huggingface.co/spaces/yanze/PuLID)
+* **2024.04.25**: release arXiv paper.
+
+
+## PuLID for FLUX
+Please check the doc and demo of PuLID-FLUX [here](docs/pulid_for_flux.md).
+
+### updates
+- [x] Local gradio demo is ready now
+- [x] Online HuggingFace demo is ready now [![flux](https://img.shields.io/badge/🤗-PuLID_FLUX_demo-orange)](https://huggingface.co/spaces/yanze/PuLID-FLUX)
+- [x] We have optimized the codes to support consumer-grade GPUS, and now **PuLID-FLUX can run on a 16GB graphic card**. Check the details [here](https://github.com/ToTheBeginning/PuLID/blob/main/docs/pulid_for_flux.md#local-gradio-demo)
+- [x] (Community Implementation) Online Replicate demo is ready now [![Replicate](https://replicate.com/zsxkib/flux-pulid/badge)](https://replicate.com/zsxkib/flux-pulid)
+- [x] Local gradio demo supports 12GB graphic card now
+- [x] v0.9.1 is ready now
+
+
+Below results are generated with PuLID-FLUX.
+![pulid_flux_results](https://github.com/user-attachments/assets/7eafb90a-fdd1-4ae7-bc41-8c428d568848)
+
+
+## Examples
+Images generated with our PuLID
+![examples](https://github.com/ToTheBeginning/PuLID/assets/11482921/65610b0d-ba4f-4dc3-a74d-bd60f8f5ce37)
+Applications
+
+https://github.com/ToTheBeginning/PuLID/assets/11482921/9bdd0c8a-99e8-4eab-ab9e-39bf796cc6b8
+
+## :european_castle: Model Zoo
+
+|                                              Version                                               | Base Model |                                                                                              Description                                                                                              |
+|:--------------------------------------------------------------------------------------------------:|:----------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
+|              [PuLID-v1](https://huggingface.co/guozinan/PuLID/blob/main/pulid_v1.bin)              |    SDXL    |                                                                                             Paper model.                                                                                              |
+|        [PuLID-v1.1](https://huggingface.co/guozinan/PuLID/blob/main/pulid_v1.1.safetensors)        |    SDXL    |                                                     Compared to PuLID-v1, better compatibility, editability, facial naturalness, and similarity.                                                      |
+| [PuLID-FLUX-v0.9.0](https://huggingface.co/guozinan/PuLID/blob/main/pulid_flux_v0.9.0.safetensors) |    FLUX    |             Our first version for PuLID-FLUX, better prompt-following and image quality (since FLUX is more powerful than SDXL). But ID fidelity is not high enough for some male inputs              |
+| [PuLID-FLUX-v0.9.1](https://huggingface.co/guozinan/PuLID/blob/main/pulid_flux_v0.9.1.safetensors) |    FLUX    | Compared to PuLID-FLUX-v0.9.0, better ID fidelity. From the quantitative metric of ID similarity, the improvement is about 5 percentage points. Meanwhile, the editability remains similar as before. |
 
-- [PyTorch >= 2.0](https://pytorch.org/) if you don't need flux-dev-fp8, otherwise [PyTorch >= 2.4.1](https://pytorch.org/)
 
+## :wrench: Dependencies and Installation
+- Python >= 3.9 (Recommend to use [Anaconda](https://www.anaconda.com/download/#linux) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html))
+- [PyTorch >= 2.0](https://pytorch.org/) if you don't need flux-dev-fp8, otherwise [PyTorch >= 2.4.1](https://pytorch.org/)
+```bash
+# clone PuLID repo
+git clone https://github.com/ToTheBeginning/PuLID.git
+cd PuLID
+# create conda env
+conda create --name pulid python=3.10
+# activate env
+conda activate pulid
+# Install dependent packages
+# 1. if you don't need flux-fp8, e.g., you are using xl or flux-bf16, install the following requirements.txt
+pip install -r requirements.txt
+# 2. if you need flux-fp8 (to put flux on consumer-grade gpu), install the following requirements_fp8.txt
+pip install -r requirements_fp8.txt
+```
 
 ## :zap: Quick Inference
 ### Local Gradio Demo
@@ -21,4 +84,53 @@ python app.py
 python app_v1.1.py --base BASE_MODEL
 Usage:
   -base: can be RunDiffusion/Juggernaut-XL-v9 or Lykon/dreamshaper-xl-lightning
-```
+```
+
+### Online HuggingFace Demo
+Thanks for the GPU grant from HuggingFace team, you can try PuLID HF demo in 
+- [https://huggingface.co/spaces/yanze/PuLID](https://huggingface.co/spaces/yanze/PuLID) for SDXL
+- [https://huggingface.co/spaces/yanze/PuLID-FLUX](https://huggingface.co/spaces/yanze/PuLID-FLUX) for FLUX
+
+## :paperclip: Related Resources
+Following are some third-party implementations of PuLID we have found in the Internet. 
+We appreciate the efforts of the respective developers for making PuLID accessible to a wider audience.
+If there are any PuLID based resources and applications that we have not mentioned here, please let us know, 
+and we will include them in this list.
+
+#### Online Demo
+- **Colab**: https://github.com/camenduru/PuLID-jupyter provided by [camenduru](https://github.com/camenduru)
+- **Replicate (PuLID)**: https://replicate.com/zsxkib/pulid provided by [zsxkib](https://github.com/zsxkib)
+- **Replicate (PuLID-FLUX)**: https://replicate.com/zsxkib/flux-pulid provided by [zsxkib](https://github.com/zsxkib)
+
+#### ComfyUI
+- https://github.com/cubiq/PuLID_ComfyUI provided by [cubiq](https://github.com/cubiq), native ComfyUI implementation
+- https://github.com/ZHO-ZHO-ZHO/ComfyUI-PuLID-ZHO provided by [ZHO](https://github.com/ZHO-ZHO-ZHO), diffusers-based implementation
+
+#### WebUI
+- [SD.Next](https://github.com/vladmandic/automatic/blob/master/CHANGELOG.md#update-for-2024-11-21) Implementation provided by [vladmandic](https://github.com/vladmandic)
+- https://github.com/Mikubill/sd-webui-controlnet/pull/2838 provided by [huchenlei](https://github.com/huchenlei)
+
+#### Other Applications
+- PuLID-FLUX multi-person generation with [Regional-Prompting-FLUX](https://github.com/instantX-research/Regional-Prompting-FLUX), provided by [Anthony](https://github.com/antonioo-c)
+
+## Disclaimer
+This project strives to impact the domain of AI-driven image generation positively. Users are granted the freedom to 
+create images using this tool, but they are expected to comply with local laws and utilize it responsibly. 
+The developers do not assume any responsibility for potential misuse by users.
+
+
+##  Citation
+If PuLID is helpful, please help to ⭐ the repo.
+
+If you find this project useful for your research, please consider citing our paper:
+```bibtex
+@InProceedings{guo2024pulid,
+  title={PuLID: Pure and Lightning ID Customization via Contrastive Alignment},
+  author={Guo, Zinan and Wu, Yanze and Chen, Zhuowei and Chen, Lang and Zhang, Peng and He, Qian},
+  booktitle={Advances in Neural Information Processing Systems},
+  year={2024}
+}
+```
+
+## :e-mail: Contact
+If you have any comments or questions, please [open a new issue](https://github.com/ToTheBeginning/PuLID/issues/new/choose) or feel free to contact [Yanze Wu](https://tothebeginning.github.io/) and [Zinan Guo](mailto:[email protected]).