Update README.md
Browse files
README.md
CHANGED
@@ -4,133 +4,6 @@ language:
|
|
4 |
- en
|
5 |
base_model:
|
6 |
- guozinan/PuLID
|
7 |
-
|
8 |
-
---
|
9 |
-
# PuLID (NeurIPS 2024)
|
10 |
-
|
11 |
-
### :open_book: PuLID: Pure and Lightning ID Customization via Contrastive Alignment
|
12 |
-
> [![arXiv](https://img.shields.io/badge/arXiv-Paper-<COLOR>.svg)](https://arxiv.org/abs/2404.16022) [![xl](https://img.shields.io/badge/🤗-HuggingFaceDemo-orange)](https://huggingface.co/spaces/yanze/PuLID) [![flux](https://img.shields.io/badge/🤗-PuLID_FLUX_demo-orange)](https://huggingface.co/spaces/yanze/PuLID-FLUX) [![Replicate](https://img.shields.io/badge/Replicate-Demo_for_PuLID-blue)](https://replicate.com/zsxkib/pulid) [![Replicate](https://img.shields.io/badge/Replicate-PuLID_FLUX-blue)](https://replicate.com/zsxkib/flux-pulid)<br>
|
13 |
-
> Zinan Guo*, Yanze Wu*✝, Zhuowei Chen, Lang Chen, Peng Zhang, Qian He <br>
|
14 |
-
> (*Equal Contribution, ✝Corresponding Author) <br>
|
15 |
-
> ByteDance Inc <br>
|
16 |
-
|
17 |
-
### :triangular_flag_on_post: Updates
|
18 |
-
* **2024.10.31**: 🔥 We are happy to release our latest [models](https://huggingface.co/guozinan/PuLID), **PuLID-v1.1** and **PuLID-FLUX-v0.9.1**. See more in [Model Zoo](#european_castle-model-zoo) and [pulid v1.1 model](docs/pulid_v1.1.md). We also update a new revision for the [arXiv paper](https://arxiv.org/abs/2404.16022), which includes more results, details, and analysis, please check it out.
|
19 |
-
* **2024.09.26**: 🎉 PuLID accepted by NeurIPS 2024
|
20 |
-
* **2024.09.12**: We're thrilled to announce the release of the **PuLID-FLUX-v0.9.0 model**. Enjoy exploring its capabilities! 😊 [Learn more about this model](docs/pulid_for_flux.md)
|
21 |
-
* **2024.05.23**: share the [preview of our upcoming v1.1 model](docs/v1.1_preview.md), please stay tuned
|
22 |
-
* **2024.05.01**: release v1 codes&models, also the [🤗HuggingFace Demo](https://huggingface.co/spaces/yanze/PuLID)
|
23 |
-
* **2024.04.25**: release arXiv paper.
|
24 |
-
|
25 |
-
|
26 |
-
## PuLID for FLUX
|
27 |
-
Please check the doc and demo of PuLID-FLUX [here](docs/pulid_for_flux.md).
|
28 |
-
|
29 |
-
### updates
|
30 |
-
- [x] Local gradio demo is ready now
|
31 |
-
- [x] Online HuggingFace demo is ready now [![flux](https://img.shields.io/badge/🤗-PuLID_FLUX_demo-orange)](https://huggingface.co/spaces/yanze/PuLID-FLUX)
|
32 |
-
- [x] We have optimized the codes to support consumer-grade GPUS, and now **PuLID-FLUX can run on a 16GB graphic card**. Check the details [here](https://github.com/ToTheBeginning/PuLID/blob/main/docs/pulid_for_flux.md#local-gradio-demo)
|
33 |
-
- [x] (Community Implementation) Online Replicate demo is ready now [![Replicate](https://replicate.com/zsxkib/flux-pulid/badge)](https://replicate.com/zsxkib/flux-pulid)
|
34 |
-
- [x] Local gradio demo supports 12GB graphic card now
|
35 |
-
- [x] v0.9.1 is ready now
|
36 |
-
|
37 |
-
|
38 |
-
Below results are generated with PuLID-FLUX.
|
39 |
-
![pulid_flux_results](https://github.com/user-attachments/assets/7eafb90a-fdd1-4ae7-bc41-8c428d568848)
|
40 |
-
|
41 |
-
|
42 |
-
## Examples
|
43 |
-
Images generated with our PuLID
|
44 |
-
![examples](https://github.com/ToTheBeginning/PuLID/assets/11482921/65610b0d-ba4f-4dc3-a74d-bd60f8f5ce37)
|
45 |
-
Applications
|
46 |
-
|
47 |
-
https://github.com/ToTheBeginning/PuLID/assets/11482921/9bdd0c8a-99e8-4eab-ab9e-39bf796cc6b8
|
48 |
-
|
49 |
-
## :european_castle: Model Zoo
|
50 |
-
|
51 |
-
| Version | Base Model | Description |
|
52 |
-
|:--------------------------------------------------------------------------------------------------:|:----------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
|
53 |
-
| [PuLID-v1](https://huggingface.co/guozinan/PuLID/blob/main/pulid_v1.bin) | SDXL | Paper model. |
|
54 |
-
| [PuLID-v1.1](https://huggingface.co/guozinan/PuLID/blob/main/pulid_v1.1.safetensors) | SDXL | Compared to PuLID-v1, better compatibility, editability, facial naturalness, and similarity. |
|
55 |
-
| [PuLID-FLUX-v0.9.0](https://huggingface.co/guozinan/PuLID/blob/main/pulid_flux_v0.9.0.safetensors) | FLUX | Our first version for PuLID-FLUX, better prompt-following and image quality (since FLUX is more powerful than SDXL). But ID fidelity is not high enough for some male inputs |
|
56 |
-
| [PuLID-FLUX-v0.9.1](https://huggingface.co/guozinan/PuLID/blob/main/pulid_flux_v0.9.1.safetensors) | FLUX | Compared to PuLID-FLUX-v0.9.0, better ID fidelity. From the quantitative metric of ID similarity, the improvement is about 5 percentage points. Meanwhile, the editability remains similar as before. |
|
57 |
-
|
58 |
|
59 |
-
|
60 |
-
- Python >= 3.9 (Recommend to use [Anaconda](https://www.anaconda.com/download/#linux) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html))
|
61 |
-
- [PyTorch >= 2.0](https://pytorch.org/) if you don't need flux-dev-fp8, otherwise [PyTorch >= 2.4.1](https://pytorch.org/)
|
62 |
-
```bash
|
63 |
-
# clone PuLID repo
|
64 |
-
git clone https://github.com/ToTheBeginning/PuLID.git
|
65 |
-
cd PuLID
|
66 |
-
# create conda env
|
67 |
-
conda create --name pulid python=3.10
|
68 |
-
# activate env
|
69 |
-
conda activate pulid
|
70 |
-
# Install dependent packages
|
71 |
-
# 1. if you don't need flux-fp8, e.g., you are using xl or flux-bf16, install the following requirements.txt
|
72 |
-
pip install -r requirements.txt
|
73 |
-
# 2. if you need flux-fp8 (to put flux on consumer-grade gpu), install the following requirements_fp8.txt
|
74 |
-
pip install -r requirements_fp8.txt
|
75 |
-
```
|
76 |
-
|
77 |
-
## :zap: Quick Inference
|
78 |
-
### Local Gradio Demo
|
79 |
-
```bash
|
80 |
-
# for v1 version
|
81 |
-
python app.py
|
82 |
-
|
83 |
-
# for v1.1 version
|
84 |
-
python app_v1.1.py --base BASE_MODEL
|
85 |
-
Usage:
|
86 |
-
-base: can be RunDiffusion/Juggernaut-XL-v9 or Lykon/dreamshaper-xl-lightning
|
87 |
-
```
|
88 |
-
|
89 |
-
### Online HuggingFace Demo
|
90 |
-
Thanks for the GPU grant from HuggingFace team, you can try PuLID HF demo in
|
91 |
-
- [https://huggingface.co/spaces/yanze/PuLID](https://huggingface.co/spaces/yanze/PuLID) for SDXL
|
92 |
-
- [https://huggingface.co/spaces/yanze/PuLID-FLUX](https://huggingface.co/spaces/yanze/PuLID-FLUX) for FLUX
|
93 |
-
|
94 |
-
## :paperclip: Related Resources
|
95 |
-
Following are some third-party implementations of PuLID we have found in the Internet.
|
96 |
-
We appreciate the efforts of the respective developers for making PuLID accessible to a wider audience.
|
97 |
-
If there are any PuLID based resources and applications that we have not mentioned here, please let us know,
|
98 |
-
and we will include them in this list.
|
99 |
-
|
100 |
-
#### Online Demo
|
101 |
-
- **Colab**: https://github.com/camenduru/PuLID-jupyter provided by [camenduru](https://github.com/camenduru)
|
102 |
-
- **Replicate (PuLID)**: https://replicate.com/zsxkib/pulid provided by [zsxkib](https://github.com/zsxkib)
|
103 |
-
- **Replicate (PuLID-FLUX)**: https://replicate.com/zsxkib/flux-pulid provided by [zsxkib](https://github.com/zsxkib)
|
104 |
-
|
105 |
-
#### ComfyUI
|
106 |
-
- https://github.com/cubiq/PuLID_ComfyUI provided by [cubiq](https://github.com/cubiq), native ComfyUI implementation
|
107 |
-
- https://github.com/ZHO-ZHO-ZHO/ComfyUI-PuLID-ZHO provided by [ZHO](https://github.com/ZHO-ZHO-ZHO), diffusers-based implementation
|
108 |
-
|
109 |
-
#### WebUI
|
110 |
-
- [SD.Next](https://github.com/vladmandic/automatic/blob/master/CHANGELOG.md#update-for-2024-11-21) Implementation provided by [vladmandic](https://github.com/vladmandic)
|
111 |
-
- https://github.com/Mikubill/sd-webui-controlnet/pull/2838 provided by [huchenlei](https://github.com/huchenlei)
|
112 |
-
|
113 |
-
#### Other Applications
|
114 |
-
- PuLID-FLUX multi-person generation with [Regional-Prompting-FLUX](https://github.com/instantX-research/Regional-Prompting-FLUX), provided by [Anthony](https://github.com/antonioo-c)
|
115 |
-
|
116 |
-
## Disclaimer
|
117 |
-
This project strives to impact the domain of AI-driven image generation positively. Users are granted the freedom to
|
118 |
-
create images using this tool, but they are expected to comply with local laws and utilize it responsibly.
|
119 |
-
The developers do not assume any responsibility for potential misuse by users.
|
120 |
-
|
121 |
-
|
122 |
-
## Citation
|
123 |
-
If PuLID is helpful, please help to ⭐ the repo.
|
124 |
-
|
125 |
-
If you find this project useful for your research, please consider citing our paper:
|
126 |
-
```bibtex
|
127 |
-
@InProceedings{guo2024pulid,
|
128 |
-
title={PuLID: Pure and Lightning ID Customization via Contrastive Alignment},
|
129 |
-
author={Guo, Zinan and Wu, Yanze and Chen, Zhuowei and Chen, Lang and Zhang, Peng and He, Qian},
|
130 |
-
booktitle={Advances in Neural Information Processing Systems},
|
131 |
-
year={2024}
|
132 |
-
}
|
133 |
-
```
|
134 |
-
|
135 |
-
## :e-mail: Contact
|
136 |
-
If you have any comments or questions, please [open a new issue](https://github.com/ToTheBeginning/PuLID/issues/new/choose) or feel free to contact [Yanze Wu](https://tothebeginning.github.io/) and [Zinan Guo](mailto:[email protected]).
|
|
|
4 |
- en
|
5 |
base_model:
|
6 |
- guozinan/PuLID
|
7 |
+
library_name: pytorch
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
8 |
|
9 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|