chaojiemao commited on
Commit
f0ac4cc
·
verified ·
1 Parent(s): 1413e3a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +124 -12
README.md CHANGED
@@ -56,22 +56,134 @@ tags:
56
  ## 🔥 ACE Models
57
  ACE++ provides a comprehensive toolkit for image editing and generation to support various applications. We encourage developers to choose the appropriate model based on their own scenarios and to fine-tune their models using data from their specific scenarios to achieve more stable results.
58
 
59
- | **Model** | *Tuning Method* | **Description** | **Examples** | **Model Path(scepter_path)** |
60
- |:------------------:|:------------------------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
61
- | ACE++ Portrait | LoRA + ACE Data | Portrait-consistent generation to <br> maintain the consistency of the portrait . | <img src="./assets/samples/portrait/human_1.jpg" alt="Image 1" height="256" style="display:inline-block;"/> <img src="./assets/samples/portrait/human_1_1.jpg" alt="Image 2" height="256" style="display:inline-block;"/><br><p style="display:inline-block;">Maintain the facial features, A girl is wearing a neat police uniform and sporting a badge. <br> She is smiling with a friendly and confident demeanor. The background is blurred, featuring a cartoon logo. </p> | [![ModelScope link](https://img.shields.io/badge/ModelScope-Model-blue)](https://www.modelscope.cn/models/iic/ACE_Plus/) <br> ms://iic/ACE_Plus@portrait/xxxx.safetensors] <br> [![HuggingFace link](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-yellow)](https://huggingface.co/ali-vilab/ACE_Plus/tree/main/portrait/) <br> hf://ali-vilab/ACE_Plus@portrait/xxxx.safetensors |
62
- | ACE++ Subject | LoRA + ACE Data | Subject-driven image generation <br> task to maintain the consistency of <br> a specific subject in different scenes. | <img src="./assets/samples/subject/subject_1.jpg" alt="Image 1" height="256" style="display:inline-block;"/> <img src="./assets/samples/subject/subject_1_1.jpg" alt="Image 2" height="256" style="display:inline-block;"/><br><p style="display:inline-block;">Display the logo in a minimalist style printed in white on a matte black ceramic coffee mug, alongside a steaming cup of coffee on a cozy cafe table. </p> | [![ModelScope link](https://img.shields.io/badge/ModelScope-Model-blue)](https://www.modelscope.cn/models/iic/ACE_Plus/) <br> ms://iic/ACE_Plus@subject/xxxx.safetensors]<br> [![HuggingFace link](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-yellow)](https://huggingface.co/ali-vilab/ACE_Plus/tree/main/subject/) <br> hf://ali-vilab/ACE_Plus@subject/xxxx.safetensors |
63
- | ACE++ LocalEditing | LoRA + ACE Data | Redrawing the mask area of images <br> while maintaining the original structural <br> information of the edited area. | <img src="./assets/samples/local/local_1.webp" alt="Image 1" height="160" style="display:inline-block;"/> <img src="./assets/samples/local/local_1_m.webp" alt="Image 1" height="160" style="display:inline-block;"/> <img src="./assets/samples/local/local_1_1.jpg" alt="Image 2" height="160" style="display:inline-block;"/><br><p style="display:inline-block;">By referencing the mask, restore a partial image from the doodle {image} that aligns with the textual explanation: "1 white old owl". </p> | [![ModelScope link](https://img.shields.io/badge/ModelScope-Model-blue)](https://www.modelscope.cn/models/iic/ACE_Plus/) <br> ms://iic/ACE_Plus@local_editing/xxxx.safetensors] <br> [![HuggingFace link](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-yellow)](https://huggingface.co/ali-vilab/ACE_Plus/tree/main/local_editing/) <br> hf://ali-vilab/ACE_Plus@local_editing/xxxx.safetensors |
64
- | ACE++ | Full Finetune + ACE Data |Fully finetuning a composite model with ACE’s <br> data to support various editing and reference <br> generation tasks through an instructive approach. | | Will be released soon. |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
65
 
66
  ## 🔥 Applications
67
  The ACE++ model supports a wide range of downstream tasks through simple adaptations. Here are some examples, and we look forward to seeing the community explore even more exciting applications utilizing the ACE++ model.
68
 
69
- | Application | ACE++ Model | Examples |
70
- |:-------------------:|:--------------:|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
71
- | Try On | ACE++ Subject | <img src="./assets/samples/application/try_on/1_ref.png" alt="Image 1" height="256" style="display:inline-block;"/><img src="./assets/samples/application/try_on/1_1_edit.png" alt="Image 1" height="256" style="display:inline-block;"/><img src="./assets/samples/application/try_on/1_1_m.png" alt="Image 1" height="256" style="display:inline-block;"/> <img src="./assets/samples/application/try_on/1_1_res.png" alt="Image 2" height="256" style="display:inline-block;"/><br> <p style="display:inline-block;">The woman dresses this skirt.</p> |
72
- | Logo Paste | ACE++ Subject | <img src="./assets/samples/application/logo_paste/1_ref.png" alt="Image 1" height="256" style="display:inline-block;"/><img src="./assets/samples/application/logo_paste/1_1_edit.png" alt="Image 1" height="256" style="display:inline-block;"/><img src="./assets/samples/application/logo_paste/1_1_m.png" alt="Image 1" height="256" style="display:inline-block;"/> <img src="./assets/samples/application/logo_paste/1_1_res.webp" alt="Image 2" height="256" style="display:inline-block;"/><br> <p style="display:inline-block;">The logo is printed on the headphones.</p> |
73
- | Photo Editing | ACE++ Subject | <img src="./assets/samples/application/photo_editing/1_ref.png" alt="Image 1" height="256" style="display:inline-block;"/><img src="./assets/samples/application/photo_editing/1_1_edit.png" alt="Image 1" height="256" style="display:inline-block;"/><img src="./assets/samples/application/photo_editing/1_1_m.png" alt="Image 1" height="256" style="display:inline-block;"/> <img src="./assets/samples/application/photo_editing/1_1_res.jpg" alt="Image 2" height="256" style="display:inline-block;"/><br> <p style="display:inline-block;">The item is put on the ground.</p> |
74
- | Movie Poster Editor | ACE++ Portrait | <img src="./assets/samples/application/movie_poster/1_ref.png" alt="Image 1" height="256" style="display:inline-block;"/><img src="./assets/samples/application/movie_poster/1_1_edit.png" alt="Image 1" height="256" style="display:inline-block;"/><img src="./assets/samples/application/movie_poster/1_1_m.png" alt="Image 1" height="256" style="display:inline-block;"/> <img src="./assets/samples/application/movie_poster/1_1_res.webp" alt="Image 2" height="256" style="display:inline-block;"/><br> <p style="display:inline-block;">The man is facing the camera and is smiling.</p> |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
75
 
76
  ## ⚙️️ Installation
77
  Download the code using the following command:
 
56
  ## 🔥 ACE Models
57
  ACE++ provides a comprehensive toolkit for image editing and generation to support various applications. We encourage developers to choose the appropriate model based on their own scenarios and to fine-tune their models using data from their specific scenarios to achieve more stable results.
58
 
59
+
60
+
61
+
62
+ ### ACE++ Portrait
63
+ Portrait-consistent generation to maintain the consistency of the portrait.
64
+
65
+ <table><tbody>
66
+ <tr>
67
+ <td>Tuning Method</td>
68
+ <td>Input</td>
69
+ <td>Output</td>
70
+ <td>Instruction</td>
71
+ <td>Models</td>
72
+ </tr>
73
+ <tr>
74
+ <td>LoRA <br>+ ACE Data</td>
75
+ <td><img src="./assets/samples/portrait/human_1.jpg" width="200"></td>
76
+ <td><img src="./assets/samples/portrait/human_1_1.jpg" width="200"></td>
77
+ <td style="word-wrap:break-word;word-break:break-all;" width="250px";>"Maintain the facial features. A girl is wearing a neat police uniform and sporting a badge. She is smiling with a friendly and confident demeanor. The background is blurred, featuring a cartoon logo."</td>
78
+ <td align="center" style="word-wrap:break-word;word-break:break-all;" width="200px";><a href="https://www.modelscope.cn/models/iic/ACE_Plus/"><img src="https://img.shields.io/badge/ModelScope-Model-blue" alt="ModelScope link"> </a> <a href="https://huggingface.co/ali-vilab/ACE_Plus/tree/main/portrait/"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-yellow" alt="HuggingFace link"> </a> </td>
79
+ </tr>
80
+ </tbody>
81
+ </table>
82
+
83
+ Models' scepter_path:
84
+ - **ModelScope:** ms://iic/ACE_Plus@portrait/xxxx.safetensors
85
+ - **HuggingFace:** hf://ali-vilab/ACE_Plus@portrait/xxxx.safetensors
86
+
87
+
88
+ ### ACE++ Subject
89
+ Subject-driven image generation task to maintain the consistency of a specific subject in different scenes.
90
+ <table><tbody>
91
+ <tr>
92
+ <td>Tuning Method</td>
93
+ <td>Input</td>
94
+ <td>Output</td>
95
+ <td>Instruction</td>
96
+ <td>Models</td>
97
+ </tr>
98
+ <tr>
99
+ <td>LoRA <br>+ ACE Data</td>
100
+ <td><img src="./assets/samples/subject/subject_1.jpg" width="200"></td>
101
+ <td><img src="./assets/samples/subject/subject_1_1.jpg" width="200"></td>
102
+ <td style="word-wrap:break-word;word-break:break-all;" width="250px";>"Display the logo in a minimalist style printed in white on a matte black ceramic coffee mug, alongside a steaming cup of coffee on a cozy cafe table."</td>
103
+ <td align="center" style="word-wrap:break-word;word-break:break-all;" width="200px";><a href="https://www.modelscope.cn/models/iic/ACE_Plus/"><img src="https://img.shields.io/badge/ModelScope-Model-blue" alt="ModelScope link"> </a> <a href="https://huggingface.co/ali-vilab/ACE_Plus/tree/main/subject/"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-yellow" alt="HuggingFace link"> </a> </td>
104
+ </tr>
105
+ </tbody>
106
+ </table>
107
+
108
+ Models' scepter_path:
109
+ - **ModelScope:** ms://iic/ACE_Plus@subject/xxxx.safetensors
110
+ - **HuggingFace:** hf://ali-vilab/ACE_Plus@subject/xxxx.safetensors
111
+
112
+
113
+ ### ACE++ LocalEditing
114
+ Redrawing the mask area of images while maintaining the original structural information of the edited area.
115
+ <table><tbody>
116
+ <tr>
117
+ <td>Tuning Method</td>
118
+ <td>Input</td>
119
+ <td>Output</td>
120
+ <td>Instruction</td>
121
+ <td>Models</td>
122
+ </tr>
123
+ <tr>
124
+ <td>LoRA <br>+ ACE Data</td>
125
+ <td><img src="./assets/samples/local/local_1.webp" width="200"><br><img src="./assets/samples/local/local_1_m.webp" width="200"></td>
126
+ <td><img src="./assets/samples/local/local_1_1.jpg" width="200"></td>
127
+ <td style="word-wrap:break-word;word-break:break-all;" width="250px";>"By referencing the mask, restore a partial image from the doodle {image} that aligns with the textual explanation: "1 white old owl"."</td>
128
+ <td align="center" style="word-wrap:break-word;word-break:break-all;" width="200px";><a href="https://www.modelscope.cn/models/iic/ACE_Plus/"><img src="https://img.shields.io/badge/ModelScope-Model-blue" alt="ModelScope link"> </a> <a href="https://huggingface.co/ali-vilab/ACE_Plus/tree/main/local_editing/"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-yellow" alt="HuggingFace link"> </a> </td>
129
+ </tr>
130
+ </tbody>
131
+ </table>
132
+
133
+ Models' scepter_path:
134
+ - **ModelScope:** ms://iic/ACE_Plus@local_editing/xxxx.safetensors
135
+ - **HuggingFace:** hf://ali-vilab/ACE_Plus@local_editing/xxxx.safetensors
136
+
137
+ ### ACE++ Fully [Coming soon]
138
+ Fully finetuning a composite model with ACE’s data to support various editing and reference generation tasks through an instructive approach.
139
 
140
  ## 🔥 Applications
141
  The ACE++ model supports a wide range of downstream tasks through simple adaptations. Here are some examples, and we look forward to seeing the community explore even more exciting applications utilizing the ACE++ model.
142
 
143
+ <table><tbody>
144
+ <tr>
145
+ <th align="center" colspan="1">Application</th>
146
+ <th align="center" colspan="1">ACE++ Model</th>
147
+ <th align="center" colspan="5">Examples</th>
148
+ </tr>
149
+ <tr>
150
+ <td>Try On</td>
151
+ <td>ACE++ Subject</td>
152
+ <td><img src="./assets/samples/application/try_on/1_ref.png" width="200"></td>
153
+ <td><img src="./assets/samples/application/try_on/1_1_edit.png" width="200"></td>
154
+ <td><img src="./assets/samples/application/try_on/1_1_m.png" width="200"></td>
155
+ <td><img src="./assets/samples/application/try_on/1_1_res.png" width="200"></td>
156
+ <td style="word-wrap:break-word;word-break:break-all;" width="100px";>"The woman dresses this skirt."</td>
157
+ </tr>
158
+ <tr>
159
+ <td>Logo Paste</td>
160
+ <td>ACE++ Subject</td>
161
+ <td><img src="./assets/samples/application/logo_paste/1_ref.png" width="200"></td>
162
+ <td><img src="./assets/samples/application/logo_paste/1_1_edit.png" width="200"></td>
163
+ <td><img src="./assets/samples/application/logo_paste/1_1_m.png" width="200"></td>
164
+ <td><img src="./assets/samples/application/logo_paste/1_1_res.webp" width="200"></td>
165
+ <td style="word-wrap:break-word;word-break:break-all;" width="100px";>"The logo is printed on the headphones."</td>
166
+ </tr>
167
+ <tr>
168
+ <td>Photo Editing</td>
169
+ <td>ACE++ Subject</td>
170
+ <td><img src="./assets/samples/application/photo_editing/1_ref.png" width="200"></td>
171
+ <td><img src="./assets/samples/application/photo_editing/1_1_edit.png" width="200"></td>
172
+ <td><img src="./assets/samples/application/photo_editing/1_1_m.png" width="200"></td>
173
+ <td><img src="./assets/samples/application/photo_editing/1_1_res.jpg" width="200"></td>
174
+ <td style="word-wrap:break-word;word-break:break-all;" width="100px";>"The item is put on the ground."</td>
175
+ </tr>
176
+ <tr>
177
+ <td>Movie Poster Editor</td>
178
+ <td>ACE++ Portrait</td>
179
+ <td><img src="./assets/samples/application/movie_poster/1_ref.png" width="200"></td>
180
+ <td><img src="./assets/samples/application/movie_poster/1_1_edit.png" width="200"></td>
181
+ <td><img src="./assets/samples/application/movie_poster/1_1_m.png" width="200"></td>
182
+ <td><img src="./assets/samples/application/movie_poster/1_1_res.webp" width="200"></td>
183
+ <td style="word-wrap:break-word;word-break:break-all;" width="100px";>"The man is facing the camera and is smiling."</td>
184
+ </tr>
185
+ </tbody>
186
+ </table>
187
 
188
  ## ⚙️️ Installation
189
  Download the code using the following command: