Ashoka74 commited on
Commit
7087aed
Β·
verified Β·
1 Parent(s): 0c5f5a6

Upload folder using huggingface_hub

Browse files
Files changed (3) hide show
  1. Dockerfile +3 -3
  2. README.md +6 -278
  3. requirements.txt +0 -0
Dockerfile CHANGED
@@ -22,7 +22,7 @@ WORKDIR /app
22
  COPY requirements.txt .
23
 
24
  # Install Python dependencies
25
- RUN pip3 install --no-cache-dir torch==2.4.1+cu121 torchvision==0.19.1+cu121 --index-url https://download.pytorch.org/whl/cu121
26
  RUN pip3 install --no-cache-dir -r requirements.txt
27
 
28
  # Create models directory
@@ -34,10 +34,10 @@ COPY . .
34
  # Download model weights if needed
35
  RUN mkdir -p /app/models && \
36
  if [ ! -f /app/models/iclight_sd15_fc.safetensors ]; then \
37
- wget -P /app/models https://huggingface.co/lllyasviel/ic-light/resolve/main/iclight_sd15_fc.safetensors; \
38
  fi && \
39
  if [ ! -f /app/models/iclight_sd15_fbc.safetensors ]; then \
40
- wget -P /app/models https://huggingface.co/lllyasviel/ic-light/resolve/main/iclight_sd15_fbc.safetensors; \
41
  fi
42
 
43
  # Expose port for Gradio
 
22
  COPY requirements.txt .
23
 
24
  # Install Python dependencies
25
+ RUN pip3 install --no-cache-dir torch==2.4.0+cu121 torchvision==0.19.1+cu121 --index-url https://download.pytorch.org/whl/cu121
26
  RUN pip3 install --no-cache-dir -r requirements.txt
27
 
28
  # Create models directory
 
34
  # Download model weights if needed
35
  RUN mkdir -p /app/models && \
36
  if [ ! -f /app/models/iclight_sd15_fc.safetensors ]; then \
37
+ wget -P /app/models https://huggingface.co/lllyasviel/ic-light/resolve/main/iclight_sd15_fc.safetensors; \
38
  fi && \
39
  if [ ! -f /app/models/iclight_sd15_fbc.safetensors ]; then \
40
+ wget -P /app/models https://huggingface.co/lllyasviel/ic-light/resolve/main/iclight_sd15_fbc.safetensors; \
41
  fi
42
 
43
  # Expose port for Gradio
README.md CHANGED
@@ -1,278 +1,6 @@
1
- ---
2
- title: ProductPlacement
3
- app_file: gradio_demo.py
4
- sdk: gradio
5
- sdk_version: 4.37.2
6
- ---
7
- # IC-Light
8
-
9
- IC-Light is a project to manipulate the illumination of images.
10
-
11
- The name "IC-Light" stands for **"Imposing Consistent Light"** (we will briefly describe this at the end of this page).
12
-
13
- Currently, we release two types of models: text-conditioned relighting model and background-conditioned model. Both types take foreground images as inputs.
14
-
15
- **Note that "iclightai dot com" is a scam website. They have no relationship with us. Do not give scam websites money! This GitHub repo is the only official IC-Light.**
16
-
17
- # News
18
-
19
- [Alternative model](https://github.com/lllyasviel/IC-Light/discussions/109) for stronger illumination modifications.
20
-
21
- Some news about flux is [here](https://github.com/lllyasviel/IC-Light/discussions/98). (A fix [update](https://github.com/lllyasviel/IC-Light/discussions/98#discussioncomment-11370266) is added at Nov 25, more demos will be uploaded soon.)
22
-
23
- # Get Started
24
-
25
- Below script will run the text-conditioned relighting model:
26
-
27
- git clone https://github.com/lllyasviel/IC-Light.git
28
- cd IC-Light
29
- conda create -n iclight python=3.10
30
- conda activate iclight
31
- pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118
32
- pip install -r requirements.txt
33
- pip install xformers
34
- python gradio_demo.py
35
-
36
- Or, to use background-conditioned demo:
37
-
38
- python gradio_demo_bg.py
39
-
40
- Model downloading is automatic.
41
-
42
- Note that the "gradio_demo.py" has an official [huggingFace Space here](https://huggingface.co/spaces/lllyasviel/IC-Light).
43
-
44
- # Screenshot
45
-
46
- ### Text-Conditioned Model
47
-
48
- (Note that the "Lighting Preference" are just initial latents - eg., if the Lighting Preference is "Left" then initial latent is left white right black.)
49
-
50
- ---
51
-
52
- **Prompt: beautiful woman, detailed face, warm atmosphere, at home, bedroom**
53
-
54
- Lighting Preference: Left
55
-
56
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/87265483-aa26-4d2e-897d-b58892f5fdd7)
57
-
58
- ---
59
-
60
- **Prompt: beautiful woman, detailed face, sunshine from window**
61
-
62
- Lighting Preference: Left
63
-
64
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/148c4a6d-82e7-4e3a-bf44-5c9a24538afc)
65
-
66
- ---
67
-
68
- **beautiful woman, detailed face, neon, Wong Kar-wai, warm**
69
-
70
- Lighting Preference: Left
71
-
72
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/f53c9de2-534a-42f4-8272-6d16a021fc01)
73
-
74
- ---
75
-
76
- **Prompt: beautiful woman, detailed face, sunshine, outdoor, warm atmosphere**
77
-
78
- Lighting Preference: Right
79
-
80
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/25d6ea24-a736-4a0b-b42d-700fe8b2101e)
81
-
82
- ---
83
-
84
- **Prompt: beautiful woman, detailed face, sunshine, outdoor, warm atmosphere**
85
-
86
- Lighting Preference: Left
87
-
88
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/dd30387b-0490-46ee-b688-2191fb752e68)
89
-
90
- ---
91
-
92
- **Prompt: beautiful woman, detailed face, sunshine from window**
93
-
94
- Lighting Preference: Right
95
-
96
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/6c9511ca-f97f-401a-85f3-92b4442000e3)
97
-
98
- ---
99
-
100
- **Prompt: beautiful woman, detailed face, shadow from window**
101
-
102
- Lighting Preference: Left
103
-
104
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/e73701d5-890e-4b15-91ee-97f16ea3c450)
105
-
106
- ---
107
-
108
- **Prompt: beautiful woman, detailed face, sunset over sea**
109
-
110
- Lighting Preference: Right
111
-
112
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/ff26ac3d-1b12-4447-b51f-73f7a5122a05)
113
-
114
- ---
115
-
116
- **Prompt: handsome boy, detailed face, neon light, city**
117
-
118
- Lighting Preference: Left
119
-
120
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/d7795e02-46f7-444f-93e7-4d6460840437)
121
-
122
- ---
123
-
124
- **Prompt: beautiful woman, detailed face, light and shadow**
125
-
126
- Lighting Preference: Left
127
-
128
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/706f70a8-d1a0-4e0b-b3ac-804e8e231c0f)
129
-
130
- (beautiful woman, detailed face, soft studio lighting)
131
-
132
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/fe0a72df-69d4-4e11-b661-fb8b84d0274d)
133
-
134
- ---
135
-
136
- **Prompt: Buddha, detailed face, sci-fi RGB glowing, cyberpunk**
137
-
138
- Lighting Preference: Left
139
-
140
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/68d60c68-ce23-4902-939e-11629ccaf39a)
141
-
142
- ---
143
-
144
- **Prompt: Buddha, detailed face, natural lighting**
145
-
146
- Lighting Preference: Left
147
-
148
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/1841d23d-0a0d-420b-a5ab-302da9c47c17)
149
-
150
- ---
151
-
152
- **Prompt: toy, detailed face, shadow from window**
153
-
154
- Lighting Preference: Bottom
155
-
156
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/dcb97439-ea6b-483e-8e68-cf5d320368c7)
157
-
158
- ---
159
-
160
- **Prompt: toy, detailed face, sunset over sea**
161
-
162
- Lighting Preference: Right
163
-
164
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/4f78b897-621d-4527-afa7-78d62c576100)
165
-
166
- ---
167
-
168
- **Prompt: dog, magic lit, sci-fi RGB glowing, studio lighting**
169
-
170
- Lighting Preference: Bottom
171
-
172
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/1db9cac9-8d3f-4f40-82e2-e3b0cafd8613)
173
-
174
- ---
175
-
176
- **Prompt: mysteriou human, warm atmosphere, warm atmosphere, at home, bedroom**
177
-
178
- Lighting Preference: Right
179
-
180
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/5d5aa7e5-8cbd-4e1f-9f27-2ecc3c30563a)
181
-
182
- ---
183
-
184
- ### Background-Conditioned Model
185
-
186
- The background conditioned model does not require careful prompting. One can just use simple prompts like "handsome man, cinematic lighting".
187
-
188
- ---
189
-
190
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/0b2a889f-682b-4393-b1ec-2cabaa182010)
191
-
192
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/477ca348-bd47-46ff-81e6-0ffc3d05feb2)
193
-
194
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/5bc9d8d9-02cd-442e-a75c-193f115f2ad8)
195
-
196
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/a35e4c57-e199-40e2-893b-cb1c549612a9)
197
-
198
- ---
199
-
200
- A more structured visualization:
201
-
202
- ![r1](https://github.com/lllyasviel/IC-Light/assets/19834515/c1daafb5-ac8b-461c-bff2-899e4c671ba3)
203
-
204
- # Imposing Consistent Light
205
-
206
- In HDR space, illumination has a property that all light transports are independent.
207
-
208
- As a result, the blending of appearances of different light sources is equivalent to the appearance with mixed light sources:
209
-
210
- ![cons](https://github.com/lllyasviel/IC-Light/assets/19834515/27c67787-998e-469f-862f-047344e100cd)
211
-
212
- Using the above [light stage](https://www.pauldebevec.com/Research/LS/) as an example, the two images from the "appearance mixture" and "light source mixture" are consistent (mathematically equivalent in HDR space, ideally).
213
-
214
- We imposed such consistency (using MLPs in latent space) when training the relighting models.
215
-
216
- As a result, the model is able to produce highly consistent relight - **so** consistent that different relightings can even be merged as normal maps! Despite the fact that the models are latent diffusion.
217
-
218
- ![r2](https://github.com/lllyasviel/IC-Light/assets/19834515/25068f6a-f945-4929-a3d6-e8a152472223)
219
-
220
- From left to right are inputs, model outputs relighting, devided shadow image, and merged normal maps. Note that the model is not trained with any normal map data. This normal estimation comes from the consistency of relighting.
221
-
222
- You can reproduce this experiment using this button (it is 4x slower because it relight image 4 times)
223
-
224
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/d9c37bf7-2136-446c-a9a5-5a341e4906de)
225
-
226
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/fcf5dd55-0309-4e8e-9721-d55931ea77f0)
227
-
228
- Below are bigger images (feel free to try yourself to get more results!)
229
-
230
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/12335218-186b-4c61-b43a-79aea9df8b21)
231
-
232
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/2daab276-fdfa-4b0c-abcb-e591f575598a)
233
-
234
- For reference, [geowizard](https://fuxiao0719.github.io/projects/geowizard/) (geowizard is a really great work!):
235
-
236
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/4ba1a96d-e218-42ab-83ae-a7918d56ee5f)
237
-
238
- And, [switchlight](https://arxiv.org/pdf/2402.18848) (switchlight is another great work!):
239
-
240
- ![image](https://github.com/lllyasviel/IC-Light/assets/19834515/fbdd961f-0b26-45d2-802e-ffd734affab8)
241
-
242
- # Model Notes
243
-
244
- * **iclight_sd15_fc.safetensors** - The default relighting model, conditioned on text and foreground. You can use initial latent to influence the relighting.
245
-
246
- * **iclight_sd15_fcon.safetensors** - Same as "iclight_sd15_fc.safetensors" but trained with offset noise. Note that the default "iclight_sd15_fc.safetensors" outperform this model slightly in a user study. And this is the reason why the default model is the model without offset noise.
247
-
248
- * **iclight_sd15_fbc.safetensors** - Relighting model conditioned with text, foreground, and background.
249
-
250
- Also, note that the original [BRIA RMBG 1.4](https://huggingface.co/briaai/RMBG-1.4) is for non-commercial use. If you use IC-Light in commercial projects, replace it with other background replacer like [BiRefNet](https://github.com/ZhengPeng7/BiRefNet).
251
-
252
- # Cite
253
-
254
- @Misc{iclight,
255
- author = {Lvmin Zhang and Anyi Rao and Maneesh Agrawala},
256
- title = {IC-Light GitHub Page},
257
- year = {2024},
258
- }
259
-
260
- # Related Work
261
-
262
- Also read ...
263
-
264
- [Total Relighting: Learning to Relight Portraits for Background Replacement](https://augmentedperception.github.io/total_relighting/)
265
-
266
- [Relightful Harmonization: Lighting-aware Portrait Background Replacement](https://arxiv.org/abs/2312.06886)
267
-
268
- [SwitchLight: Co-design of Physics-driven Architecture and Pre-training Framework for Human Portrait Relighting](https://arxiv.org/pdf/2402.18848)
269
-
270
- ## GPU Optimization
271
-
272
- For RTX 2070 and similar GPUs:
273
-
274
- 1. Install optimized dependencies:
275
-
276
- ```bash
277
- pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118
278
- ```
 
1
+ ---
2
+ title: ProductPlacement
3
+ app_file: gradio_demo.py
4
+ sdk: gradio
5
+ sdk_version: 5.7.1
6
+ ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
requirements.txt CHANGED
Binary files a/requirements.txt and b/requirements.txt differ