fantos commited on
Commit
c07cab5
ยท
verified ยท
1 Parent(s): a524c7e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +107 -2
README.md CHANGED
@@ -4,10 +4,115 @@ emoji: ๐Ÿ–ผ
4
  colorFrom: purple
5
  colorTo: red
6
  sdk: gradio
7
- sdk_version: 5.29.0
8
  app_file: app.py
9
  pinned: false
10
  license: other
11
  ---
 
12
 
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  colorFrom: purple
5
  colorTo: red
6
  sdk: gradio
7
+ sdk_version: 5.35.0
8
  app_file: app.py
9
  pinned: false
10
  license: other
11
  ---
12
+ ## FLUX.1-dev ControlNet Union Pro: Advanced Image Generation with Multiple Control Modes
13
 
14
+ This application implements a sophisticated image generation system using FLUX.1-dev with ControlNet Union Pro, offering multiple control modes for precise image generation guidance. The system allows users to generate high-quality images while maintaining specific structural or stylistic constraints from reference images.
15
+
16
+ ### Key Features
17
+
18
+ **1. Multiple Control Modes**
19
+ - **Canny**: Edge-based control using Canny edge detection
20
+ - **Depth**: 3D depth information guidance using Depth Anything V2
21
+ - **OpenPose**: Human pose-based generation
22
+ - **Grayscale**: Luminance-based control
23
+ - **Blur**: Gaussian blur for soft guidance
24
+ - **Tile**: Resolution-independent tiling control
25
+ - **LowQuality**: Noise-based control for enhancement tasks
26
+
27
+ **2. Flexible Input Options**
28
+ - Direct upload of pre-processed control images
29
+ - Automatic extraction of control conditions from reference images
30
+ - Support for various image formats and resolutions
31
+ - Intelligent image resizing and preprocessing
32
+
33
+ **3. Advanced Generation Parameters**
34
+ - **Control Strength (0-1.0)**: Adjust how strongly the control influences generation
35
+ - **Inference Steps (1-50)**: Balance between quality and speed
36
+ - **Guidance Scale (1-10)**: Control prompt adherence
37
+ - **Seed Control**: Reproducible results with manual or random seeds
38
+
39
+ **4. Technical Architecture**
40
+ - Based on FLUX.1-dev diffusion model
41
+ - Multi-ControlNet support for combined control modes
42
+ - Depth Anything V2 (Large) for accurate depth estimation
43
+ - GPU-accelerated processing with CUDA support
44
+ - Memory-optimized with VAE tiling and CPU offloading
45
+
46
+ ### How It Works
47
+
48
+ 1. **Control Image Input**: Either upload a pre-processed control image or let the system extract it from a reference image
49
+ 2. **Control Mode Selection**: Choose the appropriate control type for your use case
50
+ 3. **Prompt Input**: Describe the desired output (defaults to "Highest Quality")
51
+ 4. **Parameter Tuning**: Adjust control strength and generation settings
52
+ 5. **Generation**: The model creates an image following both the prompt and control guidance
53
+
54
+ ### Use Cases
55
+
56
+ - **Image Enhancement**: Use LowQuality mode to enhance degraded images
57
+ - **Style Transfer**: Apply artistic styles while preserving structure (Canny/Depth)
58
+ - **Pose-Guided Generation**: Create images with specific human poses
59
+ - **Consistent Character Design**: Maintain structural consistency across variations
60
+ - **Architectural Visualization**: Use depth control for accurate spatial representations
61
+ - **Texture Synthesis**: Tile mode for seamless pattern generation
62
+
63
+ The system provides real-time feedback by showing both the generated result and the preprocessed control condition, helping users understand and refine their control inputs for optimal results.
64
+
65
+ ---
66
+
67
+ ## FLUX.1-dev ControlNet Union Pro: ๋‹ค์ค‘ ์ œ์–ด ๋ชจ๋“œ๋ฅผ ํ™œ์šฉํ•œ ๊ณ ๊ธ‰ ์ด๋ฏธ์ง€ ์ƒ์„ฑ
68
+
69
+ ์ด ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜์€ FLUX.1-dev์™€ ControlNet Union Pro๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์ •๊ตํ•œ ์ด๋ฏธ์ง€ ์ƒ์„ฑ ์‹œ์Šคํ…œ์„ ๊ตฌํ˜„ํ•˜๋ฉฐ, ์ •๋ฐ€ํ•œ ์ด๋ฏธ์ง€ ์ƒ์„ฑ ๊ฐ€์ด๋“œ๋ฅผ ์œ„ํ•œ ๋‹ค์–‘ํ•œ ์ œ์–ด ๋ชจ๋“œ๋ฅผ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค. ์‚ฌ์šฉ์ž๋Š” ์ฐธ์กฐ ์ด๋ฏธ์ง€์˜ ํŠน์ • ๊ตฌ์กฐ๋‚˜ ์Šคํƒ€์ผ ์ œ์•ฝ์„ ์œ ์ง€ํ•˜๋ฉด์„œ ๊ณ ํ’ˆ์งˆ ์ด๋ฏธ์ง€๋ฅผ ์ƒ์„ฑํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
70
+
71
+ ### ์ฃผ์š” ๊ธฐ๋Šฅ
72
+
73
+ **1. ๋‹ค์ค‘ ์ œ์–ด ๋ชจ๋“œ**
74
+ - **Canny**: Canny ์—ฃ์ง€ ๊ฒ€์ถœ์„ ์‚ฌ์šฉํ•œ ์—ฃ์ง€ ๊ธฐ๋ฐ˜ ์ œ์–ด
75
+ - **Depth**: Depth Anything V2๋ฅผ ์‚ฌ์šฉํ•œ 3D ๊นŠ์ด ์ •๋ณด ๊ฐ€์ด๋“œ
76
+ - **OpenPose**: ์ธ์ฒด ํฌ์ฆˆ ๊ธฐ๋ฐ˜ ์ƒ์„ฑ
77
+ - **Grayscale**: ๋ช…๋„ ๊ธฐ๋ฐ˜ ์ œ์–ด
78
+ - **Blur**: ๋ถ€๋“œ๋Ÿฌ์šด ๊ฐ€์ด๋“œ๋ฅผ ์œ„ํ•œ ๊ฐ€์šฐ์‹œ์•ˆ ๋ธ”๋Ÿฌ
79
+ - **Tile**: ํ•ด์ƒ๋„ ๋…๋ฆฝ์ ์ธ ํƒ€์ผ๋ง ์ œ์–ด
80
+ - **LowQuality**: ํ–ฅ์ƒ ์ž‘์—…์„ ์œ„ํ•œ ๋…ธ์ด์ฆˆ ๊ธฐ๋ฐ˜ ์ œ์–ด
81
+
82
+ **2. ์œ ์—ฐํ•œ ์ž…๋ ฅ ์˜ต์…˜**
83
+ - ์‚ฌ์ „ ์ฒ˜๋ฆฌ๋œ ์ œ์–ด ์ด๋ฏธ์ง€ ์ง์ ‘ ์—…๋กœ๋“œ
84
+ - ์ฐธ์กฐ ์ด๋ฏธ์ง€์—์„œ ์ œ์–ด ์กฐ๊ฑด ์ž๋™ ์ถ”์ถœ
85
+ - ๋‹ค์–‘ํ•œ ์ด๋ฏธ์ง€ ํ˜•์‹ ๋ฐ ํ•ด์ƒ๋„ ์ง€์›
86
+ - ์ง€๋Šฅ์ ์ธ ์ด๋ฏธ์ง€ ํฌ๊ธฐ ์กฐ์ • ๋ฐ ์ „์ฒ˜๋ฆฌ
87
+
88
+ **3. ๊ณ ๊ธ‰ ์ƒ์„ฑ ๋งค๊ฐœ๋ณ€์ˆ˜**
89
+ - **Control Strength (0-1.0)**: ์ œ์–ด๊ฐ€ ์ƒ์„ฑ์— ๋ฏธ์น˜๋Š” ์˜ํ–ฅ ์กฐ์ ˆ
90
+ - **Inference Steps (1-50)**: ํ’ˆ์งˆ๊ณผ ์†๋„ ๊ฐ„ ๊ท ํ˜• ์กฐ์ ˆ
91
+ - **Guidance Scale (1-10)**: ํ”„๋กฌํ”„ํŠธ ์ค€์ˆ˜๋„ ์ œ์–ด
92
+ - **Seed Control**: ์ˆ˜๋™ ๋˜๋Š” ๋žœ๋ค ์‹œ๋“œ๋กœ ์žฌํ˜„ ๊ฐ€๋Šฅํ•œ ๊ฒฐ๊ณผ
93
+
94
+ **4. ๊ธฐ์ˆ ์  ๊ตฌ์กฐ**
95
+ - FLUX.1-dev ํ™•์‚ฐ ๋ชจ๋ธ ๊ธฐ๋ฐ˜
96
+ - ๊ฒฐํ•ฉ๋œ ์ œ์–ด ๋ชจ๋“œ๋ฅผ ์œ„ํ•œ Multi-ControlNet ์ง€์›
97
+ - ์ •ํ™•ํ•œ ๊นŠ์ด ์ถ”์ •์„ ์œ„ํ•œ Depth Anything V2 (Large)
98
+ - CUDA ์ง€์› GPU ๊ฐ€์† ์ฒ˜๋ฆฌ
99
+ - VAE ํƒ€์ผ๋ง๊ณผ CPU ์˜คํ”„๋กœ๋”ฉ์œผ๋กœ ๋ฉ”๋ชจ๋ฆฌ ์ตœ์ ํ™”
100
+
101
+ ### ์ž‘๋™ ๋ฐฉ์‹
102
+
103
+ 1. **์ œ์–ด ์ด๋ฏธ์ง€ ์ž…๋ ฅ**: ์‚ฌ์ „ ์ฒ˜๋ฆฌ๋œ ์ œ์–ด ์ด๋ฏธ์ง€ ์—…๋กœ๋“œ ๋˜๋Š” ์ฐธ์กฐ ์ด๋ฏธ์ง€์—์„œ ์ž๋™ ์ถ”์ถœ
104
+ 2. **์ œ์–ด ๋ชจ๋“œ ์„ ํƒ**: ์‚ฌ์šฉ ๋ชฉ์ ์— ๋งž๋Š” ๏ฟฝ๏ฟฝ์ ˆํ•œ ์ œ์–ด ์œ ํ˜• ์„ ํƒ
105
+ 3. **ํ”„๋กฌํ”„ํŠธ ์ž…๋ ฅ**: ์›ํ•˜๋Š” ์ถœ๋ ฅ ์„ค๋ช… (๊ธฐ๋ณธ๊ฐ’: "Highest Quality")
106
+ 4. **๋งค๊ฐœ๋ณ€์ˆ˜ ์กฐ์ •**: ์ œ์–ด ๊ฐ•๋„ ๋ฐ ์ƒ์„ฑ ์„ค์ • ์กฐ์ ˆ
107
+ 5. **์ƒ์„ฑ**: ๋ชจ๋ธ์ด ํ”„๋กฌํ”„ํŠธ์™€ ์ œ์–ด ๊ฐ€์ด๋“œ๋ฅผ ๋ชจ๋‘ ๋”ฐ๋ฅด๋Š” ์ด๋ฏธ์ง€ ์ƒ์„ฑ
108
+
109
+ ### ํ™œ์šฉ ์‚ฌ๋ก€
110
+
111
+ - **์ด๋ฏธ์ง€ ํ–ฅ์ƒ**: LowQuality ๋ชจ๋“œ๋กœ ์—ดํ™”๋œ ์ด๋ฏธ์ง€ ๊ฐœ์„ 
112
+ - **์Šคํƒ€์ผ ์ „์†ก**: ๊ตฌ์กฐ๋ฅผ ๋ณด์กดํ•˜๋ฉด์„œ ์˜ˆ์ˆ ์  ์Šคํƒ€์ผ ์ ์šฉ (Canny/Depth)
113
+ - **ํฌ์ฆˆ ๊ธฐ๋ฐ˜ ์ƒ์„ฑ**: ํŠน์ • ์ธ์ฒด ํฌ์ฆˆ๋กœ ์ด๋ฏธ์ง€ ์ƒ์„ฑ
114
+ - **์ผ๊ด€๋œ ์บ๋ฆญํ„ฐ ๋””์ž์ธ**: ๋ณ€ํ˜• ๊ฐ„ ๊ตฌ์กฐ์  ์ผ๊ด€์„ฑ ์œ ์ง€
115
+ - **๊ฑด์ถ• ์‹œ๊ฐํ™”**: ์ •ํ™•ํ•œ ๊ณต๊ฐ„ ํ‘œํ˜„์„ ์œ„ํ•œ ๊นŠ์ด ์ œ์–ด ์‚ฌ์šฉ
116
+ - **ํ…์Šค์ฒ˜ ํ•ฉ์„ฑ**: ๋งค๋„๋Ÿฌ์šด ํŒจํ„ด ์ƒ์„ฑ์„ ์œ„ํ•œ ํƒ€์ผ ๋ชจ๋“œ
117
+
118
+ ์ด ์‹œ์Šคํ…œ์€ ์ƒ์„ฑ๋œ ๊ฒฐ๊ณผ์™€ ์ „์ฒ˜๋ฆฌ๋œ ์ œ์–ด ์กฐ๊ฑด์„ ๋ชจ๋‘ ๋ณด์—ฌ์คŒ์œผ๋กœ์จ ์‹ค์‹œ๊ฐ„ ํ”ผ๋“œ๋ฐฑ์„ ์ œ๊ณตํ•˜๋ฉฐ, ์‚ฌ์šฉ์ž๊ฐ€ ์ตœ์ ์˜ ๊ฒฐ๊ณผ๋ฅผ ์œ„ํ•ด ์ œ์–ด ์ž…๋ ฅ์„ ์ดํ•ดํ•˜๊ณ  ๊ฐœ์„ ํ•˜๋Š” ๋ฐ ๋„์›€์„ ์ค๋‹ˆ๋‹ค.