README.md

---
license: other
license_name: flux-1-dev-non-commercial-license
license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.
language:
- en
pipeline_tag: text-to-image
tags:
- Stable Diffusion
- image-generation
- Flux
- diffusers
---

![Controlnet collections for Flux](https://github.com/XLabs-AI/x-flux/blob/main/assets/readme/light/flux-controlnet-collections.png?raw=true)
[<img src="https://github.com/XLabs-AI/x-flux/blob/main/assets/readme/light/join-our-discord-rev1.png?raw=true">](https://discord.gg/FHY2guThfy)

This repository provides a collection of ControlNet checkpoints for
[FLUX.1-dev model](https://huggingface.co/black-forest-labs/FLUX.1-dev) by Black Forest Labs

![Example Picture 1](./assets/depth_v2_res1.png?raw=true)

[See our github](https://github.com/XLabs-AI/x-flux-comfyui) for comfy ui workflows.
![Example Picture 1](https://github.com/XLabs-AI/x-flux-comfyui/blob/main/assets/image1.png?raw=true)

[See our github](https://github.com/XLabs-AI/x-flux) for train script, train configs and demo script for inference.

# Models

Our collection supports 3 models:
- Canny
- HED
- Depth (Midas)

Each ControlNet is trained on 1024x1024 resolution and works for 1024x1024 resolution.
We release **v2 versions** - better and realistic versions, which can be used directly in ComfyUI!   

Please, see our [ComfyUI custom nodes installation guide](https://github.com/XLabs-AI/x-flux-comfyui)


# Examples

See examples of our models results below.  
Also, some generation results with input images are provided in "Files and versions"

# Inference

To try our models, you have 2 options:
1. Use main.py from our [official repo](https://github.com/XLabs-AI/x-flux)
2. Use our custom nodes for ComfyUI and test it with provided workflows (check out folder /workflows)

See examples how to launch our models:

## Canny ControlNet (version 2)

1. Clone our [x-flux-comfyui](https://github.com/XLabs-AI/x-flux-comfyui) custom nodes
2. Launch ComfyUI
3. Try our canny_workflow.json

![Example Picture 1](./assets/canny_v2_res1.png?raw=true)
![Example Picture 1](./assets/canny_v2_res2.png?raw=true)
![Example Picture 1](./assets/canny_v2_res3.png?raw=true)


## Canny ControlNet (version 1)

1. Clone [our repo](https://github.com/XLabs-AI/x-flux), install requirements
2. Launch main.py in command line with parameters

```bash
python3 main.py \
 --prompt "a viking man with white hair looking, cinematic, MM full HD" \
 --image input_image_canny.jpg \
 --control_type canny \
 --repo_id XLabs-AI/flux-controlnet-collections --name flux-canny-controlnet.safetensors --device cuda --use_controlnet \
 --model_type flux-dev --width 768 --height 768 \
 --timestep_to_start_cfg 1 --num_steps 25 --true_gs 3.5 --guidance 4

```
![Example Picture 1](https://github.com/XLabs-AI/x-flux/blob/main/assets/readme/examples/canny_example_1.png?raw=true)

## Depth ControlNet (version 2)

1. Clone our [x-flux-comfyui](https://github.com/XLabs-AI/x-flux-comfyui) custom nodes
2. Launch ComfyUI
3. Try our depth_workflow.json

![Example Picture 1](./assets/depth_v2_res1.png?raw=true)
![Example Picture 1](./assets/depth_v2_res2.png?raw=true)

## Depth ControlNet (version 1)
1. Clone [our repo](https://github.com/XLabs-AI/x-flux), install requirements
2. Launch main.py in command line with parameters
```bash
python3 main.py \
 --prompt "Photo of the bold man with beard and laptop, full hd, cinematic photo" \
 --image input_image_depth1.jpg \
 --control_type depth \
 --repo_id XLabs-AI/flux-controlnet-collections --name flux-depth-controlnet.safetensors --device cuda --use_controlnet \
 --model_type flux-dev --width 1024 --height 1024 \
 --timestep_to_start_cfg 1 --num_steps 25 --true_gs 3.5 --guidance 4

```
![Example Picture 2](https://github.com/XLabs-AI/x-flux/blob/main/assets/readme/examples/depth_example_1.png?raw=true)

```bash
python3 main.py \
 --prompt "photo of handsome fluffy black dog standing on a forest path, full hd, cinematic photo" \
 --image input_image_depth2.jpg \
 --control_type depth \
 --repo_id XLabs-AI/flux-controlnet-collections --name flux-depth-controlnet.safetensors --device cuda --use_controlnet \
 --model_type flux-dev --width 1024 --height 1024 \
 --timestep_to_start_cfg 1 --num_steps 25 --true_gs 3.5 --guidance 4

```
![Example Picture 2](https://github.com/XLabs-AI/x-flux/blob/main/assets/readme/examples/depth_example_2.png?raw=true)

```bash
python3 main.py \
 --prompt "Photo of japanese village with houses and sakura, full hd, cinematic photo" \
 --image input_image_depth3.webp \
 --control_type depth \
 --repo_id XLabs-AI/flux-controlnet-collections --name flux-depth-controlnet.safetensors --device cuda --use_controlnet \
 --model_type flux-dev --width 1024 --height 1024 \
 --timestep_to_start_cfg 1 --num_steps 25 --true_gs 3.5 --guidance 4

```
![Example Picture 2](https://github.com/XLabs-AI/x-flux/blob/main/assets/readme/examples/depth_example_3.png?raw=true)


## HED ControlNet (version 1)
```bash
python3 main.py \
 --prompt "2d art of a sitting african rich woman, full hd, cinematic photo" \
 --image input_image_hed1.jpg \
 --control_type hed \
 --repo_id XLabs-AI/flux-controlnet-collections --name flux-hed-controlnet.safetensors --device cuda --use_controlnet \
 --model_type flux-dev --width 768 --height 768 \
 --timestep_to_start_cfg 1 --num_steps 25 --true_gs 3.5 --guidance 4

```
![Example Picture 2](https://github.com/XLabs-AI/x-flux/blob/main/assets/readme/examples/hed_example_1.png?raw=true)

```bash
python3 main.py \
 --prompt "anime ghibli style art of a running happy white dog, full hd" \
 --image input_image_hed2.jpg \
 --control_type hed \
 --repo_id XLabs-AI/flux-controlnet-collections --name flux-hed-controlnet.safetensors --device cuda --use_controlnet \
 --model_type flux-dev --width 768 --height 768 \
 --timestep_to_start_cfg 1 --num_steps 25 --true_gs 3.5 --guidance 4

```
![Example Picture 2](https://github.com/XLabs-AI/x-flux/blob/main/assets/readme/examples/hed_example_2.png?raw=true)


## License

Our weights fall under the [FLUX.1 [dev]](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md) Non-Commercial License<br/>