File size: 6,303 Bytes

6fecfbe

# vyro-workflows

Based on comfy hash 2ef459b

## To use:

The `unified-api-workflow.json` contains the following keys:

```
"keys": {
    "input": "16",
    "output_default": "57",
    "output_face_swap_stage1": "125",
    "output_face_swap_stage2": "129",
    "output_headshot":"357",
    "output_remix_stage1":"255",
    "output_remix_stage2":"190",
    "output_qr":"301"
  }
```

T2I, I2I, Variate modes all use `output_default`.
Face swap uses `output_face_swap_stage1` and `output_face_swap_stage2`, depending on whether the user wants the additional denoising stage for facial accuracy (performance can vary depending on the input image and output style).

Remix uses `output_remix_stage1` and `output_remix_stage2`. Remix uses several different image inputs:

* `init_img` is the image to be remixed. Can be blank in which case image starts from noise.
* `controlnet_input_img` is the image to be used as a controlnet input. Can be blank in which case the first `image_prompt` image is used as the controlnet input.
* `image_prompt` is one or more images to be used as styles to be blended. The `image_prompt_weights` param is a comma seperated of floats that determines the weight of each input image. If blank, all styles are weighted equally at 1.0. Valid values are from 0.0-1.0.
* `face_swap_image` is the image to be used for face swapping. Can be blank in which case no face swap is performed.

Headshot expects `init_img` to be the image to be swapped for the headshot. It ignores the `face_swap_img` param.

QR Code expects the `controlnet_input_img` to be a black & white image.

## Running in Comfy

The workflow will break when using in Comfy unless you disconnect the preview nodes for the modes you're not using.

<img width="818" alt="image" src="https://github.com/Vyro-ai/vyro-workflows/assets/122644869/8727c6c6-8ee1-4740-99b6-5023ffc9afe8">

Disconnect these preview nodes when not using Face Swap.

<img width="651" alt="image" src="https://github.com/Vyro-ai/vyro-workflows/assets/122644869/98d9adb2-b138-4049-872b-118c75bac97a">

Disconnect this preview node when not using T2I/I2I/Variate

To use face_swap or i2i, convert the `init_img` or `face_swap_img` params to inputs in the input node, then connect the image loader/base64 encoder nearby.
`<img width="842" alt="image" src="https://github.com/Vyro-ai/vyro-workflows/assets/122644869/8999d29a-aafb-4c7c-b029-db04b3b3d0be">`

## Integration notes:

Formerly, prompt analysis was done outside of the Comfy workflow. Preferrably now, inputs should be passed directly into the workflow.

init_img and face_swap_img are expected to be base64 encoded strings of PIL-compatible images.

## Requirements

Depends on the `comfyui-reactor-node` package. Please install seperately.
Make sure to install the requirements.txt file in the root of the repo. The cv2 version is very important to not conflict with reactor

### Prompt Analysis Spacy model

`cd ComfyUI/models`

`mkdir spacy`

`cd ../custom_nodes/vyro-workflows`

`unzip spa.zip -d ../../models/spacy`

### Interposer model:

`cd ComfyUI/models`

`mkdir interposers`

```
python3 -c 'from huggingface_hub import hf_hub_download

hf_hub_download("city96/SD-Latent-Interposer", local_dir="./interposers", 
filename=f"xl-to-v1_interposer-v1.1.safetensors",
force_download=True, local_dir_use_symlinks=False)'
```

### Requires the following models in the ComfyUI/models/checkpoints folder:

* sd_xl_base_1.0.safetensors
* juggernaut.safetensors
* rev_animated.safetensors
* dreamshaperv8.safetensors

### Requires the following Lora's in the ComfyUI/models/loras folder:

* 3dRenderStyleXL.safetensors
* xl_more_art_full_v1.safetensors
* juggernaut_cinematic_xl.safetensors

### Models for QR CODE (1.5 models only):

* checkpoint: https://civitai.com/models/143386/incredible-world
* lora: https://civitai.com/models/82098/add-more-details-detail-enhancer-tweaker-lora
* controlnet Tile: https://huggingface.co/lllyasviel/ControlNet-v1-1/blob/main/control_v11f1e_sd15_tile.pth
* controlnet QR code: https://huggingface.co/monster-labs/control_v1p_sd15_qrcode_monster/blob/main/control_v1p_sd15_qrcode_monster.ckpt

### Requires the following models in the ComfyUI/models/ipadapter folder:

* IPAdapter+Face SD1.5- https://huggingface.co/h94/IP-Adapter/resolve/main/models/ip-adapter-plus-face_sd15.bin
* IPAdapter+ SD1.5 - https://huggingface.co/h94/IP-Adapter/resolve/main/models/ip-adapter-plus_sd15.bin
* IPAdapter SDXL - https://huggingface.co/h94/IP-Adapter/resolve/main/sdxl_models/ip-adapter_sdxl.bin
* IPAdapter SDXL ViT-H - https://huggingface.co/h94/IP-Adapter/resolve/main/sdxl_models/ip-adapter_sdxl_vit-h.bin

### Requires the following models in the Comfyui/models/clip_vision folder:

* 1_5_clip_vision.safetensors - https://huggingface.co/h94/IP-Adapter/resolve/main/models/image_encoder/model.safetensors

### AI Headshots

* epic_realism_inpaint.safetensors - https://civitai.com/models/90018/epicrealism-pureevolution-inpainting

### Outpainting workflow compatible with multiple images

* best model : dreamshaper_8inpainting.safetensors - https://civitai.com/models/4384?modelVersionId=131004
* inpainting controlnet model : https://huggingface.co/comfyanonymous/ControlNet-v1-1_fp16_safetensors/blob/main/control_v11p_sd15_inpaint_fp16.safetensors

Recommended parameters:
steps: 50
cfg 10-14

Workflow using vyro 2 input outpaint mode -> outpaint_standalone.json

Example Output :
`<img width="842" alt="image" src="https://i.imgur.com/0jW5EJk.png">`

### Object Remover Inpaint

Workflow in :
`workflows/object_remover_1.json`

Recommended parameters:

* cfg : 6-8
* steps: 30-40
* denoise: 0.4-0.8

Requirements:

- This workflow uses lama, you need to download this lama model:
  `https://huggingface.co/hhhzzz/big-lama/resolve/main/big-lama.ckpt`
  and put it in `vyro-workflows\nodes\inpaint_utils\models\big-lama.ckpt`
- Based on another repo so you may need these requirements : `https://github.com/hhhzzyang/Comfyui_Lama/blob/main/requirements.txt`
- Used this checkpoint seems to get the best results for realistic outputs:
  `https://civitai.com/models/25694?modelVersionId=134361`

Example Output :
<img width="842" alt="image" src="https://i.imgur.com/WdouZE4.png">
<img width="842" alt="image" src="https://i.imgur.com/SvFTzeu.png">