data_process/README.md

Data Preprocessing Pipeline by AvatarArtist

This repo describes how to process your own data for using our model.

🎉 Overview

⚙️ Requirements and Installation

We recommend the requirements as follows.

Environment

git clone --depth=1 https://github.com/ant-research/AvatarArtist 
cd AvatarArtist
conda create -n avatarartis python=3.9.0
conda activate avatarartist
pip install -r requirements.txt

Download Weights

The weights are available at 🤗HuggingFace, you can download it with the following commands. Please move the required files into the pretrained_model directory:

# if you are in china mainland, run this first: export HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download --repo-type model \
KUMAPOWER/AvatarArtist \
--local-dir pretrained_model

🗝️ Usage

Process the target video to obtain the target pose motion and mesh.

python3 input_img_align_extract_ldm.py --input_dir ./demo_data/hongyu_2.mp4 --is_video --save_dir ./demo_data/data_process_out

Process the image to extract the source image.

python3 input_img_align_extract_ldm.py --input_dir ./demo_data/ip_imgs --is_img --save_dir ./demo_data/data_process_out 

Our code supports step-by-step data processing. For example, if your images are already aligned, you can proceed directly to the next step.

python3 input_img_align_extract_ldm.py --input_dir ./demo_data/ip_imgs --is_img --save_dir ./demo_data/data_process_out  --already_align

Once ready, the data will be organized in this format:

📦 datasets/
├── 📂 dataset/
│   ├── 📂 coeffs/
│   ├── 📂 images512x512/
│   ├── 📂 uvRender256x256/
│   ├── 📂 orthRender256x256_face_eye/
│   ├── 📂 motions/
├── 📂 crop_fv_tracking/
├── 📂 realign_detections/
├── 📂 realign_detections/
├── 📂 realign/
├── 📂 raw_detection/
├── 📂 align_3d_landmark/
├── 📂 raw_frames/

👍 Credits

• This code builds on Portrait4D and InvertAvatar. We have integrated and organized their data processing code. Thanks for open-sourcing!