File size: 1,741 Bytes
e34aada |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
# process dataset
we use Talking-Head-1K-Hour as the example.
## download and crop the talking person video clips
- Please follow the step in [https://github.com/tcwang0509/TalkingHead-1KH](https://github.com/tcwang0509/TalkingHead-1KH)
- Put all extracted video clips in a directory like `/home/xxx/TH1KH_512/video_raw/*.mp4`
## resample & resize video clips to 512x512 resolution and 25FPS
- You can use the example code in `data_gen/utils/process_video/resample_video_to_25fps_resize_to_512.py`
- It will generate processed video clips in `/home/xxx/TH1KH_512/video/*.mp4`
## extract segment images
- You can use the example code in `data_gen/utils/process_video/extract_segment_imgs.py`
- It will generate segment images in `/home/xxx/TH1KH_512/{gt_imgs, head_imgs, inpaint_torso_imgs, com_imgs}/*`
## extract 2d facial landmark
- You can use the example code in `data_gen/utils/process_video/extract_lm2d.py`
- It will generate 2d landmarks in `/home/xxx/TH1KH_512/lms_2d/*_lms_2d.npy`
## extract 3dmm coefficients
- You can use the example code in `data_gen/utils/process_video/fit_3dmm_landmark.py`
- It will generate 3dmm coefficients in `/home/xxx/TH1KH_512/coeff_fit_mp/*_coeff_fit_mp.npy`
## extract audio features
- You can use the example code in `data_gen/utils/process_audio/extract_mel_f0.py`
- It will generate raw wav in `/home/xxx/TH1KH_512/audio/*.wav` and mel_f0 in `/home/xxx/TH1KH_512/mel_f0/*_mel_f0.npy`
- You can use the example code in `data_gen/utils/process_audio/extract_hubert.py`
- It will generate hubert in `/home/xxx/TH1KH_512/hubert/*_hubert.npy`
## Binarize the dataset
- You can use the example code in `data_gen/runs/binarizer_th1kh.py`
- You will see a binarized dataset at `data/binary/th1kh`
|