Elle McFarlane commited on
Commit
e142172
·
1 Parent(s): 91a6b16

add readme

Browse files
Files changed (1) hide show
  1. text2motion/README.md +110 -0
text2motion/README.md ADDED
@@ -0,0 +1,110 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Text-driven Motion Generation
2
+
3
+ <!-- TOC -->
4
+
5
+ - [Installation](#installation)
6
+ - [Training](#prepare-environment)
7
+ - [Acknowledgement](#acknowledgement)
8
+
9
+ <!-- TOC -->
10
+
11
+ ## Installation
12
+
13
+ Please refer to [install.md](install.md) for detailed installation.
14
+
15
+ ## Training
16
+
17
+ Due to the requirement of a large batchsize, we highly recommend you to use DDP training. A slurm-based script is as below:
18
+
19
+ ```shell
20
+ PYTHONPATH="$(dirname $0)/..":$PYTHONPATH \
21
+ srun -p ${PARTITION} -n8 --gres=gpu:8 -u \
22
+ python -u tools/train.py \
23
+ --name kit_baseline_ddp_8gpu_8layers_1000 \
24
+ --batch_size 128 \
25
+ --times 200 \
26
+ --num_epochs 50 \
27
+ --dataset_name kit \
28
+ --distributed
29
+ ```
30
+
31
+ Besides, you can train the model on multi-GPUs with DataParallel:
32
+
33
+ ```shell
34
+ PYTHONPATH="$(dirname $0)/..":$PYTHONPATH \
35
+ python -u tools/train.py \
36
+ --name kit_baseline_dp_2gpu_8layers_1000 \
37
+ --batch_size 128 \
38
+ --times 50 \
39
+ --num_epochs 50 \
40
+ --dataset_name kit \
41
+ --num_layers 8 \
42
+ --diffusion_steps 1000 \
43
+ --data_parallel \
44
+ --gpu_id 0 1
45
+ ```
46
+
47
+ Otherwise, you can run the training code on a single GPU like:
48
+
49
+ ```shell
50
+ PYTHONPATH="$(dirname $0)/..":$PYTHONPATH \
51
+ python -u tools/train.py \
52
+ --name kit_baseline_1gpu_8layers_1000 \
53
+ --batch_size 128 \
54
+ --times 25 \
55
+ --num_epochs 50 \
56
+ --dataset_name kit
57
+ ```
58
+
59
+ Here, `times` means the duplication times of the original dataset. To retain the number of iterations, you can set `times` to 25 for 1 GPU, 50 for 2 GPUs, 100 for 4 GPUs, and 200 for 8 GPUs.
60
+
61
+ ## Evaluation
62
+
63
+ ```shell
64
+ # GPU_ID indicates which gpu you want to use
65
+ python -u tools/evaluation.py checkpoints/kit/kit_motiondiffuse/opt.txt GPU_ID
66
+ # Or you can omit this option and use cpu for evaluation
67
+ python -u tools/evaluation.py checkpoints/kit/kit_motiondiffuse/opt.txt
68
+ ```
69
+
70
+ ## Visualization
71
+
72
+ You can visualize human motion with the given language description and the expected motion length. We also provide a [Colab Demo](https://colab.research.google.com/drive/1Dp6VsZp2ozKuu9ccMmsDjyij_vXfCYb3?usp=sharing) and a [Hugging Face Demo](https://huggingface.co/spaces/mingyuan/MotionDiffuse) for your convenience.
73
+
74
+ ```shell
75
+ # Currently we only support visualization of models trained on the HumanML3D dataset.
76
+ # Motion length can not be larger than 196, which is the maximum length during training
77
+ # You can omit `gpu_id` to run visualization on your CPU
78
+ # Optionally, you can store the xyz coordinates of each joint to `npy_path`. The shape of motion data is (T, 22, 3), where T denotes the motion length, 22 is the number of joints.
79
+
80
+ python -u tools/visualization.py \
81
+ --opt_path checkpoints/t2m/t2m_motiondiffuse/opt.txt \
82
+ --text "a person is jumping" \
83
+ --motion_length 60 \
84
+ --result_path "test_sample.gif" \
85
+ --npy_path "test_sample.npy" \
86
+ --gpu_id 0
87
+ ```
88
+
89
+ Here are some visualization examples. The motion lengths are shown in the title of animations.
90
+
91
+ <table>
92
+ <tr>
93
+ <td><img src="../figures/gallery_t2m/gen_00.gif" width="100%"/></td>
94
+ <td><img src="../figures/gallery_t2m/gen_01.gif" width="100%"/></td>
95
+ <td><img src="../figures/gallery_t2m/gen_02.gif" width="100%"/></td>
96
+ <td><img src="../figures/gallery_t2m/gen_03.gif" width="100%"/></td>
97
+ </tr>
98
+ <tr>
99
+ <td><img src="../figures/gallery_t2m/gen_04.gif" width="100%"/></td>
100
+ <td><img src="../figures/gallery_t2m/gen_05.gif" width="100%"/></td>
101
+ <td><img src="../figures/gallery_t2m/gen_06.gif" width="100%"/></td>
102
+ <td><img src="../figures/gallery_t2m/gen_07.gif" width="100%"/></td>
103
+ </tr>
104
+ </table>
105
+
106
+ **Note:** You may install `matplotlib==3.3.1` to support visualization here.
107
+
108
+ ## Acknowledgement
109
+
110
+ This code is developed on top of [Generating Diverse and Natural 3D Human Motions from Text](https://github.com/EricGuo5513/text-to-motion)