ByteTrack-CPP-ncnn

Installation

Clone ncnn first, then please following build tutorial of ncnn to build on your own device.

Install eigen-3.3.9 [google], [baidu(code:ueq4)].

unzip eigen-3.3.9.zip
cd eigen-3.3.9
mkdir build
cd build
cmake ..
sudo make install

Generate onnx file

Use provided tools to generate onnx file. For example, if you want to generate onnx file of bytetrack_s_mot17.pth, please run the following command:

cd <ByteTrack_HOME>
python3 tools/export_onnx.py -f exps/example/mot/yolox_s_mix_det.py -c pretrained/bytetrack_s_mot17.pth.tar

Then, a bytetrack_s.onnx file is generated under .

Generate ncnn param and bin file

Put bytetrack_s.onnx under ncnn/build/tools/onnx and then run:

cd ncnn/build/tools/onnx
./onnx2ncnn bytetrack_s.onnx bytetrack_s.param bytetrack_s.bin

Since Focus module is not supported in ncnn. Warnings like:

Unsupported slice step !

will be printed. However, don't worry! C++ version of Focus layer is already implemented in src/bytetrack.cpp.

Modify param file

Open bytetrack_s.param, and modify it. Before (just an example):

235 268
Input            images                   0 1 images
Split            splitncnn_input0         1 4 images images_splitncnn_0 images_splitncnn_1 images_splitncnn_2 images_splitncnn_3
Crop             Slice_4                  1 1 images_splitncnn_3 467 -23309=1,0 -23310=1,2147483647 -23311=1,1
Crop             Slice_9                  1 1 467 472 -23309=1,0 -23310=1,2147483647 -23311=1,2
Crop             Slice_14                 1 1 images_splitncnn_2 477 -23309=1,0 -23310=1,2147483647 -23311=1,1
Crop             Slice_19                 1 1 477 482 -23309=1,1 -23310=1,2147483647 -23311=1,2
Crop             Slice_24                 1 1 images_splitncnn_1 487 -23309=1,1 -23310=1,2147483647 -23311=1,1
Crop             Slice_29                 1 1 487 492 -23309=1,0 -23310=1,2147483647 -23311=1,2
Crop             Slice_34                 1 1 images_splitncnn_0 497 -23309=1,1 -23310=1,2147483647 -23311=1,1
Crop             Slice_39                 1 1 497 502 -23309=1,1 -23310=1,2147483647 -23311=1,2
Concat           Concat_40                4 1 472 492 482 502 503 0=0
...

Change first number for 235 to 235 - 9 = 226(since we will remove 10 layers and add 1 layers, total layers number should minus 9).
Then remove 10 lines of code from Split to Concat, but remember the last but 2nd number: 503.
Add YoloV5Focus layer After Input (using previous number 503):

YoloV5Focus      focus                    1 1 images 503

After(just an exmaple):

226 328
Input            images                   0 1 images
YoloV5Focus      focus                    1 1 images 503
...

Use ncnn_optimize to generate new param and bin

# suppose you are still under ncnn/build/tools/onnx dir.
../ncnnoptimize bytetrack_s.param bytetrack_s.bin bytetrack_s_op.param bytetrack_s_op.bin 65536

Copy files and build ByteTrack

Copy or move 'src', 'include' folders and 'CMakeLists.txt' file into ncnn/examples. Copy bytetrack_s_op.param, bytetrack_s_op.bin and /videos/palace.mp4 into ncnn/build/examples. Then, build ByteTrack:

cd ncnn/build/examples
cmake ..
make

Run the demo

You can run the ncnn demo with 5 FPS (96-core Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz):

./bytetrack palace.mp4

You can modify 'num_threads' to optimize the running speed in bytetrack.cpp according to the number of your CPU cores:

yolox.opt.num_threads = 20;

Acknowledgement

ncnn