1 Introduction
-
The annotation protocol was not consistent across all sequences since some of the ground truth was collected from various online sources;
-
the distribution of crowd density was not balanced for training and test sequences;
-
some of the sequences were well-known (e.g., PETS09-S2L1) and methods were overfitted to them, which made them not ideal for testing purposes;
-
the provided public detections did not show good performance on the benchmark, which made some participants switch to other pedestrian detectors.
-
To present the MOTChallenge benchmark for a fair evaluation of multi-target tracking methods, along with its first releases: MOT15, MOT16, and MOT17;
-
to analyze the performance of 73 state-of-the-art trackers on MOT15, 74 trackers on MOT16, and 57 on MOT17 to analyze trends in MOT over the years. We analyze the main weaknesses of current trackers and discuss promising research directions for the community to advance the field of multi-target tracking.
2 Related work
3 History of MOTChallenge
4 MOT15 Release
4.1 Sequences
4.2 Detections
4.3 Weaknesses of MOT15
-
Annotations we collected annotations online for the existing sequences, while we manually annotated the new sequences. Some of the collected annotations were not accurate enough, especially in scenes with moving cameras.
-
Difficulty generally, we wanted to include some well-known sequences, e.g., PETS2009, in the MOT15 benchmark. However, these sequences have turned out to be too simple for state-of-the-art trackers why we concluded to create a new and more challenging benchmark.
5 MOT16 and MOT17 Releases
5.1 MOT16 Sequences
5.2 Detections
6 Evaluation
6.1 Multiple Object Tracking Accuracy
6.2 Multiple Object Tracking Precision
6.3 Identification Precision, Identification Recall, and F1 Score
6.4 Track Quality Measures
Method | MOTA | IDF1 | MOTP | FAR | MT | ML | FP | FN | IDSW | FM | IDSWR | FMR |
---|---|---|---|---|---|---|---|---|---|---|---|---|
MPNTrack (Brasó and Leal-Taixé 2020) | 51.54 | 58.61 | 76.05 | 1.32 | 225 | 187 | 7620 | 21,780 | 375 | 872 | 5.81 | 13.51 |
Tracktor++v2 (Bergmann et al. 2019) | 46.60 | 47.57 | 76.36 | 0.80 | 131 | 201 | 4624 | 26,896 | 1290 | 1702 | 22.94 | 30.27 |
TrctrD15 (Xu et al. 2020) | 44.09 | 45.99 | 75.26 | 1.05 | 124 | 192 | 6085 | 26,917 | 1347 | 1868 | 23.97 | 33.24 |
Tracktor++ (Bergmann et al. 2019) | 44.06 | 46.73 | 75.03 | 1.12 | 130 | 189 | 6477 | 26,577 | 1318 | 1790 | 23.23 | 31.55 |
KCF (Chu et al. 2019) | 38.90 | 44.54 | 70.56 | 1.27 | 120 | 227 | 7321 | 29,501 | 720 | 1440 | 13.85 | 27.70 |
AP_HWDPL_p (Long et al. 2017) | 38.49 | 47.10 | 72.56 | 0.69 | 63 | 270 | 4005 | 33,203 | 586 | 1263 | 12.75 | 27.48 |
STRN (Xu et al. 2019) | 38.06 | 46.62 | 72.06 | 0.94 | 83 | 241 | 5451 | 31,571 | 1033 | 2665 | 21.25 | 54.82 |
AMIR15 (Sadeghian et al. 2017) | 37.57 | 46.01 | 71.66 | 1.37 | 114 | 193 | 7933 | 29,397 | 1026 | 2024 | 19.67 | 38.81 |
JointMC (Keuper et al. 2018) | 35.64 | 45.12 | 71.90 | 1.83 | 167 | 283 | 10,580 | 28,508 | 457 | 969 | 8.53 | 18.08 |
RAR15pub (Fang et al. 2018) | 35.11 | 45.40 | 70.94 | 1.17 | 94 | 305 | 6771 | 32,717 | 381 | 1523 | 8.15 | 32.58 |
HybridDAT (Yang et al. 2017) | 34.97 | 47.72 | 72.57 | 1.46 | 82 | 304 | 8455 | 31,140 | 358 | 1267 | 7.26 | 25.69 |
INARLA (Wu et al. 2019) | 34.69 | 42.06 | 70.72 | 1.71 | 90 | 216 | 9855 | 29,158 | 1112 | 2848 | 21.16 | 54.20 |
STAM (Chu et al. 2017) | 34.33 | 48.26 | 70.55 | 0.89 | 82 | 313 | 5154 | 34,848 | 348 | 1463 | 8.04 | 33.80 |
QuadMOT (Son et al. 2017) | 33.82 | 40.43 | 73.42 | 1.37 | 93 | 266 | 7898 | 32,061 | 703 | 1430 | 14.70 | 29.91 |
NOMT (Choi 2015) | 33.67 | 44.55 | 71.94 | 1.34 | 88 | 317 | 7762 | 32,547 | 442 | 823 | 9.40 | 17.50 |
DCCRF (Zhou et al. 2018a) | 33.62 | 39.08 | 70.91 | 1.02 | 75 | 271 | 5917 | 34,002 | 866 | 1566 | 19.39 | 35.07 |
TDAM (Yang and Jia 2016) | 33.03 | 46.05 | 72.78 | 1.74 | 96 | 282 | 10,064 | 30,617 | 464 | 1506 | 9.25 | 30.02 |
CDA_DDALpb (Bae and Yoon 2018) | 32.80 | 38.79 | 70.70 | 0.86 | 70 | 304 | 4983 | 35,690 | 614 | 1583 | 14.65 | 37.77 |
MHT_DAM (Kim et al. 2015) | 32.36 | 45.31 | 71.83 | 1.57 | 115 | 316 | 9064 | 32,060 | 435 | 826 | 9.10 | 17.27 |
LFNF (Sheng et al. 2017) | 31.64 | 33.10 | 72.03 | 1.03 | 69 | 301 | 5943 | 35,095 | 961 | 1106 | 22.41 | 25.79 |
GMPHD_OGM (Song et al. 2019) | 30.72 | 38.82 | 71.64 | 1.13 | 83 | 275 | 6502 | 35,030 | 1034 | 1351 | 24.05 | 31.43 |
PHD_GSDL (Fu et al. 2018) | 30.51 | 38.82 | 71.20 | 1.13 | 55 | 297 | 6534 | 35,284 | 879 | 2208 | 20.65 | 51.87 |
MDP (Xiang et al. 2015) | 30.31 | 44.68 | 71.32 | 1.68 | 94 | 277 | 9717 | 32,422 | 680 | 1500 | 14.40 | 31.76 |
MCF_PHD (Wojke and Paulus 2016) | 29.89 | 38.18 | 71.70 | 1.54 | 86 | 317 | 8892 | 33,529 | 656 | 989 | 14.44 | 21.77 |
CNNTCM (Wang et al. 2016) | 29.64 | 36.82 | 71.78 | 1.35 | 81 | 317 | 7786 | 34,733 | 712 | 943 | 16.38 | 21.69 |
RSCNN (Mahgoub et al. 2017) | 29.50 | 36.97 | 73.07 | 2.05 | 93 | 262 | 11,866 | 30,474 | 976 | 1176 | 19.36 | 23.33 |
TBSS15 (Zhou et al. 2018b) | 29.21 | 37.23 | 71.28 | 1.05 | 49 | 316 | 6068 | 36,779 | 649 | 1508 | 16.17 | 37.57 |
SCEA (Yoon et al. 2016) | 29.08 | 37.15 | 71.11 | 1.05 | 64 | 341 | 6060 | 36,912 | 604 | 1182 | 15.13 | 29.61 |
SiameseCNN (Leal-Taixe et al. 2016) | 29.04 | 34.27 | 71.20 | 0.89 | 61 | 349 | 5160 | 37798 | 639 | 1316 | 16.61 | 34.20 |
HAM_INTP15 (Yoon et al. 2018a) | 28.62 | 41.45 | 71.13 | 1.30 | 72 | 317 | 7485 | 35,910 | 460 | 1038 | 11.07 | 24.98 |
GMMA_intp (Song et al. 2018) | 27.32 | 36.59 | 70.92 | 1.36 | 47 | 311 | 7848 | 35,817 | 987 | 1848 | 23.67 | 44.31 |
oICF (Kieritz et al. 2016) | 27.08 | 40.49 | 69.96 | 1.31 | 46 | 351 | 7594 | 36,757 | 454 | 1660 | 11.30 | 41.32 |
TO (Manen et al. 2016) | 25.66 | 32.74 | 72.17 | 0.83 | 31 | 414 | 4779 | 40,511 | 383 | 600 | 11.24 | 17.61 |
LP_SSVM (Wang and Fowlkes 2016) | 25.22 | 34.05 | 71.68 | 1.45 | 42 | 382 | 8369 | 36,932 | 646 | 849 | 16.19 | 21.28 |
HAM_SADF (Yoon et al. 2018a) | 25.19 | 37.80 | 71.38 | 1.27 | 41 | 420 | 7330 | 38,275 | 357 | 745 | 9.47 | 19.76 |
ELP (McLaughlin et al. 2015) | 24.99 | 26.21 | 71.17 | 1.27 | 54 | 316 | 7345 | 37,344 | 1396 | 1804 | 35.60 | 46.00 |
AdTobKF (Loumponias et al. 2018) | 24.82 | 34.50 | 70.78 | 1.07 | 29 | 375 | 6201 | 39,321 | 666 | 1300 | 18.50 | 36.11 |
LINF1 (Fagot-Bouquet et al. 2016) | 24.53 | 34.82 | 71.33 | 1.01 | 40 | 466 | 5864 | 40,207 | 298 | 744 | 8.62 | 21.53 |
TENSOR (Shi et al. 2018) | 24.32 | 24.13 | 71.58 | 1.15 | 40 | 336 | 6644 | 38,582 | 1271 | 1304 | 34.16 | 35.05 |
TFMOT (Boragule and Jeon 2017) | 23.81 | 32.30 | 71.35 | 0.78 | 35 | 447 | 4533 | 41,873 | 404 | 792 | 12.69 | 24.87 |
JPDA_m (Rezatofighi et al. 2015) | 23.79 | 33.77 | 68.17 | 1.10 | 36 | 419 | 6373 | 40,084 | 365 | 869 | 10.50 | 25.00 |
MotiCon (Leal-Taixé et al. 2014) | 23.07 | 29.38 | 70.87 | 1.80 | 34 | 375 | 10,404 | 35,844 | 1018 | 1061 | 24.44 | 25.47 |
DEEPDA_MOT (Yoon et al. 2019a) | 22.53 | 25.92 | 70.92 | 1.27 | 46 | 447 | 7346 | 39,092 | 1159 | 1538 | 31.86 | 42.28 |
SegTrack (Milan et al. 2015) | 22.51 | 31.48 | 71.65 | 1.36 | 42 | 461 | 7890 | 39,020 | 697 | 737 | 19.10 | 20.20 |
EAMTTpub (Sanchez-Matilla et al. 2016) | 22.30 | 32.84 | 70.79 | 1.37 | 39 | 380 | 7924 | 38,982 | 833 | 1485 | 22.79 | 40.63 |
SAS_MOT15 (Maksai and Fua 2019) | 22.16 | 27.15 | 71.10 | 0.97 | 22 | 444 | 5591 | 41,531 | 700 | 1240 | 21.60 | 38.27 |
OMT_DFH (Ju et al. 2017a) | 21.16 | 37.34 | 69.94 | 2.29 | 51 | 335 | 13,218 | 34,657 | 563 | 1255 | 12.92 | 28.79 |
MTSTracker (Nguyen Thi Lan Anh et al. 2017) | 20.64 | 31.87 | 70.32 | 2.62 | 65 | 266 | 15,161 | 32,212 | 1387 | 2357 | 29.16 | 49.55 |
TC_SIAMESE (Yoon et al. 2018b) | 20.22 | 32.59 | 71.09 | 1.06 | 19 | 487 | 6127 | 42,596 | 294 | 825 | 9.59 | 26.90 |
DCO_X (Milan et al. 2016) | 19.59 | 31.45 | 71.39 | 1.84 | 37 | 396 | 10,652 | 38,232 | 521 | 819 | 13.79 | 21.68 |
CEM (Milan et al. 2014) | 19.30 | N/A | 70.74 | 2.45 | 61 | 335 | 14,180 | 34,591 | 813 | 1023 | 18.60 | 23.41 |
RNN_LSTM (Milan et al. 2017) | 18.99 | 17.12 | 70.97 | 2.00 | 40 | 329 | 11,578 | 36,706 | 1490 | 2081 | 37.01 | 51.69 |
RMOT (Yoon et al. 2015) | 18.63 | 32.56 | 69.57 | 2.16 | 38 | 384 | 12,473 | 36,835 | 684 | 1282 | 17.08 | 32.01 |
TSDA_OAL (Ju et al. 2017b) | 18.61 | 36.07 | 69.68 | 2.83 | 68 | 305 | 16,350 | 32,853 | 806 | 1544 | 17.32 | 33.18 |
GMPHD_15 (Song and Jeon 2016) | 18.47 | 28.38 | 70.90 | 1.36 | 28 | 399 | 7864 | 41,766 | 459 | 1266 | 14.33 | 39.54 |
SMOT (Dicle et al. 2013) | 18.23 | 0.00 | 71.23 | 1.52 | 20 | 395 | 8780 | 40,310 | 1148 | 2132 | 33.38 | 61.99 |
ALExTRAC (Bewley et al. 2016b) | 16.95 | 17.30 | 71.18 | 1.60 | 28 | 378 | 9233 | 39,933 | 1859 | 1872 | 53.11 | 53.48 |
TBD (Geiger et al. 2014) | 15.92 | 0.00 | 70.86 | 2.58 | 46 | 345 | 14,943 | 34,777 | 1939 | 1963 | 44.68 | 45.23 |
GSCR (Fagot-Bouquet et al. 2015) | 15.78 | 27.90 | 69.38 | 1.31 | 13 | 440 | 7597 | 43,633 | 514 | 1010 | 17.73 | 34.85 |
TC_ODAL (Bae and Yoon 2014) | 15.13 | 0.00 | 70.53 | 2.24 | 23 | 402 | 12,970 | 38,538 | 637 | 1716 | 17.09 | 46.04 |
DP_NMS (Pirsiavash et al. 2011) | 14.52 | 19.69 | 70.76 | 2.28 | 43 | 294 | 13,171 | 34,814 | 4537 | 3090 | 104.69 | 71.30 |
Method | MOTA | IDF1 | MOTP | FAR | MT | ML | FP | FN | IDSW | FM | IDSWR | FMR |
---|---|---|---|---|---|---|---|---|---|---|---|---|
MPNTrack (Brasó and Leal-Taixé 2020) | 58.56 | 61.69 | 78.88 | 0.84 | 207 | 258 | 4949 | 70,252 | 354 | 684 | 5.76 | 11.13 |
Tracktor++v2 (Bergmann et al. 2019) | 56.20 | 54.91 | 79.20 | 0.40 | 157 | 272 | 2394 | 76,844 | 617 | 1068 | 10.66 | 18.46 |
TrctrD16 (Xu et al. 2020) | 54.83 | 53.39 | 77.47 | 0.50 | 145 | 281 | 2955 | 78,765 | 645 | 1515 | 11.36 | 26.67 |
Tracktor++ (Bergmann et al. 2019) | 54.42 | 52.54 | 78.22 | 0.55 | 144 | 280 | 3280 | 79,149 | 682 | 1480 | 12.05 | 26.15 |
NOTA_16 (Chen et al. 2019) | 49.83 | 55.33 | 74.49 | 1.22 | 136 | 286 | 7248 | 83,614 | 614 | 1372 | 11.34 | 25.34 |
HCC (Ma et al. 2018b) | 49.25 | 50.67 | 79.00 | 0.90 | 135 | 303 | 5333 | 86,795 | 391 | 535 | 7.46 | 10.21 |
eTC (Wang et al. 2019) | 49.15 | 56.11 | 75.49 | 1.42 | 131 | 306 | 8400 | 83,702 | 606 | 882 | 11.20 | 16.31 |
KCF16 (Chu et al. 2019) | 48.80 | 47.19 | 75.66 | 0.99 | 120 | 289 | 5875 | 86,567 | 906 | 1116 | 17.25 | 21.25 |
LMP (Tang et al. 2017) | 48.78 | 51.26 | 79.04 | 1.12 | 138 | 304 | 6654 | 86,245 | 481 | 595 | 9.13 | 11.29 |
TLMHT (Sheng et al. 2018a) | 48.69 | 55.29 | 76.43 | 1.12 | 119 | 338 | 6632 | 86,504 | 413 | 642 | 7.86 | 12.22 |
STRN_MOT16 (Xu et al. 2019) | 48.46 | 53.90 | 73.75 | 1.53 | 129 | 265 | 9038 | 84,178 | 747 | 2919 | 13.88 | 54.23 |
GCRA (Ma et al. 2018a) | 48.16 | 48.55 | 77.50 | 0.86 | 98 | 312 | 5104 | 88,586 | 821 | 1117 | 15.97 | 21.73 |
FWT (Henschel et al. 2018) | 47.77 | 44.28 | 75.51 | 1.50 | 145 | 290 | 8886 | 85,487 | 852 | 1534 | 16.04 | 28.88 |
MOTDT (Long et al. 2018) | 47.63 | 50.94 | 74.81 | 1.56 | 115 | 291 | 9253 | 85,431 | 792 | 1858 | 14.90 | 34.96 |
NLLMPa (Levinkov et al. 2017) | 47.58 | 47.34 | 78.51 | 0.99 | 129 | 307 | 5844 | 89,093 | 629 | 768 | 12.30 | 15.02 |
EAGS16 (Sheng et al. 2018b) | 47.41 | 50.13 | 75.95 | 1.41 | 131 | 324 | 8369 | 86,931 | 575 | 913 | 10.99 | 17.45 |
JCSTD (Tian et al. 2019) | 47.36 | 41.10 | 74.43 | 1.36 | 109 | 276 | 8076 | 86,638 | 1266 | 2697 | 24.12 | 51.39 |
ASTT (Tao et al. 2018) | 47.24 | 44.27 | 76.08 | 0.79 | 124 | 316 | 4680 | 90,877 | 633 | 814 | 12.62 | 16.23 |
eHAF16 (Sheng et al. 2018c) | 47.22 | 52.44 | 75.69 | 2.13 | 141 | 325 | 12,586 | 83,107 | 542 | 787 | 9.96 | 14.46 |
AMIR (Sadeghian et al. 2017) | 47.17 | 46.29 | 75.82 | 0.45 | 106 | 316 | 2681 | 92,856 | 774 | 1675 | 15.77 | 34.14 |
JointMC (MCjoint) (Keuper et al. 2018) | 47.10 | 52.26 | 76.27 | 1.13 | 155 | 356 | 6703 | 89,368 | 370 | 598 | 7.26 | 11.73 |
YOONKJ16 (Yoon et al. 2020) | 46.96 | 50.05 | 75.76 | 1.33 | 125 | 317 | 7901 | 88,179 | 627 | 945 | 12.14 | 18.30 |
NOMT_16 (Choi 2015) | 46.42 | 53.30 | 76.56 | 1.65 | 139 | 314 | 9753 | 87,565 | 359 | 504 | 6.91 | 9.70 |
JMC (Tang et al. 2016) | 46.28 | 46.31 | 75.68 | 1.08 | 118 | 301 | 6373 | 90,914 | 657 | 1114 | 13.10 | 22.22 |
DD_TAMA16 (Yoon et al. 2019b) | 46.20 | 49.43 | 75.42 | 0.87 | 107 | 334 | 5126 | 92,367 | 598 | 1127 | 12.12 | 22.84 |
DMAN_16 (Zhu et al. 2018) | 46.08 | 54.82 | 73.77 | 1.34 | 132 | 324 | 7909 | 89,874 | 532 | 1616 | 10.49 | 31.87 |
STAM16 (Chu et al. 2017) | 45.98 | 50.05 | 74.92 | 1.16 | 111 | 331 | 6895 | 91,117 | 473 | 1422 | 9.46 | 28.43 |
RAR16pub (Fang et al. 2018) | 45.87 | 48.77 | 74.84 | 1.16 | 100 | 318 | 6871 | 91,173 | 648 | 1992 | 12.96 | 39.85 |
MHT_DAM_16 (Kim et al. 2015) | 45.83 | 46.06 | 76.34 | 1.08 | 123 | 328 | 6412 | 91,758 | 590 | 781 | 11.88 | 15.72 |
MTDF (Fu et al. 2019) | 45.72 | 40.07 | 72.63 | 2.03 | 107 | 276 | 12,018 | 84,970 | 1987 | 3377 | 37.21 | 63.24 |
INTERA_MOT (Lan et al. 2018) | 45.40 | 47.66 | 74.41 | 2.27 | 137 | 294 | 13,407 | 85,547 | 600 | 930 | 11.30 | 17.52 |
EDMT (Chen et al. 2017a) | 45.34 | 47.86 | 75.94 | 1.88 | 129 | 303 | 11,122 | 87,890 | 639 | 946 | 12.34 | 18.27 |
DCCRF16 (Zhou et al. 2018a) | 44.76 | 39.67 | 75.63 | 0.95 | 107 | 321 | 5613 | 94,133 | 968 | 1378 | 20.01 | 28.49 |
TBSS (Zhou et al. 2018b) | 44.58 | 42.64 | 75.18 | 0.70 | 93 | 333 | 4136 | 96,128 | 790 | 1419 | 16.71 | 30.01 |
OTCD_1_16 (Liu et al. 2019) | 44.36 | 45.62 | 75.36 | 0.97 | 88 | 361 | 5759 | 94,927 | 759 | 1787 | 15.83 | 37.28 |
QuadMOT16 (Son et al. 2017) | 44.10 | 38.27 | 76.40 | 1.08 | 111 | 341 | 6388 | 94775 | 745 | 1096 | 15.52 | 22.83 |
CDA_DDALv2 (Bae and Yoon 2018) | 43.89 | 45.13 | 74.69 | 1.09 | 81 | 337 | 6450 | 95,175 | 676 | 1795 | 14.14 | 37.55 |
LFNF16 (Sheng et al. 2017) | 43.61 | 41.62 | 76.63 | 1.12 | 101 | 347 | 6616 | 95,363 | 836 | 938 | 17.53 | 19.67 |
oICF_16 (Kieritz et al. 2016) | 43.21 | 49.33 | 74.31 | 1.12 | 86 | 368 | 6651 | 96,515 | 381 | 1404 | 8.10 | 29.83 |
MHT_bLSTM6 (Kim et al. 2018) | 42.10 | 47.84 | 75.85 | 1.97 | 113 | 337 | 11,637 | 93,172 | 753 | 1156 | 15.40 | 23.64 |
LINF1_16 (Fagot-Bouquet et al. 2016) | 41.01 | 45.69 | 74.85 | 1.33 | 88 | 389 | 7896 | 99,224 | 430 | 963 | 9.43 | 21.13 |
PHD_GSDL16 (Fu et al. 2018) | 41.00 | 43.14 | 75.90 | 1.10 | 86 | 315 | 6498 | 99,257 | 1810 | 3650 | 39.73 | 80.11 |
GMPHD_ReId Baisa (2019b) | 40.42 | 49.71 | 75.25 | 1.11 | 85 | 329 | 6572 | 101,266 | 792 | 2529 | 17.81 | 56.88 |
AM_ADM (Lee et al. 2018) | 40.12 | 43.79 | 75.45 | 1.44 | 54 | 351 | 8503 | 99,891 | 789 | 1736 | 17.45 | 38.40 |
EAMTT_pub (Sanchez-Matilla et al. 2016) | 38.83 | 42.43 | 75.15 | 1.37 | 60 | 373 | 8114 | 102,452 | 965 | 1657 | 22.03 | 37.83 |
OVBT (Ban et al. 2016) | 38.40 | 37.82 | 75.39 | 1.95 | 57 | 359 | 11,517 | 99,463 | 1321 | 2140 | 29.07 | 47.09 |
GMMCP (Dehghan et al. 2015) | 38.10 | 35.50 | 75.84 | 1.12 | 65 | 386 | 6607 | 105,315 | 937 | 1669 | 22.18 | 39.51 |
LTTSC-CRF (Le et al. 2016) | 37.59 | 42.06 | 75.94 | 2.02 | 73 | 419 | 11,969 | 101,343 | 481 | 1012 | 10.83 | 22.79 |
JCmin_MOT (Boragule and Jeon 2017) | 36.65 | 36.16 | 75.86 | 0.50 | 57 | 413 | 2936 | 111,890 | 667 | 831 | 17.27 | 21.51 |
HISP_T (Baisa 2018) | 35.87 | 28.93 | 76.07 | 1.08 | 59 | 380 | 6412 | 107,918 | 2594 | 2298 | 63.56 | 56.31 |
LP2D_16 (Leal-Taixé et al. 2014) | 35.74 | 34.18 | 75.84 | 0.86 | 66 | 385 | 5084 | 111,163 | 915 | 1264 | 23.44 | 32.39 |
GM_PHD_DAL (Baisa 2019a) | 35.13 | 26.58 | 76.59 | 0.40 | 53 | 390 | 2350 | 111,886 | 4047 | 5338 | 104.75 | 138.17 |
TBD_16 (Geiger et al. 2014) | 33.74 | 0.00 | 76.53 | 0.98 | 55 | 411 | 5804 | 112,587 | 2418 | 2252 | 63.22 | 58.88 |
GM_PHD_N1T (Baisa and Wallace 2019) | 33.25 | 25.47 | 76.84 | 0.30 | 42 | 425 | 1750 | 116,452 | 3499 | 3594 | 96.85 | 99.47 |
CEM_16 (Milan et al. 2014) | 33.19 | N/A | 75.84 | 1.16 | 59 | 413 | 6837 | 114,322 | 642 | 731 | 17.21 | 19.60 |
GMPHD_HDA (Song and Jeon 2016) | 30.52 | 33.37 | 75.42 | 0.87 | 35 | 453 | 5169 | 120,970 | 539 | 731 | 16.02 | 21.72 |
SMOT_16 (Dicle et al. 2013) | 29.75 | N/A | 75.18 | 2.94 | 40 | 362 | 17,426 | 107,552 | 3108 | 4483 | 75.79 | 109.32 |
JPDA_m_16 (Rezatofighi et al. 2015) | 26.17 | N/A | 76.34 | 0.62 | 31 | 512 | 3689 | 130,549 | 365 | 638 | 12.85 | 22.47 |
DP_NMS_16 (Pirsiavash et al. 2011) | 26.17 | 31.19 | 76.34 | 0.62 | 31 | 512 | 3689 | 130,557 | 365 | 638 | 12.86 | 22.47 |
Method | MOTA | IDF1 | MOTP | FAR | MT | ML | FP | FN | IDSW | FM | IDSWR | FMR |
---|---|---|---|---|---|---|---|---|---|---|---|---|
MPNTrack (Brasó and Leal-Taixé 2020) | 58.85 | 61.75 | 78.62 | 0.98 | 679 | 788 | 17,413 | 213,594 | 1185 | 2265 | 19.07 | 36.45 |
Tracktor++v2 (Bergmann et al. 2019) | 56.35 | 55.12 | 78.82 | 0.50 | 498 | 831 | 8866 | 235,449 | 1987 | 3763 | 34.10 | 64.58 |
TrctrD17 (Xu et al. 2020) | 53.72 | 53.77 | 77.23 | 0.66 | 458 | 861 | 11,731 | 247,447 | 1947 | 4792 | 34.68 | 85.35 |
Tracktor++ (Bergmann et al. 2019) | 53.51 | 52.33 | 77.98 | 0.69 | 459 | 861 | 12,201 | 248,047 | 2072 | 4611 | 36.98 | 82.28 |
JBNOT (Henschel et al. 2019) | 52.63 | 50.77 | 77.12 | 1.78 | 465 | 844 | 31,572 | 232,659 | 3050 | 3792 | 51.90 | 64.53 |
FAMNet (Chu and Ling 2019) | 52.00 | 48.71 | 76.48 | 0.80 | 450 | 787 | 14,138 | 253,616 | 3072 | 5318 | 55.80 | 96.60 |
eTC17 (Wang et al. 2019) | 51.93 | 58.13 | 76.34 | 2.04 | 544 | 836 | 36,164 | 232,783 | 2288 | 3071 | 38.95 | 52.28 |
eHAF17 (Sheng et al. 2018c) | 51.82 | 54.72 | 77.03 | 1.87 | 551 | 893 | 33,212 | 236,772 | 1834 | 2739 | 31.60 | 47.19 |
YOONKJ17 (Yoon et al. 2020) | 51.37 | 53.98 | 77.00 | 1.64 | 500 | 878 | 29,051 | 243,202 | 2118 | 3072 | 37.23 | 53.99 |
FWT_17 (Henschel et al. 2018) | 51.32 | 47.56 | 77.00 | 1.36 | 505 | 830 | 24,101 | 247,921 | 2648 | 4279 | 47.24 | 76.33 |
NOTA (Chen et al. 2019) | 51.27 | 54.46 | 76.68 | 1.13 | 403 | 833 | 20,148 | 252,531 | 2285 | 5798 | 41.36 | 104.95 |
JointMC (jCC) (Keuper et al. 2018) | 51.16 | 54.50 | 75.92 | 1.46 | 493 | 872 | 25,937 | 247,822 | 1802 | 2984 | 32.13 | 53.21 |
STRN_MOT17 (Xu et al. 2019) | 50.90 | 55.98 | 75.58 | 1.42 | 446 | 797 | 25,295 | 249,365 | 2397 | 9363 | 42.95 | 167.78 |
MOTDT17 (Long et al. 2018) | 50.85 | 52.70 | 76.58 | 1.36 | 413 | 841 | 24,069 | 250,768 | 2474 | 5317 | 44.53 | 95.71 |
MHT_DAM_17 (Kim et al. 2015) | 50.71 | 47.18 | 77.52 | 1.29 | 491 | 869 | 22,875 | 252,889 | 2314 | 2865 | 41.94 | 51.92 |
TLMHT_17 (Sheng et al. 2018a) | 50.61 | 56.51 | 77.65 | 1.25 | 415 | 1022 | 22,213 | 255,030 | 1407 | 2079 | 25.68 | 37.94 |
EDMT17 (Chen et al. 2017a) | 50.05 | 51.25 | 77.26 | 1.82 | 509 | 855 | 32,279 | 247,297 | 2264 | 3260 | 40.31 | 58.04 |
GMPHDOGM17 (Song et al. 2019) | 49.94 | 47.15 | 77.01 | 1.35 | 464 | 895 | 24,024 | 255,277 | 3125 | 3540 | 57.07 | 64.65 |
MTDF17 (Fu et al. 2019) | 49.58 | 45.22 | 75.48 | 2.09 | 444 | 779 | 37,124 | 241,768 | 5567 | 9260 | 97.41 | 162.03 |
PHD_GM (Sanchez-Matilla and Cavallaro 2019) | 48.84 | 43.15 | 76.74 | 1.48 | 449 | 830 | 26,260 | 257,971 | 4407 | 6448 | 81.19 | 118.79 |
OTCD_1_17 (Liu et al. 2019) | 48.57 | 47.90 | 76.91 | 1.04 | 382 | 970 | 18,499 | 268,204 | 3502 | 5588 | 66.75 | 106.51 |
HAM_SADF17 (Yoon et al. 2018a) | 48.27 | 51.14 | 77.22 | 1.18 | 402 | 981 | 20,967 | 269,038 | 1871 | 3020 | 35.76 | 57.72 |
DMAN (Zhu et al. 2018) | 48.24 | 55.69 | 75.69 | 1.48 | 454 | 902 | 26,218 | 263,608 | 2194 | 5378 | 41.18 | 100.94 |
AM_ADM17 (Lee et al. 2018) | 48.11 | 52.07 | 76.69 | 1.41 | 316 | 934 | 25,061 | 265,495 | 2214 | 5027 | 41.82 | 94.95 |
PHD_GSDL17 (Fu et al. 2018) | 48.04 | 49.63 | 77.15 | 1.31 | 402 | 838 | 23,199 | 265,954 | 3998 | 8886 | 75.63 | 168.09 |
MHT_bLSTM (Kim et al. 2018) | 47.52 | 51.92 | 77.49 | 1.46 | 429 | 981 | 25,981 | 268,042 | 2069 | 3124 | 39.41 | 59.51 |
MASS (Karunasekera et al. 2019) | 46.95 | 45.99 | 76.11 | 1.45 | 399 | 856 | 25,733 | 269,116 | 4478 | 11,994 | 85.62 | 229.31 |
GMPHD_Rd17 (Baisa 2019b) | 46.83 | 54.06 | 76.41 | 2.17 | 464 | 784 | 38,452 | 257,678 | 3865 | 8097 | 71.14 | 149.03 |
IOU17 (Bochinski et al. 2017) | 45.48 | 39.40 | 76.85 | 1.13 | 369 | 953 | 19,993 | 281,643 | 5988 | 7404 | 119.56 | 147.84 |
LM_NN_17 (Babaee et al. 2019) | 45.13 | 43.17 | 78.93 | 0.61 | 348 | 1088 | 10,834 | 296,451 | 2286 | 2463 | 48.17 | 51.90 |
FPSN (Lee and Kim 2019) | 44.91 | 48.43 | 76.61 | 1.90 | 388 | 844 | 33,757 | 269,952 | 7136 | 14,491 | 136.82 | 277.84 |
HISP_T17 (Baisa 2019c) | 44.62 | 38.79 | 77.19 | 1.43 | 355 | 913 | 25,478 | 276,395 | 10,617 | 7487 | 208.12 | 146.76 |
GMPHD_DAL (Baisa 2019a) | 44.40 | 36.23 | 77.42 | 1.08 | 350 | 927 | 19,170 | 283,380 | 11,137 | 13,900 | 223.74 | 279.25 |
SAS_MOT17 (Maksai and Fua 2019) | 44.24 | 57.18 | 76.42 | 1.66 | 379 | 1044 | 29,473 | 283,611 | 1529 | 2644 | 30.74 | 53.16 |
GMPHD_SHA (Song and Jeon 2016) | 43.72 | 39.17 | 76.53 | 1.46 | 276 | 1012 | 25,935 | 287,758 | 3838 | 5056 | 78.33 | 103.18 |
SORT17 (Bewley et al. 2016a) | 43.14 | 39.84 | 77.77 | 1.60 | 295 | 997 | 28,398 | 287,582 | 4852 | 7127 | 98.96 | 145.36 |
EAMTT_17 (Sanchez-Matilla et al. 2016) | 42.63 | 41.77 | 76.03 | 1.73 | 300 | 1006 | 30,711 | 288,474 | 4488 | 5720 | 91.83 | 117.04 |
GMPHD_N1Tr (Baisa and Wallace 2019) | 42.12 | 33.87 | 77.66 | 1.03 | 280 | 1005 | 18,214 | 297,646 | 10,698 | 10,864 | 226.43 | 229.94 |
GMPHD_KCF (Kutschbach et al. 2017) | 39.57 | 36.64 | 74.54 | 2.87 | 208 | 1019 | 50,903 | 284,228 | 5811 | 7414 | 117.10 | 149.40 |
GM_PHD (Eiselein et al. 2012) | 36.36 | 33.92 | 76.20 | 1.34 | 97 | 1349 | 23,723 | 330,767 | 4607 | 11,317 | 111.34 | 273.51 |
7 Analysis of State-of-the-Art Trackers
7.1 Trends in Tracking
Method | Box–box affinity | App. | Opt. | Extra inputs | OA | TR | ON |
---|---|---|---|---|---|---|---|
MPNTrack (Brasó and Leal-Taixé 2020) | Appearance, geometry (L) | ✓ | MCF, LP | MC | ✗ | ✓ | ✗ |
DeepMOT (Xu et al. 2020) | Re-id (L) | ✓ | – | MC | ✗ | ✓ | ✓ |
TT17 (Zhang et al. 2020) | Appearance, geometry (L) | ✓ | MHT/MWIS | ✗ | ✗ | ✗ | ✗ |
CRF_TRACK (Xiang et al. 2020) | Appearance, geometry (L) | ✓ | CRF | re-id | ✗ | ✗ | ✗ |
Tracktor (Bergmann et al. 2019) | Re-id (L) | ✓ | – | MC, re-id | ✗ | ✓ | ✓ |
KCF (Chu et al. 2019) | Re-id (L) | ✓ | Multicut | re-id | ✓ | ✓ | ✓ |
STRN (Xu et al. 2019) | Geometry, appearance (L) | ✓ | Hungarian algorithm | – | ✗ | ✗ | ✓ |
JBNOT (Henschel et al. 2019) | Joint, body distances | ✗ | Frank–Wolfe algortihm | Body joint det. | ✗ | ✗ | ✗ |
FAMNet (Chu and Ling 2019) | (L) | ✓ | Rank-1 tensor approx. | – | ✗ | ✓ | ✓ |
MHT_bLSTM (Kim et al. 2018) | Appearance, motion (L) | ✓ | MHT/MWIS | Pre-trained CNN | ✓ | ✗ | ✓ |
JointMC (Keuper et al. 2018) | DeepMatching (L), geometric | ✓ | Multicut | OF, non-nms dets | ✗ | ✗ | ✗ |
RAR (Fang et al. 2018) | Appearance, motion (L) | ✓ | Hungarian algorithm | – | ✗ | ✗ | ✓ |
HCC (Ma et al. 2018b) | Re-id (L) | ✓ | Multicut | External re-id | \(\circ \) | ✗ | ✗ |
FWT (Henschel et al. 2018) | DeepMatching, geometric | ✓ | Frank-Wolfe algorithm | Head detector | ✗ | ✗ | ✗ |
DMAN (Zhu et al. 2018) | Appearance (L), geometry | ✓ | – | – | ✓ | ✓ | ✓ |
eHAF (Zhu et al. 2018) | Appearance, motion | ✓ | MHT/MWIS | Super-pixels, OF | ✗ | ✗ | ✗ |
QuadMOT (Son et al. 2017) | Re-id (L), motion | ✓ | Min-max label prop. | – | ✓ | ✗ | ✓ |
STAM (Chu et al. 2017) | Appearance (L), motion | ✓ | – | – | ✓ | ✓ | ✓ |
AMIR (Sadeghian et al. 2017) | Motion, appearance, interactions (L) | ✓ | Hungarian algortihm | – | ✗ | ✗ | ✓ |
LMP (Tang et al. 2017) | Re-id (L) | ✓ | Multicut | Non-nms det., re-id | ✓ | ✗ | ✗ |
NLLMPa (Levinkov et al. 2017) | DeepMatching | ✓ | Multicut | Non-NMS dets | ✗ | ✗ | ✗ |
LP_SSVM (Wang and Fowlkes 2016) | Appearance, motion (L) | ✓ | MCF, greedy | – | ✗ | ✗ | ✗ |
SiameseCNN (Leal-Taixe et al. 2016) | Appearance (L), geometry, motion | ✓ | MCF, LP | OF | ✗ | ✗ | ✗ |
SCEA (Yoon et al. 2016) | Appearance, geometry | ✓ | Clustering | – | ✗ | ✗ | ✓ |
JMC (Tang et al. 2016) | DeepMatching | ✓ | Multicut | Non-NMS dets | ✗ | ✗ | ✗ |
LINF1 (Fagot-Bouquet et al. 2016) | Sparse representation | ✓ | MCMC | – | ✗ | ✗ | ✗ |
EAMTTpub (Sanchez-Matilla et al. 2016) | 2D distances | ✗ | Particle Filter | Non-NMS dets | ✗ | ✗ | ✓ |
OVBT (Ban et al. 2016) | Dynamics from flow | ✓ | Variational EM | OF | ✗ | ✗ | ✓ |
LTTSC-CRF (Le et al. 2016) | SURF | ✓ | CRF | SURF | ✗ | ✗ | ✗ |
GMPHD_HDA (Song and Jeon 2016) | HoG similarity, color histogram | ✓ | GM-PHD filter | HoG | ✗ | ✗ | ✓ |
DCO_X (Milan et al. 2016) | Motion, geometry | ✓ | CRF | – | ✗ | ✗ | ✗ |
ELP (McLaughlin et al. 2015) | Motion | ✗ | MCF, LP | – | ✗ | ✗ | ✗ |
GMMCP (Dehghan et al. 2015) | Appearance, motion | ✓ | GMMCP/CRF | – | ✗ | ✗ | ✗ |
MDP (Xiang et al. 2015) | Motion (flow), geometry, appearance | ✓ | Hungarian algorithm | OF | ✗ | ✗ | ✓ |
MHT_DAM (Kim et al. 2015) | (L) | ✓ | MHT/MWIS | – | ✓ | ✗ | ✓ |
NOMT (Choi 2015) | Interest point traj. | ✓ | CRF | OF | ✗ | ✗ | ✗ |
JPDA_m (Rezatofighi et al. 2015) | Mahalanobis distance | ✗ | LP | – | ✗ | ✗ | ✗ |
SegTrack (Milan et al. 2015) | Shape, geometry, motion | ✓ | CRF | OF, super-pixels | ✗ | ✗ | ✗ |
TBD (Geiger et al. 2014) | IoU + NCC | ✓ | Hungarian algorithm | – | ✗ | ✗ | ✗ |
CEM (Milan et al. 2014) | Motion | ✗ | Greedy sampling | – | ✗ | ✗ | ✗ |
MotiCon (Leal-Taixé et al. 2014) | Motion descriptors | ✗ | MCF, LP | OF | ✗ | ✗ | ✗ |
SMOT (Dicle et al. 2013) | Target dynamics | ✗ | Hankel Least Squares | – | ✗ | ✗ | ✗ |
DP_NMS (Pirsiavash et al. 2011) | 2D image distances | ✗ | k-shortest paths | – | ✗ | ✗ | ✗ |
LP2D (Leal-Taixé et al. 2011) | 2D image distances, IoU | ✗ | MCF, LP | – | ✗ | ✗ | ✗ |