Top

International Journal of Computer Vision

Published in:

17-03-2020

Semi-online Multi-people Tracking by Re-identification

Authors: Long Lan, Xinchao Wang, Gang Hua, Thomas S. Huang, Dacheng Tao

Published in: International Journal of Computer Vision | Issue 7/2020

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

In this paper, we propose a novel semi-online approach to tracking multiple people. In contrast to conventional offline approaches that take the whole image sequence as input, our semi-online approach tracks people in a frame-by-frame manner by exploring the time, space and multi-camera relationship of detection hypotheses in the near future frames. We cast the multi-people tracking task as a re-identification problem, and explicitly account for objects’ appearance changes and longer-term associations. We model our approach using a Multi-Label Markov Random Field, and introduce a fast \(\alpha \)-expansion algorithm to solve it efficiently. To our best knowledge, this is the first semi-online approach achieved by re-identification. It yields very promising tracking results especially in challenging cases, such as scenarios of the crowded streets where pedestrians frequently occlude each other, scenes captured with moving cameras where objects may disappear and reappear randomly, and videos under changing illuminations wherein the appearances of objects are influenced.

previous article MAP Inference Via -Sphere Linear Program Reformulation

next article The Open Images Dataset V4

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

https://motchallenge.net/results/MOT17/, accessed on Aug 10, 2019.

Arora, C., & Globerson, A. (2013). Higher order matching for consistent multiple target tracking. In ICCV (pp. 177–184).

Bae, S., & Yoon, K. (2014). Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning. In CVPR (pp. 1218–1225).

Bae, S., & Yoon, K. (2017). Confidence-based data association and discriminative deep appearance learning for robust online multi-object tracking. In TPAMI.

Benfold, B., & Reid, I. (2011). Stable multi-target tracking in real-time surveillance video. In CVPR (pp. 3457–3464).

Berclaz, J., Fleuret, F., Turetken, E., & Fua, P. (2011). Multiple object tracking using k-shortest paths optimization. TPAMI, 33(9), 1806–1819.CrossRef

Bergmann, P., Meinhardt, T., & Leal-Taixe, L. (2019). Tracking without bells and whistles. In ICCV.

Boros, E., & Hammer, P. (2002). Pseudo-Boolean optimization. Discrete Applied Mathematics, 123(1), 155–225.MathSciNetCrossRef

Boykov, Y., Veksler, O., & Zabih, R. (2001). Fast approximate energy minimization via graph cuts. TPAMI, 23(11), 1222–1239.CrossRef

Breitenstein, M., Reichlin, F., Leibe, B., Koller-Meier, E., & Van-Gool, L. (2011). Online multiperson tracking-by-detection from a single, uncalibrated camera. TPAMI, 33(9), 1820–1833.CrossRef

Brendel, W., Amer, M., & Todorovic, S. (2011). Multiobject tracking as maximum weight independent set. In ICCV (pp. 1273–1280).

Butt, A., & Collins, R. (2013). Multi-target tracking by Lagrangian relaxation to min-cost network flow. In CVPR (pp. 1846–1853).

Chan, A., Liang, Z., & Vasconcelos, N. (2008). Privacy preserving crowd monitoring: Counting people without people models or tracking. In CVPR (pp. 1–7).

Chari, V., Lacoste-Julien, S., Laptev, I., & Sivic, J. (2015). On pairwise costs for network flow multi-object tracking. In CVPR (pp. 5537–5545).

Cheng, D., Gong, Y., Zhou, S., Wang, J., & Zheng, N. (2016). Person re-identification by multi-channel parts-based CNN with improved triplet loss function. In CVPR (pp. 1335–1344).

Chen, D., Li, H., Xiao, T., Yi, S., & Wang, X. (2018). Video person re-identification with competitive snippet-similarity aggregation and co-attentive snippet embedding. CVPR (pp. 1169–1178).

Chen, J., Sheng, H., Zhang, Y., & Xiong, Z. (2017). Enhancing detection model for multiple hypothesis tracking. In CVPR workshop (pp. 2143–2152).

Chen, L., Ai, H., Zhuang, Z., & Shang, C. (2018). Real-time multiple people tracking with deeply learned candidate selection and person re-identification. ICME, 5, 8.

Chen, S., Fern, A., & Todorovic, S. (2014). Multi-object tracking via constrained sequential labeling. In CVPR (pp. 1130–1137).

Choi, W. (2015). Near-online multi-target tracking with aggregated local flow descriptor. In ICCV (pp. 3029–3037).

Chu, P., & Ling, H. (2019). Famnet: Joint learning of feature, affinity and multi-dimensional assignment for online multiple object tracking. In ICCV.

Collins, R., Liu, Y., & Leordeanu, M. (2005). Online selection of discriminative tracking features. TPAMI, 27(10), 1631–1643.CrossRef

Dehghan, A., Assari, S. M., & Shah, M. (2015). Gmmcp tracker: Globally optimal generalized maximum multi clique problem for multiple object tracking. In CVPR (pp. 4091–4099).

Dehghan, A., Tian, Y., Torr, P., & Shah, M. (2015). Target identity-aware network flow for online multiple target tracking. In CVPR (pp. 1146–1154).

Delong, A., Osokin, A., Isack, H., & Boykov, Y. (2012). Fast approximate energy minimization with label costs. IJCV, 96(1), 1–27.MathSciNetCrossRef

Fagot-Bouquet, L., Audigier, R., Dhome, Y., & Lerasle, F. (2016). Improving multi-frame data association with sparse representations for robust near-online multi-object tracking. In ECCV (pp. 774–790).

Fan, J., Shen, X., & Wu, Y. (2012). Scribble tracker: A matting-based approach for robust tracking. TPAMI, 34(8), 1633–1644.CrossRef

Fan, J., Shen, X., & Wu, Y. (2013). What are we tracking: A unified approach of tracking and recognition. TIP, 22(2), 549–560.MathSciNetMATH

Fleuret, F., Shitrit, H., & Fua, P. (2014). Re-identification for improved people tracking. In Person re-identification (pp. 309–330).

Fortmann, T., Yaakov, B., & Scheffe, M. (1983). Sonar tracking of multiple targets using joint probabilistic data association. JOE, 8(3), 173–184.

Fu, Y., Wei, Y., Zhou, Y., Shi, H., Huang, G., Wang, X., et al. (2019). Horizontal pyramid matching for person re-identification. In AAAI (pp. 8295–8302).

Galoogahi, H., Fagg, A., Huang, C., Ramanan, D., & Lucey, S. (2017). Need for speed: A benchmark for higher frame rate object tracking. arXiv preprint arXiv:1703.05884.

Geng, M., Wang, Y., Xiang, T., & Tian, Y. (2016). Deep transfer learning for person re-identification. arXiv preprint arXiv:1611.05244.

Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR (pp. 580–587).

Hamid-Rezatofighi, S., Milan, A., Zhang, Z., Shi, Q., Dick, A., & Reid, I. (2015). Joint probabilistic data association revisited. In ICCV (pp. 3047–3055).

Henschel, R., Zou, Y., & Rosenhahn, B. (2019). Multiple people tracking using body and joint detections. In CVPR Workshops.

Hofmann, M., Wolf, D., & Rigoll, G. (2013). Hypergraphs for joint multi-view reconstruction and multi-object tracking. In CVPR (pp. 3650–3657).

Hong-Yoon, J., Lee, C., Yang, M., & Yoon, K. (2016). Online multi-object tracking via structural constraint event aggregation. In CVPR (pp. 1392–1400).

http://www.cvg.reading.ac.uk/PETS2009/a.html.

https://motchallenge.net/results/MOT16/.

Hu, W., Li, X., Luo, W., Zhang, X., Maybank, S., & Zhang, Z. (2012). Single and multiple object tracking using log-Euclidean Riemannian subspace and block-division appearance model. TPAMI, 34(12), 2420–2440.CrossRef

Janai, J., Güney, F., Behl, A., & Geiger, A. (2017). Computer vision for autonomous vehicles: Problems, datasets and state-of-the-art. arXiv preprint arXiv:1704.05519.

Jerripothula, K., Cai, J., & Yuan, J. (2016). Cats: Co-saliency activated tracklet selection for video co-localization. In ECCV (pp. 187–202).

Keuper, M., Tang, S., Yu, Z., Andres, B., Brox, T., & Schiele, B. (2016). A multi-cut formulation for joint segmentation and tracking of multiple objects. arXiv preprint arXiv:1607.06317.

Keuper, M., Tang, S., Zhongjie, Y., Andres, B., Brox, T., & Schiele, B. (2016). A multi-cut formulation for joint segmentation and tracking of multiple objects. arXiv preprint arXiv:1607.06317.

Kim, C., Li, F., Ciptadi, A., & Rehg, J. (2015). Multiple hypothesis tracking revisited. In ICCV (pp. 4696–4704).

Kim, C., Li, F., & Rehg, J. M. (2018). Multi-object tracking with neural gating using bilinear lSTM. In ECCV (pp. 200–215).

Kuo, C., Huang, C., & Nevatia, R. (2010). Multi-target tracking by on-line learned discriminative appearance models. In CVPR (pp. 685–692).

Kuo, C., & Nevatia, R. (2011). How does person identity recognition help multi-person tracking? In CVPR (pp. 1217–1224).

Lafferty, J., McCallum, A., Pereira, F., et al. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. ICML, 1, 282–289.

Lan, L., Tao, D., Gong, C., Guan, N., & Luo, Z. (2016). Online multi-object tracking by quadratic pseudo-Boolean optimization. In IJCAI (pp. 3396–3402).

Lan, L., Wang, X., Zhang, S., Tao, D., Gao, W., & Huang, T. S. (2018). Interacting tracklets for multi-object tracking. In TIP.

Leal-Taixé, L., Canton-Ferrer, C., & Schindler, K. (2016). Learning by tracking: Siamese cnn for robust target association. In CVPR workshops (pp. 33–40).

Leal-Taixé, L., Pons-Moll, G., & Rosenhahn, B. (2012). Branch-and-price global optimization for multi-view multi-target tracking. In CVPR (pp. 1987–1994).

Lenz, P., Geiger, A., & Urtasun, R. (2015). Followme: Efficient online min-cost flow tracking with bounded memory and computation. In ICCV (pp. 4364–4372).

Levinkov, E., Uhrig, J., Tang, S., Omran, M., Insafutdinov, E., Kirillov, A., Rother, C., Brox, T., Schiele, B., & Andres, B. (2017). Joint graph decomposition and node labeling: Problem, algorithms, applications. In CVPR.

Liang, P., Blasch, E., & Ling, H. (2015). Encoding color information for visual tracking: Algorithms and benchmark. TIP, 24(12), 5630–5644.MathSciNetMATH

Liao, S., Hu, Y., Zhu, X., & Li, S. (2015). Person re-identification by local maximal occurrence representation and metric learning. In CVPR (pp. 2197–2206).

Liu, J., Carr, P., Collins, R., & Liu, Y. (2013). Tracking sports players with context-conditioned motion models. In CVPR (pp. 1830–1837).

Liwicki, S., Zafeiriou, S., Tzimiropoulos, G., & Pantic, M. (2012). Efficient online subspace learning with an indefinite kernel for visual tracking and recognition. In TNNLS (pp. 1624–1636).

Liwicki, S., Zafeiriou, S., & Pantic, M. (2015). Online kernel slow feature analysis for temporal video segmentation and tracking. TIP, 24(10), 2955–2970.MathSciNetMATH

Maksai, A., & Fua, P. (2019). Eliminating exposure bias and metric mismatch in multiple object tracking. In CVPR (pp. 4639–4648).

Maksai, A., Wang, X., & Fua, P. (2016). What players do with the ball: A physically constrained interaction modeling. In CVPR (pp. 972–981).

Maksai, A., Wang, X., Fleuret, F., & Fua, P. (2017). Non-Markovian globally consistent multi-object tracking. In ICCV (pp. 2563–2573).

McLaughlin, N., del Rincon, J. M., & Miller, P. (2016). Recurrent convolutional network for video-based person re-identification. In CVPR (pp. 1325–1334).

Milan, A., Leal-Taixé, L., Reid, I. D., Roth, S., & Schindler, K. (2016). Mot16: A benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831.

Milan, A., Leal-Taixé, L., Schindler, K., & Reid, I. (2015). Joint tracking and segmentation of multiple targets. In CVPR (pp. 5397–5406).

Milan, A., Roth, S., & Schindler, K. (2013). Continuous energy minimization for multitarget tracking. TPAMI, 36(1), 58–72.CrossRef

Milan, A., Roth, S., & Schindler, K. (2014). Continuous energy minimization for multitarget tracking. TPAMI, 36(1), 58–72.CrossRef

Milan, A., Schindler, K., & Roth, S. (2016). Multi-target tracking by discrete-continuous energy minimization. TPAMI, 38(10), 2054–2068.CrossRef

Nillius, P., Sullivan, J., & Carlsson, S. (2006). Multi-target tracking—linking identities using bayesian network inference. In ECCV (pp. 2187–2194).

Pang, Y., Shi, X., Jia, B., Blasch, E., Sheaff, C., Pham, K., et al. (2015). Multiway histogram intersection for multi-target tracking. In ICIF (pp. 1938–1945).

Pirsiavash, H., Ramanan, D., & Fowlkes, C. (2011). Globally-optimal greedy algorithms for tracking a variable number of objects. In CVPR (pp. 1201–1208).

Possegger, H., Mauthner, T., Roth, P., & Bischof, H. (2014). Occlusion geodesics for online multi-object tracking. In CVPR (pp. 1306–1313).

Qiu, J., Wang, X., Fua, P., & Tao, D. (2020). Matching seqlets: An unsupervised approach for locality preserving sequence matching. In TPAMI.

Qiu, J., Wang, X., Maybank, S., & Tao, D. (2019). World from blur. In CVPR (pp. 8493–8504).

Ramanan, D., Forsyth, D., & Zisserman, A. (2006). Tracking people by learning their appearance. TPAMI, 29(1), 65–81.CrossRef

Ristani, E., & Tomasi, C. (2018). Features for multi-target multi-camera tracking and re-identification. In CVPR (pp. 6036–6046).

Roberto, H., Leal-Taixé, L., & Bodo, R. (2017). Fusion of head and full-body detectors for multi-object tracking. arXiv preprint arXiv:1705.08314.

Sadeghian, A., Alahi, A., & Savarese, S. (2017). Tracking the untrackable: Learning to track multiple cues with long-term dependencies. In ICCV (pp. 300–311).

Sanchez-Matilla, R., Poiesi, F., & Cavallaro, A. (2016). Online multi-target tracking with strong and weak detections. In ECCV (pp. 84–99).

Shen, C., Wang, X., Song, J., Sun, L., & Song, M. (2019). Amalgamating knowledge towards comprehensive classification. In AAAI (pp. 3068–3075).

Shitrit, H. B., Berclaz, J., Fleuret, F., & Fua, P. (2014). Multi-commodity network flow for tracking multiple people. TPAMI, 36(8), 1614–1627.CrossRef

Shu, G., Dehghan, A., Oreifej, O., Hand, E., & Shah, M. (2012). Part-based multiple-person tracking with partial occlusion handling. In CVPR (pp. 1815–1821).

Song, J., Chen, Y., Wang, X., Shen, C., & Song, M. (2019). Deep model transferability from attribution maps. In NeurIPS (pp. 6179–6189).

Su, C., Zhang, S., Xing, J., Gao, W., & Tian, Q. (2016). Deep attributes driven multi-camera person re-identification. In ECCV (pp. 475–491).

Sullivan, J., & Carlsson, S. (2006). Tracking and labelling of interacting multiple targets. In ECCV (pp. 619–632).

Tang, S., Andres, B., Andriluka, M., & Schiele, B. (2015). Subgraph decomposition for multi-target tracking. In CVPR (pp. 5033–5041).

Tang, S., Andres, B., Andriluka, M., & Schiele, B. (2016). Multi-person tracking by multicut and deep matching. In ECCV workshops (pp. 100–111).

Tang, S., Andriluka, M., Andres, B., & Schiele, B. (2017). Multiple people tracking by lifted multicut and person reidentification. In CVPR (pp. 3539–3548).

Tsai, D., Flagg, M., Nakazawa, A., & Rehg, J. (2012). Motion coherent tracking using multi-label MRF optimization. IJCV, 100(2), 190–202.MathSciNetCrossRef

Turetken, E., Wang, X., Becker, C., Haubold, C., & Fua, P. (2017). Network flow integer programming to track elliptical cells in time-lapse sequences. TMI, 36(4), 942–951.

Wang, X., Fan, B., Chang, S., Wang, Z., Liu, X., Tao, D., & Huang, T. (2017). Greedy batch-based minimum-cost flows for tracking multiple objects. In TIP.

Wang, J., Huang, S., Wang, X., & Tao, D. (2019). Not all parts are created equal: 3D pose estimation by modelling bi-directional dependencies of body parts. In ICCV.

Wang, X., Türetken, E., Fleuret, F., & Fua, P. (2014). Tracking interacting objects optimally using integer programming. In ECCV (pp. 17–32).

Wang, H., Ullah, M., Klaser, A., Laptev, I., & Schmid, C. (2009). Evaluation of local spatio-temporal features for action recognition. In BMVC (pp. 124–1).

Wang, X., Ablavsky, V., Shitrit, H., & Fua, P. (2014). Take your eyes off the ball: Improving ball-tracking by focusing on team play. CVIU, 119, 102–115.

Wang, T., Gong, S., Zhu, X., & Wang, S. (2016). Person re-identification by discriminative selection in video ranking. TPAMI, 38(12), 2501–2514.CrossRef

Wang, X., Li, Z., & Tao, D. (2011). Subspaces indexing model on grassmann manifold for image search. TIP, 20(9), 2627–2635.MathSciNetMATH

Wang, X., Turetken, E., Fleuret, F., & Fua, P. (2016). Tracking interacting objects using intertwined flows. TPAMI, 38(11), 2312–2326.CrossRef

Weinzaepfel, P., Revaud, J., Harchaoui, Z., & Schmid, C. (2013). Deepflow: Large displacement optical flow with deep matching. In ICCV (pp. 1385–1392).

Wen, L., Lei, Z., Chang, M., Qi, H., & Lyu, S. (2016). Multi-camera multi-target tracking with space-time-view hyper-graph. In IJCV (pp. 1–21).

Wen, L., Lei, Z., Lyu, S., Li, S., & Yang, M. (2016). Exploiting hierarchical dense structures on hypergraphs for multi-object tracking. TPAMI, 38(10), 1983–1996.CrossRef

Wu, Z., Thangali, A., Sclaroff, S., & Betke, M. (2012). Coupling detection and data association for multiple object tracking. In CVPR (pp. 1948–1955).

Wu, Z., & Betke, M. (2016). Global optimization for coupled detection and data association in multiple object tracking. CVIU, 143, 25–37.

Xiang, Y., Alahi, A., & Savarese, S. (2015). Learning to track: Online multi-object tracking by decision making. In ICCV (pp. 4705–4713).

Yang, B., & Nevatia, R. (2012). An online learned CRF model for multi-target tracking. In CVPR (pp. 2034–2041).

Yang, M., Yuan, J., & Wu, Y. (2007). Spatial selection for attentional visual tracking. In CVPR (pp. 1–8).

Ye, J., Ji, Y., Wang, X., Ou, K., Tao, D., & Song, M. (2019). Student becoming the master: Knowledge amalgamation for joint scene parsing, depth estimation, and more. In CVPR (pp. 2829–2838).

Yin, X., Wang, X., Yu, J., Zhang, M., Fua, P., & Tao, D. (2018). Fisheyerecnet: A multi-context collaborative deep network for fisheye image rectification. In ECCV (pp. 475–490).

Yoon, J., Yang, M., Lim, J., & Yoon, K. (2015). Bayesian multi-object tracking using motion context from multiple objects. In WACV (pp. 33–40).

Yu, S., Yang, Y., Li, X., & Hauptmann, A. (2016). Long-term identity-aware multi-person tracking for surveillance video summarization. arXiv preprint arXiv:1604.07468.

Yu, X., Liu, T., Wang, X., & Tao, D. (2017). On compressing deep models by low rank and sparse decomposition. In CVPR (pp. 67–76).

Zhai, M., Roshtkhari, M., & Mori, G. (2016). Deep learning of appearance models for online object tracking. arXiv preprint arXiv:1607.02568.

Zhang, J., Wang, N., & Zhang, L. (2017). Multi-shot pedestrian re-identification via sequential decision making. arXiv preprint arXiv:1712.07257.

Zhang, L., Li, Y., & Nevatia, R. (2008). Global data association for multi-object tracking using network flows. In CVPR (pp. 1–8).

Zheng, L., Bie, Z., Sun, Y., Wang, J., Su, C., Wang, S., et al. (2016). Mars: A video benchmark for large-scale person re-identification. In ECCV (pp. 868–884).

Zheng, W., Gong, S., & Xiang, T. (2016). Towards open-world person re-identification by one-shot group-based verification. TPAMI, 38(3), 591–606.CrossRef

Zhu, J., Yang, H., Liu, N., Kim, M., Zhang, W., & Yang, M. (2018). Online multi-object tracking with dual matching attention networks. In ECCV (pp. 366–382).

Title: Semi-online Multi-people Tracking by Re-identification
Authors: Long Lan
Xinchao Wang
Gang Hua
Thomas S. Huang
Dacheng Tao
Publication date: 17-03-2020
Publisher: Springer US
Published in: International Journal of Computer Vision / Issue 7/2020
Print ISSN: 0920-5691
Electronic ISSN: 1573-1405
DOI: https://doi.org/10.1007/s11263-020-01314-1

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 7/2020

Deep Image Prior

Enhanced Balanced Min Cut

Light Structure from Pin Motion: Geometric Point Light Source Calibration

Exploiting Semantics for Face Image Deblurring

MAP Inference Via -Sphere Linear Program Reformulation

Is There Anything New to Say About SIFT Matching?

Premium Partner