Skip to main content
Erschienen in: Neural Computing and Applications 20/2022

14.06.2022 | Original Article

Similarity based person re-identification for multi-object tracking using deep Siamese network

verfasst von: Harun Suljagic, Ertugrul Bayraktar, Numan Celebi

Erschienen in: Neural Computing and Applications | Ausgabe 20/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The process of object tracking involves consistently identifying each instance across frames depending on initial set of object detection(s). Moreover, in multiple object tracking (MOT), the process through tracking-by-detection paradigm consists of performing two common steps consecutively, which are detection and data association. In MOT, it is targeted to associate detections across frames by localizing and identifying all objects of interest. MOT algorithms further keep tracking even the most challenging issues such as revisiting the same view, missing detections, occlusion and temporarily unseen objects, same-appearance objects coexisting in the same frame occur. Hence, re-identification (re-id) appears to be the most powerful tool for assigning the correct identities to each individual instance when aforementioned issues arise. In this work, we propose a similarity-based person re-id framework, called SAT, using a Siamese neural network via shared weights. Once detections are obtained from the backbone SAT applies a Siamese feature extraction model and then we introduce a similarity array for assessing tracklet(s) and detection(s). We examine the performance of SAT on several benchmarks with extensive experiments and statistical tests, where we improve the current state-of-the-art according to commonly used performance metrics with higher accuracy, less ID switches, less false positive and negative rates.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Zhang Y et al (2020) Multiplex labeling graph for near-online tracking in crowded scenes. IEEE Internet Things J 7:7892–7902CrossRef Zhang Y et al (2020) Multiplex labeling graph for near-online tracking in crowded scenes. IEEE Internet Things J 7:7892–7902CrossRef
2.
Zurück zum Zitat Yoon Y, Kim D, Song Y, Yoon K, Jeon M (2021) Online multiple pedestrians tracking using deep temporal appearance matching association. Inf Sci 561:326–351MathSciNetCrossRef Yoon Y, Kim D, Song Y, Yoon K, Jeon M (2021) Online multiple pedestrians tracking using deep temporal appearance matching association. Inf Sci 561:326–351MathSciNetCrossRef
3.
Zurück zum Zitat Cakir S, Cetin A (2021) Visual object tracking using Fourier domain phase information. Signal Image Video Process 16:119–126CrossRef Cakir S, Cetin A (2021) Visual object tracking using Fourier domain phase information. Signal Image Video Process 16:119–126CrossRef
4.
Zurück zum Zitat Braso G, Lear-Taixe L (2020) Learning a neural solver for multiple object tracking. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 6246–6256 Braso G, Lear-Taixe L (2020) Learning a neural solver for multiple object tracking. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 6246–6256
5.
Zurück zum Zitat Wojke N, Bewley A, Paulus D (2018) Simple online and realtime tracking with a deep association metric. In: Proceedings of international conference on image processing, ICIP, pp 3645–3649 Wojke N, Bewley A, Paulus D (2018) Simple online and realtime tracking with a deep association metric. In: Proceedings of international conference on image processing, ICIP, pp 3645–3649
6.
Zurück zum Zitat Chen L, Ai H, Chen R, Zhuang Z (2019) Aggregate tracklet appearance features for multi-object tracking. IEEE Signal Process. Lett. 26:1613–1617CrossRef Chen L, Ai H, Chen R, Zhuang Z (2019) Aggregate tracklet appearance features for multi-object tracking. IEEE Signal Process. Lett. 26:1613–1617CrossRef
7.
Zurück zum Zitat Wu Y et al (2019) Instance-aware representation learning and association for online multi-person tracking. Pattern Recognit. 94:25–34CrossRef Wu Y et al (2019) Instance-aware representation learning and association for online multi-person tracking. Pattern Recognit. 94:25–34CrossRef
8.
Zurück zum Zitat Ciaparrone G, Luque F, Sanchey L, Tabik S et al (2020) Deep learning in video multi-object tracking: a survey. Neurocomputing 381:61–88CrossRef Ciaparrone G, Luque F, Sanchey L, Tabik S et al (2020) Deep learning in video multi-object tracking: a survey. Neurocomputing 381:61–88CrossRef
9.
Zurück zum Zitat Yang F, Chang X, Sakti S, Wu Y, Nakamura S (2021) Remot: a model-agnostic refinement for multiple object tracking. Image Vis Comput 106:104091CrossRef Yang F, Chang X, Sakti S, Wu Y, Nakamura S (2021) Remot: a model-agnostic refinement for multiple object tracking. Image Vis Comput 106:104091CrossRef
10.
Zurück zum Zitat Liu Q, Chu Q, Liu B, Yu N (2020) Gsm: graph similarity model for multi-object tracking. In: Proceedings of the twenty-ninth international joint conference on artificial intelligence, pp 530–536 Liu Q, Chu Q, Liu B, Yu N (2020) Gsm: graph similarity model for multi-object tracking. In: Proceedings of the twenty-ninth international joint conference on artificial intelligence, pp 530–536
11.
Zurück zum Zitat Xu Y, Cao Y, Zhang Z (2019) Spatial-temporal relation networks for multi-object tracking. In: Proceedings of the IEEE international conference on computer vision, pp 3987–3997 Xu Y, Cao Y, Zhang Z (2019) Spatial-temporal relation networks for multi-object tracking. In: Proceedings of the IEEE international conference on computer vision, pp 3987–3997
12.
Zurück zum Zitat Sadeghian A, Alahi A, Saverse S (2017) Tracking the untrackable: learning to track multiple cues with long-term dependencies. In: Proceedings of the IEEE international conference on computer vision, pp 300–311 Sadeghian A, Alahi A, Saverse S (2017) Tracking the untrackable: learning to track multiple cues with long-term dependencies. In: Proceedings of the IEEE international conference on computer vision, pp 300–311
13.
Zurück zum Zitat Xu Y, Osep A, Ban Y, Horaud R (2020) How to train your deep multi-object tracker. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 6786–6795 Xu Y, Osep A, Ban Y, Horaud R (2020) How to train your deep multi-object tracker. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 6786–6795
14.
Zurück zum Zitat Chu Q et al (2017) Online multi-object tracking using cnn-based single object tracker with spatial-temporal attention mechanism. In: Proceedings of the IEEE international conference on computer vision, pp 4846–4855 Chu Q et al (2017) Online multi-object tracking using cnn-based single object tracker with spatial-temporal attention mechanism. In: Proceedings of the IEEE international conference on computer vision, pp 4846–4855
15.
Zurück zum Zitat Yang M, Wu Y, Jia Y (2017) A hybrid data association framework for robust online multi-object tracking. IEEE Trans Image Process 26:5667–5679MathSciNetCrossRef Yang M, Wu Y, Jia Y (2017) A hybrid data association framework for robust online multi-object tracking. IEEE Trans Image Process 26:5667–5679MathSciNetCrossRef
16.
Zurück zum Zitat Leal-Taixé L, Milan A, Reid I, Roth S, Schindler K (2015) Motchallenge 2015: towards a benchmark for multi-target tracking. arXiv:1504.01942 Leal-Taixé L, Milan A, Reid I, Roth S, Schindler K (2015) Motchallenge 2015: towards a benchmark for multi-target tracking. arXiv:​1504.​01942
17.
19.
Zurück zum Zitat Geiger A, Lenz P, Urtasun R (2012) Are we ready for autonomous driving? The kitti vision benchmark suite Geiger A, Lenz P, Urtasun R (2012) Are we ready for autonomous driving? The kitti vision benchmark suite
20.
Zurück zum Zitat Wang T, Gong S, Zhu X, Wang S (2014) Person re-identification by video ranking. Springer, Berlin, pp 688–703 Wang T, Gong S, Zhu X, Wang S (2014) Person re-identification by video ranking. Springer, Berlin, pp 688–703
21.
22.
Zurück zum Zitat Chavdarova T et al (2018) Wildtrack: a multi-camera hd dataset for dense unscripted pedestrian detection, pp 5030–5039 Chavdarova T et al (2018) Wildtrack: a multi-camera hd dataset for dense unscripted pedestrian detection, pp 5030–5039
23.
Zurück zum Zitat Li M, Zhu X, Gong S (2019) Unsupervised tracklet person re-identification. IEEE Trans Pattern Anal Mach Intell 42(7):1770–1782CrossRef Li M, Zhu X, Gong S (2019) Unsupervised tracklet person re-identification. IEEE Trans Pattern Anal Mach Intell 42(7):1770–1782CrossRef
24.
Zurück zum Zitat Luiten J et al (2020) Hota: a higher order metric for evaluating multi-object tracking. Int J Comput Vis: IJCV 129:548–578CrossRef Luiten J et al (2020) Hota: a higher order metric for evaluating multi-object tracking. Int J Comput Vis: IJCV 129:548–578CrossRef
25.
Zurück zum Zitat Fabbri M et al (2021) Motsynth: how can synthetic data help pedestrian detection and tracking?, pp 10849–10859 Fabbri M et al (2021) Motsynth: how can synthetic data help pedestrian detection and tracking?, pp 10849–10859
26.
Zurück zum Zitat Peng J et al (2020) Tpm: multiple object tracking with tracklet-plane matching. Pattern Recogn 107:107480CrossRef Peng J et al (2020) Tpm: multiple object tracking with tracklet-plane matching. Pattern Recogn 107:107480CrossRef
28.
Zurück zum Zitat Nousi P, Triantafyllidou D, Tefas A, Pitas I (2020) Re-identification framework for long term visual object tracking based on object detection and classification. Signal Process Image Commun 88:115969CrossRef Nousi P, Triantafyllidou D, Tefas A, Pitas I (2020) Re-identification framework for long term visual object tracking based on object detection and classification. Signal Process Image Commun 88:115969CrossRef
30.
Zurück zum Zitat Yu T, Li D, Yang Y, Timothy H, Xiang T (2019) Robust person re-identification by modelling feature uncertainty. In: Proceedings of the IEEE international conference on computer vision, pp 552–561 Yu T, Li D, Yang Y, Timothy H, Xiang T (2019) Robust person re-identification by modelling feature uncertainty. In: Proceedings of the IEEE international conference on computer vision, pp 552–561
31.
Zurück zum Zitat Chen A, Biglari-Abhari M, Wang K (2019) Investigating fast re-identification for multi-camera indoor person tracking. Comput Electr Eng 77:273–288CrossRef Chen A, Biglari-Abhari M, Wang K (2019) Investigating fast re-identification for multi-camera indoor person tracking. Comput Electr Eng 77:273–288CrossRef
32.
Zurück zum Zitat Li Y, Liu L, Zhu L, Zhang H (2021) Person re-identification based on multi-scale feature learning. Knowl Based Syst 228:107281CrossRef Li Y, Liu L, Zhu L, Zhang H (2021) Person re-identification based on multi-scale feature learning. Knowl Based Syst 228:107281CrossRef
33.
Zurück zum Zitat Lin Y, Xie L, Wu Y, Yan C, Tian Q (2020) Unsupervised person re-identification via softened similarity learning. CoRR arXiv:2004.03547 Lin Y, Xie L, Wu Y, Yan C, Tian Q (2020) Unsupervised person re-identification via softened similarity learning. CoRR arXiv:​2004.​03547
34.
Zurück zum Zitat Mansouri N, Ammar S, Kessentini Y (2021) Re-ranking person re-identification using attributes learning. Neural Comput Appl 33:12827–12843CrossRef Mansouri N, Ammar S, Kessentini Y (2021) Re-ranking person re-identification using attributes learning. Neural Comput Appl 33:12827–12843CrossRef
35.
Zurück zum Zitat Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: a benchmark. In: Proceedings of the IEEE international conference on computer vision, pp 1116–1124 Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: a benchmark. In: Proceedings of the IEEE international conference on computer vision, pp 1116–1124
36.
Zurück zum Zitat Ristani E, Solera F, Zou RS, Cucchiara R, Tomasi C (2016) Performance measures and a data set for multi-target, multi-camera tracking. In: European conference on computer vision. Springer, Cham, pp 17–35 Ristani E, Solera F, Zou RS, Cucchiara R, Tomasi C (2016) Performance measures and a data set for multi-target, multi-camera tracking. In: European conference on computer vision. Springer, Cham, pp 17–35
37.
Zurück zum Zitat Liao L et al (2020) A half-precision compressive sensing framework for end-to-end person re-identification. Neural Comput Appl 32(4):1141–1155CrossRef Liao L et al (2020) A half-precision compressive sensing framework for end-to-end person re-identification. Neural Comput Appl 32(4):1141–1155CrossRef
39.
Zurück zum Zitat Zhou S, Wang Y, Zhang F, Wu J (2021) Cross-view similarity exploration for unsupervised cross-domain person re-identification. Neural Comput Appl 33(9):4001–4011CrossRef Zhou S, Wang Y, Zhang F, Wu J (2021) Cross-view similarity exploration for unsupervised cross-domain person re-identification. Neural Comput Appl 33(9):4001–4011CrossRef
40.
Zurück zum Zitat Zhu X, Jing X-Y, Ma F, Cheng L, Ren Y (2019) Simultaneous visual-appearance-level and spatial-temporal-level dictionary learning for video-based person re-identification. Neural Comput Appl 31(11):7303–7315CrossRef Zhu X, Jing X-Y, Ma F, Cheng L, Ren Y (2019) Simultaneous visual-appearance-level and spatial-temporal-level dictionary learning for video-based person re-identification. Neural Comput Appl 31(11):7303–7315CrossRef
41.
Zurück zum Zitat Hirzer M, Beleznai C, Roth PM, Bischof H (2011) Person re-identification by descriptive and discriminative classification. In: Scandinavian conference on image analysis. Springer, Berlin, Heidelberg, pp 91–102 Hirzer M, Beleznai C, Roth PM, Bischof H (2011) Person re-identification by descriptive and discriminative classification. In: Scandinavian conference on image analysis. Springer, Berlin, Heidelberg, pp 91–102
43.
44.
Zurück zum Zitat Meinhardt T, Kirillov A, Leal-Taixé L, Feichtenhofer C (2021) Trackformer: Multi-object tracking with transformers. CoRR arXiv:2101.02702 Meinhardt T, Kirillov A, Leal-Taixé L, Feichtenhofer C (2021) Trackformer: Multi-object tracking with transformers. CoRR arXiv:​2101.​02702
45.
47.
Zurück zum Zitat Yang K, Song H, Zhang K, Liu Q (2020) Hierarchical attentive siamese network for real-time visual tracking. Neural Comput Appl 32(18):14335–14346CrossRef Yang K, Song H, Zhang K, Liu Q (2020) Hierarchical attentive siamese network for real-time visual tracking. Neural Comput Appl 32(18):14335–14346CrossRef
48.
Zurück zum Zitat Wu Y, Lim J, Yang MH (2013) Online object tracking: a benchmark. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2411–2418 Wu Y, Lim J, Yang MH (2013) Online object tracking: a benchmark. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2411–2418
49.
Zurück zum Zitat Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708 Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708
54.
Zurück zum Zitat Bayraktar E, Boyraz P (2017) Analysis of feature detector and descriptor combinations with a localization experiment for various performance metrics. Turki J Electr Eng Comput Sci 25(3):2444–2454CrossRef Bayraktar E, Boyraz P (2017) Analysis of feature detector and descriptor combinations with a localization experiment for various performance metrics. Turki J Electr Eng Comput Sci 25(3):2444–2454CrossRef
55.
Zurück zum Zitat Bayraktar E, Basarkan ME, Celebi N (2020) A low-cost uav framework towards ornamental plant detection and counting in the wild. ISPRS J Photogramm Remote Sens 167:1–11CrossRef Bayraktar E, Basarkan ME, Celebi N (2020) A low-cost uav framework towards ornamental plant detection and counting in the wild. ISPRS J Photogramm Remote Sens 167:1–11CrossRef
60.
Zurück zum Zitat Ciaparrone G et al (2020) Deep learning in video multi-object tracking: a survey. Neurocomputing 381:61–88CrossRef Ciaparrone G et al (2020) Deep learning in video multi-object tracking: a survey. Neurocomputing 381:61–88CrossRef
61.
Zurück zum Zitat Khalkhali MB, Vahedian A, Yazdi HS (2019) Multi-target state estimation using interactive kalman filter for multi-vehicle tracking. IEEE Trans Intell Transp Syst 21(3):1131–1144CrossRef Khalkhali MB, Vahedian A, Yazdi HS (2019) Multi-target state estimation using interactive kalman filter for multi-vehicle tracking. IEEE Trans Intell Transp Syst 21(3):1131–1144CrossRef
62.
Zurück zum Zitat Li X, Wang K, Wang W, Li Y (2010) A multiple object tracking method using kalman filter. Piscataway, IEEE, pp 1862–1866 Li X, Wang K, Wang W, Li Y (2010) A multiple object tracking method using kalman filter. Piscataway, IEEE, pp 1862–1866
63.
Zurück zum Zitat Arulampalam MS, Maskell S, Gordon N, Clapp T (2002) A tutorial on particle filters for online nonlinear/non-gaussian bayesian tracking. IEEE Trans Signal Process 50(2):174–188CrossRef Arulampalam MS, Maskell S, Gordon N, Clapp T (2002) A tutorial on particle filters for online nonlinear/non-gaussian bayesian tracking. IEEE Trans Signal Process 50(2):174–188CrossRef
64.
Zurück zum Zitat Smal I, Draegestein K, Galjart N, Niessen W, Meijering E (2008) Particle filtering for multiple object tracking in dynamic fluorescence microscopy images: application to microtubule growth analysis. IEEE Trans Med Imaging 27(6):789–804CrossRef Smal I, Draegestein K, Galjart N, Niessen W, Meijering E (2008) Particle filtering for multiple object tracking in dynamic fluorescence microscopy images: application to microtubule growth analysis. IEEE Trans Med Imaging 27(6):789–804CrossRef
65.
Zurück zum Zitat Cui Y, Zhang J, He Z, Hu J (2019) Multiple pedestrian tracking by combining particle filter and network flow model. Neurocomputing 351:217–227CrossRef Cui Y, Zhang J, He Z, Hu J (2019) Multiple pedestrian tracking by combining particle filter and network flow model. Neurocomputing 351:217–227CrossRef
66.
68.
Zurück zum Zitat Xu Y, Osep A, Ban Y, Horaud R, Leal-Taixé L, Alameda-Pineda X (2020) How to train your deep multi-object tracker. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6787–6796 Xu Y, Osep A, Ban Y, Horaud R, Leal-Taixé L, Alameda-Pineda X (2020) How to train your deep multi-object tracker. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6787–6796
70.
Zurück zum Zitat Papakis I, Sarkar A, Karpatne A (2020) Gcnnmatch: graph convolutional neural networks for multi-object tracking via sinkhorn normalization. CoRR arXiv:2010.00067 Papakis I, Sarkar A, Karpatne A (2020) Gcnnmatch: graph convolutional neural networks for multi-object tracking via sinkhorn normalization. CoRR arXiv:​2010.​00067
71.
Zurück zum Zitat Wang G, Wang Y, Gu R, Hu W, Hwang J (2021) Split and connect: a universal tracklet booster for multi-object tracking. CoRR arXiv:2105.02426 Wang G, Wang Y, Gu R, Hu W, Hwang J (2021) Split and connect: a universal tracklet booster for multi-object tracking. CoRR arXiv:​2105.​02426
73.
Zurück zum Zitat Smeulders AW et al (2013) Visual tracking: an experimental survey. IEEE Trans Pattern Anal Mach Intell 36(7):1442–1468 Smeulders AW et al (2013) Visual tracking: an experimental survey. IEEE Trans Pattern Anal Mach Intell 36(7):1442–1468
75.
Zurück zum Zitat Kaplan EL, Meier P (1958) Nonparametric estimation from incomplete observations. J Am Stat Assoc 53(282):457–481MathSciNetCrossRef Kaplan EL, Meier P (1958) Nonparametric estimation from incomplete observations. J Am Stat Assoc 53(282):457–481MathSciNetCrossRef
76.
Zurück zum Zitat Luiten J et al (2021) Hota: a higher order metric for evaluating multi-object tracking. Int J Comput Vis 129(2):548–578CrossRef Luiten J et al (2021) Hota: a higher order metric for evaluating multi-object tracking. Int J Comput Vis 129(2):548–578CrossRef
Metadaten
Titel
Similarity based person re-identification for multi-object tracking using deep Siamese network
verfasst von
Harun Suljagic
Ertugrul Bayraktar
Numan Celebi
Publikationsdatum
14.06.2022
Verlag
Springer London
Erschienen in
Neural Computing and Applications / Ausgabe 20/2022
Print ISSN: 0941-0643
Elektronische ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-022-07456-2

Weitere Artikel der Ausgabe 20/2022

Neural Computing and Applications 20/2022 Zur Ausgabe

Premium Partner