Skip to main content

2016 | OriginalPaper | Buchkapitel

Fully-Convolutional Siamese Networks for Object Tracking

verfasst von : Luca Bertinetto, Jack Valmadre, João F. Henriques, Andrea Vedaldi, Philip H. S. Torr

Erschienen in: Computer Vision – ECCV 2016 Workshops

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The problem of arbitrary object tracking has traditionally been tackled by learning a model of the object’s appearance exclusively online, using as sole training data the video itself. Despite the success of these methods, their online-only approach inherently limits the richness of the model they can learn. Recently, several attempts have been made to exploit the expressive power of deep convolutional networks. However, when the object to track is not known beforehand, it is necessary to perform Stochastic Gradient Descent online to adapt the weights of the network, severely compromising the speed of the system. In this paper we equip a basic tracking algorithm with a novel fully-convolutional Siamese network trained end-to-end on the ILSVRC15 dataset for object detection in video. Our tracker operates at frame-rates beyond real-time and, despite its extreme simplicity, achieves state-of-the-art performance in multiple benchmarks.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Smeulders, A.W.M., Chu, D.M., Cucchiara, R., Calderara, S., Dehghan, A., Shah, M.: Visual tracking: an experimental survey. PAMI 36(7), 1442–1468 (2014)CrossRef Smeulders, A.W.M., Chu, D.M., Cucchiara, R., Calderara, S., Dehghan, A., Shah, M.: Visual tracking: an experimental survey. PAMI 36(7), 1442–1468 (2014)CrossRef
2.
Zurück zum Zitat Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. PAMI 34(7), 1409–1422 (2012)CrossRef Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. PAMI 34(7), 1409–1422 (2012)CrossRef
3.
Zurück zum Zitat Hare, S., Saffari, A., Torr, P.H.S.: Struck: Structured output tracking with kernels. In: ICCV 2011. IEEE (2011) Hare, S., Saffari, A., Torr, P.H.S.: Struck: Structured output tracking with kernels. In: ICCV 2011. IEEE (2011)
4.
Zurück zum Zitat Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. PAMI 37(3), 583–596 (2015)CrossRef Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. PAMI 37(3), 583–596 (2015)CrossRef
5.
Zurück zum Zitat Ma, C., Huang, J.B., Yang, X., Yang, M.H.: Hierarchical convolutional features for visual tracking. In: ICCV (2015) Ma, C., Huang, J.B., Yang, X., Yang, M.H.: Hierarchical convolutional features for visual tracking. In: ICCV (2015)
6.
Zurück zum Zitat Danelljan, M., Hager, G., Khan, F., Felsberg, M.: Convolutional features for correlation filter based visual tracking. In: ICCV 2015 Workshop, pp. 58–66 (2015) Danelljan, M., Hager, G., Khan, F., Felsberg, M.: Convolutional features for correlation filter based visual tracking. In: ICCV 2015 Workshop, pp. 58–66 (2015)
7.
Zurück zum Zitat Wang, N., Li, S., Gupta, A., Yeung, D.Y.: Transferring rich feature hierarchies for robust visual tracking. arXiv CoRR (2015) Wang, N., Li, S., Gupta, A., Yeung, D.Y.: Transferring rich feature hierarchies for robust visual tracking. arXiv CoRR (2015)
8.
Zurück zum Zitat Wang, L., Ouyang, W., Wang, X., Lu, H.: Visual tracking with fully convolutional networks. In: ICCV (2015) Wang, L., Ouyang, W., Wang, X., Lu, H.: Visual tracking with fully convolutional networks. In: ICCV (2015)
9.
Zurück zum Zitat Nam, H., Han, B.: Learning multi-domain convolutional neural networks for visual tracking. arXiv CoRR (2015) Nam, H., Han, B.: Learning multi-domain convolutional neural networks for visual tracking. arXiv CoRR (2015)
10.
Zurück zum Zitat Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. IJCV 115(3), 211–252 (2015)MathSciNetCrossRef Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. IJCV 115(3), 211–252 (2015)MathSciNetCrossRef
11.
Zurück zum Zitat Wu, Y., Lim, J., Yang, M.H.: Online object tracking: a benchmark. In: CVPR 2013 (2013) Wu, Y., Lim, J., Yang, M.H.: Online object tracking: a benchmark. In: CVPR 2013 (2013)
12.
Zurück zum Zitat Kristan, M., Matas, J., Leonardis, A., Felsberg, M., Cehovin, L., Fernandez, G., Vojir, T., Hager, G., Nebehay, G., Pflugfelder, R.: The visual object tracking VOT2015 challenge results. In: ICCV 2015 Workshop, pp. 1–23 (2015) Kristan, M., Matas, J., Leonardis, A., Felsberg, M., Cehovin, L., Fernandez, G., Vojir, T., Hager, G., Nebehay, G., Pflugfelder, R.: The visual object tracking VOT2015 challenge results. In: ICCV 2015 Workshop, pp. 1–23 (2015)
13.
Zurück zum Zitat Razavian, A., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: An astounding baseline for recognition. In: CVPR 2014 Workshop (2014) Razavian, A., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: An astounding baseline for recognition. In: CVPR 2014 Workshop (2014)
14.
Zurück zum Zitat Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: BMVC 2015 (2015) Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: BMVC 2015 (2015)
15.
Zurück zum Zitat Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., van der Smagt, P., Cremers, D., Brox, T.: FlowNet: Learning optical flow with convolutional networks. In: ICCV 2015 (2015) Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., van der Smagt, P., Cremers, D., Brox, T.: FlowNet: Learning optical flow with convolutional networks. In: ICCV 2015 (2015)
16.
Zurück zum Zitat Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS 2012 (2012) Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS 2012 (2012)
17.
Zurück zum Zitat Bromley, J., Bentz, J.W., Bottou, L., Guyon, I., LeCun, Y., Moore, C., Säckinger, E., Shah, R.: Signature verification using a “Siamese” time delay neural network. Int. J. Pattern Recogn. Artif. Intell. (1993) Bromley, J., Bentz, J.W., Bottou, L., Guyon, I., LeCun, Y., Moore, C., Säckinger, E., Shah, R.: Signature verification using a “Siamese” time delay neural network. Int. J. Pattern Recogn. Artif. Intell. (1993)
18.
Zurück zum Zitat Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: DeepFace: closing the gap to human-level performance in face verification. CVPR 2014, 1701–1708 (2014) Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: DeepFace: closing the gap to human-level performance in face verification. CVPR 2014, 1701–1708 (2014)
19.
Zurück zum Zitat Zagoruyko, S., Komodakis, N.: Learning to compare image patches via convolutional neural networks. In: CVPR 2015 (2015) Zagoruyko, S., Komodakis, N.: Learning to compare image patches via convolutional neural networks. In: CVPR 2015 (2015)
20.
Zurück zum Zitat Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: CVPR 2015, pp. 815–823 (2015) Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: CVPR 2015, pp. 815–823 (2015)
21.
Zurück zum Zitat Simo-Serra, E., Trulls, E., Ferraz, L., Kokkinos, I., Fua, P., Moreno-Noguer, F.: Discriminative learning of deep convolutional feature point descriptors. ICCV 2015, 118–126 (2015) Simo-Serra, E., Trulls, E., Ferraz, L., Kokkinos, I., Fua, P., Moreno-Noguer, F.: Discriminative learning of deep convolutional feature point descriptors. ICCV 2015, 118–126 (2015)
22.
Zurück zum Zitat Koch, G., Zemel, R., Salakhutdinov, R.: Siamese neural networks for one-shot image recognition. In: ICML 2015 Deep Learning Workshop (2015) Koch, G., Zemel, R., Salakhutdinov, R.: Siamese neural networks for one-shot image recognition. In: ICML 2015 Deep Learning Workshop (2015)
23.
Zurück zum Zitat Luo, W., Schwing, A.G., Urtasun, R.: Efficient deep learning for stereo matching. In: CVPR 2016, pp. 5695–5703 (2016) Luo, W., Schwing, A.G., Urtasun, R.: Efficient deep learning for stereo matching. In: CVPR 2016, pp. 5695–5703 (2016)
24.
Zurück zum Zitat Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: ICML 2015, pp. 448–456 (2015) Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: ICML 2015, pp. 448–456 (2015)
25.
Zurück zum Zitat Gan, Q., Guo, Q., Zhang, Z., Cho, K.: First step toward model-free, anonymous object tracking with recurrent neural networks. arXiv CoRR (2015) Gan, Q., Guo, Q., Zhang, Z., Cho, K.: First step toward model-free, anonymous object tracking with recurrent neural networks. arXiv CoRR (2015)
26.
Zurück zum Zitat Kahou, S.E., Michalski, V., Memisevic, R.: RATM: Recurrent Attentive Tracking Model. arXiv CoRR (2015) Kahou, S.E., Michalski, V., Memisevic, R.: RATM: Recurrent Attentive Tracking Model. arXiv CoRR (2015)
27.
Zurück zum Zitat Denil, M., Bazzani, L., Larochelle, H., de Freitas, N.: Learning where to attend with deep architectures for image tracking. Neural Computation (2012) Denil, M., Bazzani, L., Larochelle, H., de Freitas, N.: Learning where to attend with deep architectures for image tracking. Neural Computation (2012)
28.
Zurück zum Zitat Held, D., Thrun, S., Savarese, S.: Learning to track at 100 FPS with deep regression networks. arXiv CoRR (2016) Held, D., Thrun, S., Savarese, S.: Learning to track at 100 FPS with deep regression networks. arXiv CoRR (2016)
29.
Zurück zum Zitat Chen, K., Tao, W.: Once for all: a two-flow convolutional neural network for visual tracking. arXiv CoRR (2016) Chen, K., Tao, W.: Once for all: a two-flow convolutional neural network for visual tracking. arXiv CoRR (2016)
30.
Zurück zum Zitat Tao, R., Gavves, E., Smeulders, A.W.M.: Siamese instance search for tracking. arXiv CoRR (2016) Tao, R., Gavves, E., Smeulders, A.W.M.: Siamese instance search for tracking. arXiv CoRR (2016)
31.
Zurück zum Zitat Vedaldi, A., Lenc, K.: MatConvNet – Convolutional Neural Networks for MATLAB (2015) Vedaldi, A., Lenc, K.: MatConvNet – Convolutional Neural Networks for MATLAB (2015)
32.
Zurück zum Zitat He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: ICCV 2015 (2015) He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: ICCV 2015 (2015)
33.
Zurück zum Zitat Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O., Torr, P.H.S.: Staple: Complementary learners for real-time tracking. In: CVPR 2016 (2016) Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O., Torr, P.H.S.: Staple: Complementary learners for real-time tracking. In: CVPR 2016 (2016)
34.
Zurück zum Zitat Ma, C., Yang, X., Zhang, C., Yang, M.H.: Long-term correlation tracking. In: CVPR 2015 (2015) Ma, C., Yang, X., Zhang, C., Yang, M.H.: Long-term correlation tracking. In: CVPR 2015 (2015)
35.
Zurück zum Zitat Zhu, G., Wang, J., Wu, Y., Lu, H.: Collaborative correlation tracking. In: BMVC 2015 (2015) Zhu, G., Wang, J., Wu, Y., Lu, H.: Collaborative correlation tracking. In: BMVC 2015 (2015)
36.
Zurück zum Zitat Choi, J., Jin Chang, H., Jeong, J., Demiris, Y., Young Choi, J.: Visual tracking using attention-modulated disintegration and integration. In: CVPR 2016 (2016) Choi, J., Jin Chang, H., Jeong, J., Demiris, Y., Young Choi, J.: Visual tracking using attention-modulated disintegration and integration. In: CVPR 2016 (2016)
37.
Zurück zum Zitat Ning, J., Yang, J., Jiang, S., Zhang, L., Yang, M.H.: Object tracking via dual linear structured SVM and explicit feature map. In: CVPR 2016 (2016) Ning, J., Yang, J., Jiang, S., Zhang, L., Yang, M.H.: Object tracking via dual linear structured SVM and explicit feature map. In: CVPR 2016 (2016)
38.
Zurück zum Zitat Danelljan, M., Häger, G., Khan, F., Felsberg, M.: Accurate scale estimation for robust visual tracking. In: BMVC 2014 (2014) Danelljan, M., Häger, G., Khan, F., Felsberg, M.: Accurate scale estimation for robust visual tracking. In: BMVC 2014 (2014)
39.
Zurück zum Zitat Huang12, D., Luo, L., Wen12, M., Chen12, Z., Zhang12, C.: Enable scale and aspect ratio adaptability in visual tracking with detection proposals Huang12, D., Luo, L., Wen12, M., Chen12, Z., Zhang12, C.: Enable scale and aspect ratio adaptability in visual tracking with detection proposals
40.
Zurück zum Zitat Liris, F.: The visual object tracking vot2014 challenge results (2014) Liris, F.: The visual object tracking vot2014 challenge results (2014)
41.
Zurück zum Zitat Zhu, G., Porikli, F., Li, H.: Tracking randomly moving objects on edge box proposals. arXiv CoRR (2015) Zhu, G., Porikli, F., Li, H.: Tracking randomly moving objects on edge box proposals. arXiv CoRR (2015)
42.
Zurück zum Zitat Danelljan, M., Hager, G., Shahbaz Khan, F., Felsberg, M.: Learning spatially regularized correlation filters for visual tracking. In: ICCV 2015, pp. 4310–4318 (2015) Danelljan, M., Hager, G., Shahbaz Khan, F., Felsberg, M.: Learning spatially regularized correlation filters for visual tracking. In: ICCV 2015, pp. 4310–4318 (2015)
43.
Zurück zum Zitat Hua, Y., Alahari, K., Schmid, C.: Online object tracking with proposal selection. ICCV 2015, 3092–3100 (2015) Hua, Y., Alahari, K., Schmid, C.: Online object tracking with proposal selection. ICCV 2015, 3092–3100 (2015)
44.
Zurück zum Zitat Wang, N., Yeung, D.Y.: Ensemble-based tracking: aggregating crowdsourced structured time series data. ICML 2014, 1107–1115 (2014) Wang, N., Yeung, D.Y.: Ensemble-based tracking: aggregating crowdsourced structured time series data. ICML 2014, 1107–1115 (2014)
45.
Zurück zum Zitat Li, Y., Zhu, J.: A scale adaptive kernel correlation filter tracker with feature integration. In: ECCV 2014 Workshops (2014) Li, Y., Zhu, J.: A scale adaptive kernel correlation filter tracker with feature integration. In: ECCV 2014 Workshops (2014)
46.
Zurück zum Zitat Li, A., Lin, M., Wu, Y., Yang, M.H., Yan, S.: NUS-PRO: a new visual tracking challenge. PAMI 38(2), 335–349 (2016)CrossRef Li, A., Lin, M., Wu, Y., Yang, M.H., Yan, S.: NUS-PRO: a new visual tracking challenge. PAMI 38(2), 335–349 (2016)CrossRef
47.
Zurück zum Zitat Possegger, H., Mauthner, T., Bischof, H.: In defense of color-based model-free tracking. In: CVPR 2015 (2015) Possegger, H., Mauthner, T., Bischof, H.: In defense of color-based model-free tracking. In: CVPR 2015 (2015)
Metadaten
Titel
Fully-Convolutional Siamese Networks for Object Tracking
verfasst von
Luca Bertinetto
Jack Valmadre
João F. Henriques
Andrea Vedaldi
Philip H. S. Torr
Copyright-Jahr
2016
DOI
https://doi.org/10.1007/978-3-319-48881-3_56