Skip to main content
Erschienen in: Pattern Analysis and Applications 4/2018

05.05.2018 | Original Article

Batch-based activity recognition from egocentric photo-streams revisited

verfasst von: Alejandro Cartas, Juan Marín, Petia Radeva, Mariella Dimiccoli

Erschienen in: Pattern Analysis and Applications | Ausgabe 4/2018

Einloggen

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Wearable cameras can gather large amounts of image data that provide rich visual information about the daily activities of the wearer. Motivated by the large number of health applications that could be enabled by the automatic recognition of daily activities, such as lifestyle characterization for habit improvement, context-aware personal assistance and tele-rehabilitation services, we propose a system to classify 21 daily activities from photo-streams acquired by a wearable photo-camera. Our approach combines the advantages of a late fusion ensemble strategy relying on convolutional neural networks at image level with the ability of recurrent neural networks to account for the temporal evolution of high-level features in photo-streams without relying on event boundaries. The proposed batch-based approach achieved an overall accuracy of 89.85%, outperforming state-of-the-art end-to-end methodologies. These results were achieved on a dataset consists of 44,902 egocentric pictures from three persons captured during 26 days in average.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
Literatur
1.
Zurück zum Zitat Bolaños M, Dimiccoli M, Radeva P (2017) Toward storytelling from visual lifelogging: an overview. IEEE Trans Hum Mach Syst 47(1):77–90 Bolaños M, Dimiccoli M, Radeva P (2017) Toward storytelling from visual lifelogging: an overview. IEEE Trans Hum Mach Syst 47(1):77–90
2.
Zurück zum Zitat Breiman L, Friedman J, Stone CJ, Olshen RA (1984) Classification and regression trees. CRC Press, Boca RatonMATH Breiman L, Friedman J, Stone CJ, Olshen RA (1984) Classification and regression trees. CRC Press, Boca RatonMATH
3.
Zurück zum Zitat Cartas A, Dimiccoli M, Radeva P (2017) Batch-based activity recognition from egocentric photo-streams. In: Proceedings of the international conference on computer vision (ICCV), workshop on egocentric perception, interaction and computing. IEEE Cartas A, Dimiccoli M, Radeva P (2017) Batch-based activity recognition from egocentric photo-streams. In: Proceedings of the international conference on computer vision (ICCV), workshop on egocentric perception, interaction and computing. IEEE
4.
Zurück zum Zitat Cartas A, Marín J, Radeva P, Dimiccoli M (2017) Recognizing activities of daily living from egocentric images. In: Proceedings of the Iberian conference on pattern recognition and image analysis (IbPRIA). Springer, Cham, pp 87–95CrossRef Cartas A, Marín J, Radeva P, Dimiccoli M (2017) Recognizing activities of daily living from egocentric images. In: Proceedings of the Iberian conference on pattern recognition and image analysis (IbPRIA). Springer, Cham, pp 87–95CrossRef
5.
Zurück zum Zitat Castro D, Hickson S, Bettadapura V, Thomaz E, Abowd G, Christensen H, Essa I (2015) Predicting daily activities from egocentric images using deep learning. In: Proceedings of the 2015 ACM international symposium on wearable computers. ACM, pp 75–82 Castro D, Hickson S, Bettadapura V, Thomaz E, Abowd G, Christensen H, Essa I (2015) Predicting daily activities from egocentric images using deep learning. In: Proceedings of the 2015 ACM international symposium on wearable computers. ACM, pp 75–82
7.
Zurück zum Zitat Dimiccoli M, Bolaños M, Talavera E, Aghaei M, Nikolov SG, Radeva P (2016) Sr-clustering: semantic regularized clustering for egocentric photo streams segmentation. Comput Vis Image Underst 155:55–69CrossRef Dimiccoli M, Bolaños M, Talavera E, Aghaei M, Nikolov SG, Radeva P (2016) Sr-clustering: semantic regularized clustering for egocentric photo streams segmentation. Comput Vis Image Underst 155:55–69CrossRef
8.
Zurück zum Zitat Donahue J, Hendricks LA, Guadarrama S, Rohrbach M, Venugopalan S, Saenko K, Darrell T (2015) Long-term recurrent convolutional networks for visual recognition and description. In: CVPR Donahue J, Hendricks LA, Guadarrama S, Rohrbach M, Venugopalan S, Saenko K, Darrell T (2015) Long-term recurrent convolutional networks for visual recognition and description. In: CVPR
9.
Zurück zum Zitat Fathi A, Farhadi A, Rehg JM (2011) Understanding egocentric activities. In: 2011 international conference on computer vision. IEEE, pp 407–414 Fathi A, Farhadi A, Rehg JM (2011) Understanding egocentric activities. In: 2011 international conference on computer vision. IEEE, pp 407–414
10.
Zurück zum Zitat Fathi A, Li Y, Rehg JM (2012) Learning to recognize daily actions using gaze. In: European conference on computer vision. Springer, pp 314–327 Fathi A, Li Y, Rehg JM (2012) Learning to recognize daily actions using gaze. In: European conference on computer vision. Springer, pp 314–327
12.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: The IEEE conference on computer vision and pattern recognition (CVPR) He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: The IEEE conference on computer vision and pattern recognition (CVPR)
14.
Zurück zum Zitat Ma M, Fan H, Kitani KM (2016) Going deeper into first-person activity recognition. In: The IEEE conference on computer vision and pattern recognition (CVPR) Ma M, Fan H, Kitani KM (2016) Going deeper into first-person activity recognition. In: The IEEE conference on computer vision and pattern recognition (CVPR)
15.
Zurück zum Zitat Martin-Lesende I, Vrotsou K, Vergara I, Bueno A, Diez A et al (2015) Design and validation of the vida questionnaire, for assessing instrumental activities of daily living in elderly people. J Gerontol Geriatr Res 4(214):2 Martin-Lesende I, Vrotsou K, Vergara I, Bueno A, Diez A et al (2015) Design and validation of the vida questionnaire, for assessing instrumental activities of daily living in elderly people. J Gerontol Geriatr Res 4(214):2
16.
Zurück zum Zitat Mukhopadhyay SC (2015) Wearable sensors for human activity monitoring: a review. IEEE Sens J 15(3):1321–1330CrossRef Mukhopadhyay SC (2015) Wearable sensors for human activity monitoring: a review. IEEE Sens J 15(3):1321–1330CrossRef
18.
Zurück zum Zitat Oliveira-Barra G, Dimiccoli M, Radeva P (2017) Leveraging activity indexing for egocentric image retrieval. In: Iberian conference on pattern recognition and image analysis. Springer, pp 295–303 Oliveira-Barra G, Dimiccoli M, Radeva P (2017) Leveraging activity indexing for egocentric image retrieval. In: Iberian conference on pattern recognition and image analysis. Springer, pp 295–303
19.
Zurück zum Zitat Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830MathSciNetMATH Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830MathSciNetMATH
20.
Zurück zum Zitat Pirsiavash H, Ramanan D (2012) Detecting activities of daily living in first-person camera views. In: IEEE conference on computer vision and pattern recognition (CVPR), 2012. IEEE, pp 2847–2854 Pirsiavash H, Ramanan D (2012) Detecting activities of daily living in first-person camera views. In: IEEE conference on computer vision and pattern recognition (CVPR), 2012. IEEE, pp 2847–2854
21.
Zurück zum Zitat Schüssler-Fiorenza Rose SM, Stineman MG, Pan Q, Bogner H, Kurichi JE, Streim JE, Xie D (2016) Potentially avoidable hospitalizations among people at different activity of daily living limitation stages. Health Serv Res 52:132–155CrossRef Schüssler-Fiorenza Rose SM, Stineman MG, Pan Q, Bogner H, Kurichi JE, Streim JE, Xie D (2016) Potentially avoidable hospitalizations among people at different activity of daily living limitation stages. Health Serv Res 52:132–155CrossRef
23.
Zurück zum Zitat Singh S, Arora C, Jawahar CV (2016) First person action recognition using deep learned descriptors. In: The IEEE conference on computer vision and pattern recognition (CVPR) Singh S, Arora C, Jawahar CV (2016) First person action recognition using deep learned descriptors. In: The IEEE conference on computer vision and pattern recognition (CVPR)
24.
Zurück zum Zitat Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: The IEEE conference on computer vision and pattern recognition (CVPR) Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: The IEEE conference on computer vision and pattern recognition (CVPR)
25.
Zurück zum Zitat Yue-Hei Ng J, Hausknecht M, Vijayanarasimhan S, Vinyals O, Monga R, Toderici G (2015) Beyond short snippets: deep networks for video classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4694–4702 Yue-Hei Ng J, Hausknecht M, Vijayanarasimhan S, Vinyals O, Monga R, Toderici G (2015) Beyond short snippets: deep networks for video classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4694–4702
Metadaten
Titel
Batch-based activity recognition from egocentric photo-streams revisited
verfasst von
Alejandro Cartas
Juan Marín
Petia Radeva
Mariella Dimiccoli
Publikationsdatum
05.05.2018
Verlag
Springer London
Erschienen in
Pattern Analysis and Applications / Ausgabe 4/2018
Print ISSN: 1433-7541
Elektronische ISSN: 1433-755X
DOI
https://doi.org/10.1007/s10044-018-0708-1

Weitere Artikel der Ausgabe 4/2018

Pattern Analysis and Applications 4/2018 Zur Ausgabe