Skip to main content
Erschienen in: Soft Computing 7/2019

18.11.2017 | Methodologies and Application

A soft-computing-based approach to artificial visual attention using human eye-fixation paradigm: toward a human-like skill in robot vision

verfasst von: Kurosh Madani, Viachaslau Kachurka, Christophe Sabourin, Vladimir Golovko

Erschienen in: Soft Computing | Ausgabe 7/2019

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Fitting the skills of the natural vision is an appealing perspective for artificial vision systems, especially in robotics applications dealing with visual perception of the complex surrounding environment where robots and humans mutually evolve and/or cooperate, or in a more general way, those prospecting human–robot interaction. Focusing the visual attention dilemma through human eye-fixation paradigm, in this work we propose a model for artificial visual attention combining a statistical foundation of visual saliency and a genetic tuning of the related parameters for robots’ visual perception. The computational issue of our model relies on the one hand on center-surround statistical features’ calculations with a nonlinear fusion of different resulting maps, and on the other hand on an evolutionary tuning of human’s gazing way resulting in emergence of a kind of artificial eye-fixation-based visual attention. Statistical foundation and bottom-up nature of the proposed model provide as well the advantage to make it usable without needing prior information as a comprehensive solid theoretical basement. The eye-fixation paradigm has been considered as a keystone of the human-like gazing attribute, molding the robot’s visual behavior toward the human’s one. The same paradigm, providing MIT1003 and TORONTO image datasets, has served as evaluation benchmark for experimental validation of the proposed system. The reported experimental results show viability of the incorporated genetic tuning process in shoving the conduct of the artificial system toward the human-like gazing mechanism. In addition, a comparison to currently best algorithms used in the aforementioned field is reported through MIT300 dataset. While not being designed for eye-fixation prediction task, the proposed system remains comparable to most of algorithms within this leading group of currently best state-of-the-art algorithms used in the aforementioned field. Moreover, about ten times faster than the currently best state-of-the-art algorithms, the promising execution speed of our approach makes it suitable for an effective implementation fitting requirement for real-time robotics applications.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Literatur
Zurück zum Zitat Achanta R, Estrada F, Wils P, Susstrunk S (2008) Salient Region Detection and Segmentation. In: Proceedings of international conference on computer vision systems, vol 5008, LNCS, Springer, Berlin/Heidelberg, pp 66–75 Achanta R, Estrada F, Wils P, Susstrunk S (2008) Salient Region Detection and Segmentation. In: Proceedings of international conference on computer vision systems, vol 5008, LNCS, Springer, Berlin/Heidelberg, pp 66–75
Zurück zum Zitat Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned Salient Region Detection. In: Proceedings of IEEE international conference on computer vision and pattern recognition, pp 1597–1604 Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned Salient Region Detection. In: Proceedings of IEEE international conference on computer vision and pattern recognition, pp 1597–1604
Zurück zum Zitat Borji A, Tavakoli HR, Sihite DN, Itti L (2013a) Analysis of scores, datasets, and models in visual saliency prediction. In: Proceedings of IEEE ICCV, pp 921–928 Borji A, Tavakoli HR, Sihite DN, Itti L (2013a) Analysis of scores, datasets, and models in visual saliency prediction. In: Proceedings of IEEE ICCV, pp 921–928
Zurück zum Zitat Borji A, Itti L (2013) State-of-the-art in visual attention modeling. IEEE Trans Pattern Anal Mach Intell 35(1):185–207CrossRef Borji A, Itti L (2013) State-of-the-art in visual attention modeling. IEEE Trans Pattern Anal Mach Intell 35(1):185–207CrossRef
Zurück zum Zitat Borji A, Sihite DN, Itti L (2013) Quantitative analysis of human-model agreement in visual saliency modeling: a comparative study. IEEE Trans Image Process 22(1):55–69MathSciNetCrossRefMATH Borji A, Sihite DN, Itti L (2013) Quantitative analysis of human-model agreement in visual saliency modeling: a comparative study. IEEE Trans Image Process 22(1):55–69MathSciNetCrossRefMATH
Zurück zum Zitat Bruce NDB, Tsotsos JK (2009) Saliency, attention, and visual search: an information theoretic approach. J Vis 9(3):1–24CrossRef Bruce NDB, Tsotsos JK (2009) Saliency, attention, and visual search: an information theoretic approach. J Vis 9(3):1–24CrossRef
Zurück zum Zitat Chella A, Macaluso I (2009) The perception loop in CiceRobot, a museum guide robot. Neurocomputing 72(4–6):760–766CrossRef Chella A, Macaluso I (2009) The perception loop in CiceRobot, a museum guide robot. Neurocomputing 72(4–6):760–766CrossRef
Zurück zum Zitat Contreras-Reyes JE, Arellano-Valle RB (2012) Küllback-Leibler divergence measure for multivariate skew-normal distributions. Entropy 14(9):1606–1626MathSciNetCrossRefMATH Contreras-Reyes JE, Arellano-Valle RB (2012) Küllback-Leibler divergence measure for multivariate skew-normal distributions. Entropy 14(9):1606–1626MathSciNetCrossRefMATH
Zurück zum Zitat Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27:861–874CrossRef Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27:861–874CrossRef
Zurück zum Zitat Cornia M, Baraldi L, Serra G, Cucchiara R (2016a) Predicting human eye fixations via an LSTM-based saliency attentive model. CoRR. arXiv:1611.09571 Cornia M, Baraldi L, Serra G, Cucchiara R (2016a) Predicting human eye fixations via an LSTM-based saliency attentive model. CoRR. arXiv:​1611.​09571
Zurück zum Zitat Cornia M., Baraldi L., Serra G., Cucchiara R (2016b) A deep multi-level network for saliency prediction. In Proceedings of international conference on pattern recognition (ICPR) Cornia M., Baraldi L., Serra G., Cucchiara R (2016b) A deep multi-level network for saliency prediction. In Proceedings of international conference on pattern recognition (ICPR)
Zurück zum Zitat Harel J, Koch C, Perona P (2007) Graph-based visual saliency. Adv Neural Inf Process Syst 19:545–552 Harel J, Koch C, Perona P (2007) Graph-based visual saliency. Adv Neural Inf Process Syst 19:545–552
Zurück zum Zitat Hayhoe M, Ballard D (2005) Eye movements in natural behavior. Trends Cogn Sci 9:188–194CrossRef Hayhoe M, Ballard D (2005) Eye movements in natural behavior. Trends Cogn Sci 9:188–194CrossRef
Zurück zum Zitat Holzbach A, Cheng G (2014) A scalable and efficient method for salient region detection using sampled template collation. In: Proceedings of IEEE ICIP, pp 1110–1114 Holzbach A, Cheng G (2014) A scalable and efficient method for salient region detection using sampled template collation. In: Proceedings of IEEE ICIP, pp 1110–1114
Zurück zum Zitat Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intel 20:1254–1259CrossRef Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intel 20:1254–1259CrossRef
Zurück zum Zitat Jiang M, Xu J, Zhao Q (2014) Saliency in crowd. In: Proceedings of ECCV, Lecture Notes in Computer Science, vol 8695, pp 17–32 Jiang M, Xu J, Zhao Q (2014) Saliency in crowd. In: Proceedings of ECCV, Lecture Notes in Computer Science, vol 8695, pp 17–32
Zurück zum Zitat Judd T, Ehinger K, Durand F, Torralba A (2009) Learning to predict where humans look. In: Proceedings of IEEE ICCV, pp 2106–2113 Judd T, Ehinger K, Durand F, Torralba A (2009) Learning to predict where humans look. In: Proceedings of IEEE ICCV, pp 2106–2113
Zurück zum Zitat Kachurka V, Madani K, Sabourin C, Golovko V (2014) A statistical approach to human-like visual attention and saliency detection for robot vision: application to wildland fires detection. In: Proceedings of ICNNAI 2014, Brest, Byelorussia, June 3–6, CCIS series, vol 440. Springer, pp 124–135 Kachurka V, Madani K, Sabourin C, Golovko V (2014) A statistical approach to human-like visual attention and saliency detection for robot vision: application to wildland fires detection. In: Proceedings of ICNNAI 2014, Brest, Byelorussia, June 3–6, CCIS series, vol 440. Springer, pp 124–135
Zurück zum Zitat Kachurka V, Madani K, Sabourin C, Golovko V (2015) From human eye fixation to human-like autonomous artificial vision. In: Proceedings of the international work-conference on artificial neural networks (IWANN 2015), LNCS series, vol 9094, Part I. Springer, pp 171–184 Kachurka V, Madani K, Sabourin C, Golovko V (2015) From human eye fixation to human-like autonomous artificial vision. In: Proceedings of the international work-conference on artificial neural networks (IWANN 2015), LNCS series, vol 9094, Part I. Springer, pp 171–184
Zurück zum Zitat Kadir T, Brady M (2001) Saliency, scale and image description. J Vis 45(2):83–105MATH Kadir T, Brady M (2001) Saliency, scale and image description. J Vis 45(2):83–105MATH
Zurück zum Zitat Kienzle W, Franz MO, Schölkopf B, Wichmann FA (2009) Center-surround patterns emerge as optimal predictors for human saccade targets. J Vis 9:1–15CrossRef Kienzle W, Franz MO, Schölkopf B, Wichmann FA (2009) Center-surround patterns emerge as optimal predictors for human saccade targets. J Vis 9:1–15CrossRef
Zurück zum Zitat Koehler K, Guo F, Zhang S, Eckstein MP (2014) What do saliency models predict? J Vis 14(3):1–27CrossRef Koehler K, Guo F, Zhang S, Eckstein MP (2014) What do saliency models predict? J Vis 14(3):1–27CrossRef
Zurück zum Zitat Kümmerer M, Wallis TS, Bethge M (2016) DeepGaze II: reading fixations from deep features trained on object recognition. arXiv:1610.01563 Kümmerer M, Wallis TS, Bethge M (2016) DeepGaze II: reading fixations from deep features trained on object recognition. arXiv:​1610.​01563
Zurück zum Zitat Kümmerer M, Theis L, Bethge M, DeepGaze I (2015) Boosting saliency prediction with feature maps trained on ImageNet. In: Proceedings of international conference on learning representations (ICLR) Kümmerer M, Theis L, Bethge M, DeepGaze I (2015) Boosting saliency prediction with feature maps trained on ImageNet. In: Proceedings of international conference on learning representations (ICLR)
Zurück zum Zitat Liang Z, Chi Z, Fu H, Feng D (2012) Salient object detection using content-sensitive hypergraph representation and partitioning. Pattern Rec 45(11):3886–3901CrossRef Liang Z, Chi Z, Fu H, Feng D (2012) Salient object detection using content-sensitive hypergraph representation and partitioning. Pattern Rec 45(11):3886–3901CrossRef
Zurück zum Zitat Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum H-Y (2001) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353–367 Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum H-Y (2001) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353–367
Zurück zum Zitat Liu T, Sun J, Zheng N. N, Shum HY (2007) Learning to detect a salient object. In: Proceedings of IEEE ICCV, pp 1–8 Liu T, Sun J, Zheng N. N, Shum HY (2007) Learning to detect a salient object. In: Proceedings of IEEE ICCV, pp 1–8
Zurück zum Zitat Moreno R, Ramik DM, Graña M, Madani K (2012) Image segmentation on the spherical coordinate representation of the RGB color space. IET Image Proc 6(9):1275–1283CrossRef Moreno R, Ramik DM, Graña M, Madani K (2012) Image segmentation on the spherical coordinate representation of the RGB color space. IET Image Proc 6(9):1275–1283CrossRef
Zurück zum Zitat Navalpakkam V, Itti L (2006) An integrated model of top-down and bottom-up attention for optimizing detection speed. Proc of IEEE CVPR II:2049–2056 Navalpakkam V, Itti L (2006) An integrated model of top-down and bottom-up attention for optimizing detection speed. Proc of IEEE CVPR II:2049–2056
Zurück zum Zitat Pan J, Canton C, McGuinness K, O’Connor NE, Torres J (2017) SalGAN: visual saliency prediction with generative adversarial networks. In: Proceedings of scene understanding workshop (SUNw), CVPR 2017, July 21 to July 26, 2017, Honolulu, Hawaii, USA Pan J, Canton C, McGuinness K, O’Connor NE, Torres J (2017) SalGAN: visual saliency prediction with generative adversarial networks. In: Proceedings of scene understanding workshop (SUNw), CVPR 2017, July 21 to July 26, 2017, Honolulu, Hawaii, USA
Zurück zum Zitat Panerai F, Metta G, Sandini G (2002) Learning visual stabilization reflexes in robots with moving eyes. Neurocomputing 48(1–4):323–337CrossRefMATH Panerai F, Metta G, Sandini G (2002) Learning visual stabilization reflexes in robots with moving eyes. Neurocomputing 48(1–4):323–337CrossRefMATH
Zurück zum Zitat Peters RJ, Iyer A, Itti L, Koch C (2005) Components of bottom-up gaze allocation in natural images. Vis Res 45(18):2397–2416CrossRef Peters RJ, Iyer A, Itti L, Koch C (2005) Components of bottom-up gaze allocation in natural images. Vis Res 45(18):2397–2416CrossRef
Zurück zum Zitat Rajashekar U, Vander Linde I, Bovik AC, Cormack LK (2008) GAFFE: a Gaze- attentive fixation finding engine. IEEE Trans Image Process 17(4):564–573MathSciNetCrossRef Rajashekar U, Vander Linde I, Bovik AC, Cormack LK (2008) GAFFE: a Gaze- attentive fixation finding engine. IEEE Trans Image Process 17(4):564–573MathSciNetCrossRef
Zurück zum Zitat Ramik DM (2012) Contribution to complex visual information processing and autonomous knowledge extraction: application to autonomous robotics. Ph.D. dissertation, University Paris-Est, Pub. No. 2012PEST1100 Ramik DM (2012) Contribution to complex visual information processing and autonomous knowledge extraction: application to autonomous robotics. Ph.D. dissertation, University Paris-Est, Pub. No. 2012PEST1100
Zurück zum Zitat Ramik DM, Sabourin C, Madani K (2011) Hybrid salient object extraction approach with automatic estimation of visual attention scale. In: Proceedings of IEEE SITIS, pp 438–445 Ramik DM, Sabourin C, Madani K (2011) Hybrid salient object extraction approach with automatic estimation of visual attention scale. In: Proceedings of IEEE SITIS, pp 438–445
Zurück zum Zitat Ramik DM, Sabourin C, Moreno R, Madani K (2014) A machine learning based intelligent vision system for autonomous object detection and recognition. J Appl Intell 40(2):358–375CrossRef Ramik DM, Sabourin C, Moreno R, Madani K (2014) A machine learning based intelligent vision system for autonomous object detection and recognition. J Appl Intell 40(2):358–375CrossRef
Zurück zum Zitat Ramik DM, Madani K, Sabourin C (2015) A soft-computing basis for robots’ cognitive autonomous learning. Soft Comput J 19:2407–2421CrossRef Ramik DM, Madani K, Sabourin C (2015) A soft-computing basis for robots’ cognitive autonomous learning. Soft Comput J 19:2407–2421CrossRef
Zurück zum Zitat Riche N, Duvinage M, Mancas M, Gosselin B, Dutoit T (2013) Saliency and human fixations: state-of-the-art and study of comparison metrics. In: Proceedings of IEEE ICCV, pp 1153–1160 Riche N, Duvinage M, Mancas M, Gosselin B, Dutoit T (2013) Saliency and human fixations: state-of-the-art and study of comparison metrics. In: Proceedings of IEEE ICCV, pp 1153–1160
Zurück zum Zitat Riche N, Mancas M, Duvinage M, Mibulumukini M, Gosselin B, Dutoit T (2013) RARE2012: A multi-scale rarity-based saliency detection with its comparative statistical analysis. Signal Process Image Commun J 28(6):642–658CrossRef Riche N, Mancas M, Duvinage M, Mibulumukini M, Gosselin B, Dutoit T (2013) RARE2012: A multi-scale rarity-based saliency detection with its comparative statistical analysis. Signal Process Image Commun J 28(6):642–658CrossRef
Zurück zum Zitat Shen Ch, Zhao Q (2014) Webpage saliency. In: Proceedings of ECCV, Lecture Notes in Computer Science, vol 8695, pp 34–46 Shen Ch, Zhao Q (2014) Webpage saliency. In: Proceedings of ECCV, Lecture Notes in Computer Science, vol 8695, pp 34–46
Zurück zum Zitat Subramanian R, Katti H, Sebe N, Kankanhalli M, Chua T-S (2010) An eye fixation database for saliency detection in images. In: Proceedings of ECCV, pp 30–43 Subramanian R, Katti H, Sebe N, Kankanhalli M, Chua T-S (2010) An eye fixation database for saliency detection in images. In: Proceedings of ECCV, pp 30–43
Zurück zum Zitat Tatler BW (2007) The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor bases and image feature distributions. J Vis 14:1–17 Tatler BW (2007) The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor bases and image feature distributions. J Vis 14:1–17
Zurück zum Zitat Tavakoli HR, Laaksonen J (2016) Bottom-up fixation prediction using unsupervised hierarchical models. In: Proceedings of ACCV 2016, Workshop on Assistive Vision, LNCS 10116. Spinger, pp 287–302 Tavakoli HR, Laaksonen J (2016) Bottom-up fixation prediction using unsupervised hierarchical models. In: Proceedings of ACCV 2016, Workshop on Assistive Vision, LNCS 10116. Spinger, pp 287–302
Zurück zum Zitat Triesch J, Ballard DH, Hayhoe MM, Sullivan BT (2003) What you see is what you need. J Vis 3:86–94CrossRef Triesch J, Ballard DH, Hayhoe MM, Sullivan BT (2003) What you see is what you need. J Vis 3:86–94CrossRef
Zurück zum Zitat Vig E, Dorr M, Cox D (2014) Large-scale optimization of hierarchical features for saliency prediction in natural images. In: Proceedings of IEEE CVPR, pp 2798–2805 Vig E, Dorr M, Cox D (2014) Large-scale optimization of hierarchical features for saliency prediction in natural images. In: Proceedings of IEEE CVPR, pp 2798–2805
Zurück zum Zitat Võ ML-H, Smith TJ, Mital PK, Henderson JM (2012) Do the eyes really have it? Dynamic allocation of attention when viewing moving faces? J Vis 12(13):1–14CrossRef Võ ML-H, Smith TJ, Mital PK, Henderson JM (2012) Do the eyes really have it? Dynamic allocation of attention when viewing moving faces? J Vis 12(13):1–14CrossRef
Zurück zum Zitat Zhang J, Sclaroff S (2013) Saliency detection: a Boolean map approach. In: Proceedings of IEEE ICCV, pp 153–160 Zhang J, Sclaroff S (2013) Saliency detection: a Boolean map approach. In: Proceedings of IEEE ICCV, pp 153–160
Metadaten
Titel
A soft-computing-based approach to artificial visual attention using human eye-fixation paradigm: toward a human-like skill in robot vision
verfasst von
Kurosh Madani
Viachaslau Kachurka
Christophe Sabourin
Vladimir Golovko
Publikationsdatum
18.11.2017
Verlag
Springer Berlin Heidelberg
Erschienen in
Soft Computing / Ausgabe 7/2019
Print ISSN: 1432-7643
Elektronische ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-017-2931-x

Weitere Artikel der Ausgabe 7/2019

Soft Computing 7/2019 Zur Ausgabe