ABSTRACT
This paper proposes an automatic method for predicting the inter-observer visual congruency (IOVC). The IOVC reflects the congruence or the variability among different subjects looking at the same image. Predicting this congruence is of interest for image processing applications where the visual perception of a picture matters such as website design, advertisement, etc. This paper makes several new contributions. First, a computational model of the IOVC is proposed. This new model is a mixture of low-level visual features extracted from the input picture where model's parameters are learned by using a large eye-tracking database. Once the parameters have been learned, it can be used for any new picture. Second, regarding low-level visual feature extraction, we propose a new scheme to compute the depth of field of a picture. Finally, once the training and the feature extraction have been carried out, a score ranging from 0 (minimal congruency) to 1 (maximal congruency) is computed. A value of 1 indicates that observers would focus on the same locations and suggests that the picture presents strong locations of interest. A second database of eye movements is used to assess the performance of the proposed model. Results show that our IOVC criterion outperforms the Feature Congestion measure \cite{Rosenholtz2007}. To illustrate the interest of the proposed model, we have used it to automatically rank personalized photograph.
- R. Althoff and N. Cohen. Eye-movement-based memory effect: a reprocessing effect in face perception. Jounral Of Experimental Psychology-Learning Memory and Cognition, 25(4):997--1010, 1999.Google ScholarCross Ref
- R. Baddeley and B. Tatler. High frequency edges (but not contrast predict where we fixate: A bayesian system identification analysis. Vision Research, 46:2824--2833, 2006.Google ScholarCross Ref
- S. Bhattacharya, R. Sukthankar, and M. Shah. A coherent framework for photo-quality assessment and enhancement based on visual aesthetics. In in ACM Multimedia International conference, 2010. Google ScholarDigital Library
- C. Christoudias, B. Georgescu, and P. Meer. Synergism in low-level vision. In 16th International Conference on Pattern Recognition, volume IV, pages 105--155, 2002. Google ScholarDigital Library
- H. Chua, J. Boland, and R. Nisbett. Cultural variation in eye movements during scene perception. In Proceedings of the National Academy of Sciences, volume 102, pages 12629--12633, 2005.Google ScholarCross Ref
- D. Cohen-Or, O. Sorkine, R. Gal, T. Leyvand, and Y. Xu. Color harmonization. In ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH), volume 56, pages 624--630, 2006. Google ScholarDigital Library
- D. Comaniciu and P. Meer. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Analysis and Machine Intelligence, 24:603--619, 2002. Google ScholarDigital Library
- L. Cowen, L. Ball, and J. Delin. An eye-movement analysis of web-page usability. In L. S. V. Ltd, editor, People and Computers XVI-Memorable yet invisible: Proceedings of HCI 2002, pages 317--335, 2002.Google Scholar
- K. Ehinger, B. Hidalgo-Sotelo, A. Torralba, and A. Oliva. Modeling search for people in 900 scenes. Visual Cognition, 17:945--978, 2009.Google ScholarCross Ref
- H. Einhorn. Accepting erro to make less error. Journal of Personality Assessment, 50(3):387--395, 1986.Google ScholarCross Ref
- H. Frey, C. Honey, and P. Konig. What's color got to do with it? the influence of color on visual attention in different categories. Journal of Vision, 8(14), October 2008.Google ScholarCross Ref
- Gershnfel. The nature of mathematical modelling. Cambridge, Univ. Press, 1999. Google ScholarDigital Library
- H. Golberg and X. Kotval. Computer interface evaluation using eye movements: methods and constructs. International Journal of Industrial Ergonomics, 24:631--645, 1999.Google ScholarCross Ref
- R. Gordon. Attentional allocation during the perception of scenes. Journal of Experimental Psychology: Human Perception and Performance, 30:760--777, 2004.Google ScholarCross Ref
- T. Hastie, R. Tibshirani, and J. Friedman. The elements of statistical learning. Springer Series in Statistics, 2001.Google Scholar
- J. Henderson. Regarding scenes. Current Directions in Psychological Science, 16:219--222, 2007.Google ScholarCross Ref
- J. Henderson, M. Chanceaux, and T. Smith. The influence of clutter on real-world scene search: Evidence from search efficiency and eye movements. Journal of Vision, 9(1), January 2009.Google ScholarCross Ref
- M. Jordan and R. Jacobs. Hierarchical mixtures of experts and the em algorihtm. Neural Computation, 6:181--214, 1994. Google ScholarDigital Library
- T. Judd, F. Durand, and A. Torralba. Fixations on low-resolution images. Journal of Vision, 11(4), 2011.Google ScholarCross Ref
- T. Judd, K. Ehinger, F. Durand, and A. Torralba. Learning to predict where people look. In ICCV, 2009.Google ScholarCross Ref
- O. Le Meur, P. Le Callet, D. Barba, and D. Thoreau. A coherent computational approach to model the bottom-up visual attention. IEEE Trans. On PAMI, 28(5):802--817, May 2006. Google ScholarDigital Library
- A. Levin. Blind motion deblurring using image statistics. In NIPS, 2006.Google Scholar
- R. Lienhart and J. Maydt. An extended set of haar-like features for rapid object detection. In ICIP, volume 1, pages 900--903, 2002.Google ScholarCross Ref
- G. Loftus and N. Mackworth. Cognitive determinants of fixation location during picture viewing. Journal of Experimental Psychology: Human Perception and Performances, 4:565--572, 1978.Google ScholarCross Ref
- Y. Luo and X. Tang. Photo and video quality evaluation: focussing on the subject. In ECCV, pages 386--399, 2008. Google ScholarDigital Library
- Y. Matsuda. Coor design. In Asakura Shoten, 1995.Google Scholar
- R. Nisbett. The geography of thought: how Asians and Westerners think differently... and why. New York: Free Press, 2003.Google Scholar
- A. Oliva, M. Mack, M. Shrestha, and A. Peeper. Identifying the perceptual dimensions of visual complexity of scenes. In 26th annual meeting of the Cognitive Science Society Meeting, 2004.Google Scholar
- D. Parkhurst, K. Law, and E. Niebur. Modelling the role of salience in the allocation of overt visual attention. Vision Research, 42:107--123, 2002.Google ScholarCross Ref
- W. Press, S. Teukolsky, W. Vetterling, and B. Flannery. Numerical Recipes in C: the art of Scientific Computing. Cambridge University Press, New York, NY, USA, 1992. Google ScholarDigital Library
- K. Rayner. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3):372--422, 1998.Google ScholarCross Ref
- K. Rayner, M. Catelhano, and J. Yang. Eye movements when looking at unusual-weird scenes: are there cultural differences? Journal of Experimental psychology: learning, Memory and cognition, 35(1):154--259, 2009.Google Scholar
- R. Rosenholtz, Y. Li, and L. Nakano. Measuring visual clutter. Journal of Vision, 7(2), March 2007.Google ScholarCross Ref
- M. Ross and A. Oliva. Estimating perception of scene layout properties from global image features. Journal Of Vision, 10(1), Januray 2010.Google ScholarCross Ref
- C. Rother, L. Bordeaux, Y. Hamadi, and A. Black. Autocollage. In in ACM Transactions on Graphics (SIGGRAPH), 2006. Google ScholarDigital Library
- G. A. Rousselet, M. J.-M. Macé, and M. Fabre-Thorpe. Is it an animal? is it a human face? fast processing in upright and inverted natural scenes. Journal of Vision, 3:440--455, 2003.Google ScholarCross Ref
- X. Sun, H. Yao, R. Ji, and S. Liu. Photo assessment based on computatinal visual attention model. In ACM Multimedia, pages 541--544, 2009. Google ScholarDigital Library
- B. W. Tatler, R. J. Baddeley, and I. D. Gilchrist. Visual correlates of fixation selection: effects of scale and time. Vision Research, 45:643--659, 2005.Google ScholarCross Ref
- M. Tokumaru, N. Muranaka, and S. Imanishi. Color design support system considering coor harmony. In IEEE International Conference on Fuzzy Systems, pages 378--383, 2002.Google Scholar
- A. Torralba and A. Oliva. Depth estimation from image structure. IEEE Pattern Analysis and Machine Intelligence, 24(9):1226--1238, 2002. Google ScholarDigital Library
- A. Torralba and A. Oliva. Statistics of natural image catagories. network, 14:391--421, 2003.Google Scholar
- A. Torralba, A. Oliva, M. Castelhano, and J. Henderson. Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. Psychological review, 113(4):766--786, 2006.Google ScholarCross Ref
- G. Underwood and T. Foulsham. Visual saliency and semantic incongruency influence eye movements when inspecting pictures. The Quarterly journal of experimental psychology, 59(11):1931--1949, 2006.Google Scholar
- P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In CVPR, 2001.Google ScholarCross Ref
- C.-G. Yeh, Y. Ho, B. Barsky, and M. Ouhyoung. Personalized photograph ranking and selection system. In ACM Multimedia, 2010. Google ScholarDigital Library
- Q. Zhao and C. Koch. Learning a saliency map using fixated locations in natural scenes. Journal of Vision, 11(3):1--15, 2011.Google ScholarCross Ref
Index Terms
- Prediction of the inter-observer visual congruency (IOVC) and application to image ranking
Recommendations
Human Visual Scanpath Prediction Based on RGB-D Saliency
ICIGP '18: Proceedings of the 2018 International Conference on Image and Graphics ProcessingHuman visual perception is considered as a dynamic process of information acquisition, while the visual scanpath can clearly reflect the shift of our eye fixations. In the previous study of visual attention, researchers generally do the saliency ...
Visual feature extraction via eye tracking for saliency driven 2D/3D registration
ETRA '04: Proceedings of the 2004 symposium on Eye tracking research & applicationsThis paper presents a new technique for extracting visual saliency from experimental eye tracking data. An eye-tracking system is employed to determine which features that a group of human observers considered to be salient when viewing a set of video ...
Visual saliency model based on crowdsourcing eye tracking data and its application in visual design
AbstractThe visual saliency models based on low-level features of an image have the problem of low accuracy and scalability, while the visual saliency models based on deep neural networks can effectively improve the prediction performance, but require a ...
Comments