Darshan Santani
Rui Hu
ABSTRACT
In the recent past, there has been interest in characterizing the physical and social ambiance of urban spaces to understand how people perceive and form impressions of these environments based on physical and psychological constructs. Building on our earlier work on characterizing ambiance of indoor places, we present a methodology to automatically infer impressions of place ambiance, using generic deep learning features extracted from images publicly shared on Foursquare. We base our methodology on a corpus of 45,000 images from 300 popular places in six cities on Foursquare. Our results indicate the feasibility to automatically infer place ambiance with a maximum R2 of 0.53 using features extracted from a pre-trained convolutional neural network. We found that features extracted from deep learning with convolutional nets consistently outperformed individual and combinations of several low-level image features (including Color, GIST, HOG and LBP) to infer all the studied 13 ambiance dimensions. Our work constitutes a first study to automatically infer ambiance impressions of indoor places from deep features learned from images shared on social media.
- Julie Baker, Dhruv Grewal, and Ananthanarayanan Parasuraman. 1994. The Influence of Store Environment on Quality Inferences and Store Image. Journal of the Academy of Marketing Science 22, 4 (1994), 328--339.Google Scholar
Cross Ref
- Anna Bosch, Andrew Zisserman, and Xavier Munoz. 2007. Representing Shape with a Spatial Pyramid Kernel. In Proceedings of the 6th ACM International Conference on Image and Video Retrieval (CIVR '07). ACM, New York, NY, USA, 401--408. x978--1--59593--733--9 Google ScholarDigital Library
- Leo Breiman. 2001. Random Forests. Machine learning 45, 1 (2001), 5--32. Google ScholarDigital Library
- Oxford Dictionary. 2015. Definition of Ambiance in English. http://bit.ly/1dAy80r. (2015). Retrieved July 24, 2016.Google Scholar
- Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. 2014. DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition. In ICML. 647--655.Google Scholar
- Foursquare. 2015. Foursquare API August 2014 Update. https://developer.foursquare.com/docs/2014update. (2015). Retrieved July 24, 2016.Google Scholar
- L Graham and S Gosling. 2011. Can the Ambiance of a Place be Determined by the User Profiles of the People Who Visit It?. In Proceedings of AAAI International Conference on Weblogs and Social Media (ICWSM).Google Scholar
- Vincent Heung and Tianming Gu. 2012. Influence of Restaurant Atmospherics on Patron Satisfaction and Behavioral Intentions. International Journal of Hospitality Management 31, 4 (2012), 1167--1177.Google ScholarCross Ref
- Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional Architecture for Fast Feature Embedding. In Proceedings of the ACM International Conference on Multimedia. ACM, 675--678. Google ScholarDigital Library
- Frederic Jurie and Bill Triggs. 2005. Creating Efficient Codebooks for Visual Recognition. In Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV '05). IEEE Computer Society, Washington, DC, USA, 604--610. 0--7695--2334-X-01 Google ScholarDigital Library
- Nikhil Naik, Jade Philipoom, Ramesh Raskar, and Cesar Hidalgo. 2014. Streetscore -- Predicting the Perceived Safety of One Million Streetscapes. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW '14). IEEE Computer Society, Washington, DC, USA, 793--799. 978--1--4799--4308--1 Google ScholarDigital Library
- Timo Ojala, Matti Pietikainen, and Topi Maenpaa. 2002. Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24, 7 (July 2002), 971--987. 0162--8828 Google ScholarDigital Library
- Aude Oliva and Antonio Torralba. 2001. Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope. International Journal of Computer Vision 42, 3 (2001), 145--175. Google ScholarDigital Library
- Vicente Ordonez and Tamara L Berg. 2014. Learning High-Level Judgments of Urban Perception. In European Conference on Computer Vision. Springer, 494--510.Google Scholar
- Lorenzo Porzi, Samuel Rota Bulò, Bruno Lepri, and Elisa Ricci. 2015. Predicting and Understanding Urban Perception with Convolutional Neural Networks. In Proceedings of the 23rd Annual ACM Conference on Multimedia Conference. ACM, 139--148. Google ScholarDigital Library
- Daniele Quercia, Neil Keith O'Hare, and Henriette Cramer. 2014. Aesthetic Capital: What makes London Look Beautiful, Quiet, and Happy?. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM. Google ScholarDigital Library
- Ali Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. 2014. CNN Features off-the-shelf: an Astounding Baseline for Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 806--813. Google ScholarDigital Library
- Miriam Redi, Daniele Quercia, Lindsay Graham, and Samuel Gosling. 2015. Like Partying? Your Face Says It All. Predicting the Ambiance of Places with Profile Pictures. In Ninth International AAAI Conference on Web and Social Media.Google Scholar
- Salvador Ruiz-Correa, Darshan Santani, and Daniel Gatica-Perez. 2014. The Young and the City: Crowdsourcing Urban Awareness in a Developing Country. In Proceedings of the First International Conference on IoT in Urban Space (URB-IOT '14). ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), ICST, Brussels, Belgium, Belgium, 74--79. 978--1--63190-037--2 Google ScholarDigital Library
- Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, and others. 2015. Imagenet Large Scale Visual Recognition Challenge. International Journal of Computer Vision 115, 3 (2015), 211--252. Google ScholarDigital Library
- Philip Salesses, Katja Schechtner, and Cesar A. Hidalgo. 2013. The Collaborative Image of The City: Mapping the Inequality of Urban Perception. PLoS ONE 8, 7 (07 2013).Google Scholar
- Darshan Santani and Daniel Gatica-Perez. 2015. Loud and Trendy: Crowdsourcing Impressions of Social Ambiance in Popular Indoor Urban Places. In Proceedings of the 23rd Annual ACM Conference on Multimedia Conference. ACM, 211--220. Google ScholarDigital Library
- Darshan Santani, Salvador Ruiz-Correa, and Daniel Gatica-Perez. 2015. Looking at Cities in Mexico with Crowds. In Proceedings of the 2015 Annual Symposium on Computing for Development. ACM, 127--135. Google ScholarDigital Library
- Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going Deeper with Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1--9.Google ScholarCross Ref
- Lou W Turley and Ronald E Milliman. 2000. Atmospheric Effects on Shopping Behavior: A Review of the Experimental Evidence. Journal of Business Research 49, 2 (2000), 193--211.Google ScholarCross Ref
- Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. 2014. Learning Deep Features for Scene Recognition Using Places Database. In Advances in Neural Information Processing Systems. 487--495. Google ScholarDigital Library
Index Terms
- InnerView: Learning Place Ambiance from Social Media Images
Recommendations
Venues in Social Media: Examining Ambiance Perception Through Scene Semantics
MM '17: Proceedings of the 25th ACM international conference on MultimediaWe address the question of what visual cues, including scene objects and demographic attributes, contribute to the automatic inference of perceived ambiance in social media venues. We first use a state-of-art, deep scene semantic parsing method and a ...
Loud and Trendy: Crowdsourcing Impressions of Social Ambiance in Popular Indoor Urban Places
MM '15: Proceedings of the 23rd ACM international conference on MultimediaNew research cutting across architecture, urban studies, and psychology is contextualizing the understanding of urban spaces according to the perceptions of their inhabitants. One fundamental construct that relates place and experience is ambiance, ...
The Spirit of the City: Using Social Media to Capture Neighborhood Ambiance
Place ambiance has a huge influence over how we perceive places. Despite its importance, ambiance has been crucially overlooked by urban planners and scientists alike, not least because it is difficult to record and analyze at scale. We explored the ...
Comments