Sonal Goel
ABSTRACT
The data generated on social media sites continues to grow at an increasing rate with more than 36% of tweets containing images making the dominance of multimedia content evidently visible. This massive user generated content has become a reflection of world events. In order to enhance the ability and effectiveness to consume this plethora of data, summarization of these events is needed. However, very few studies have exploited the images attached with social media events to summarize them using "mid-level visual elements". These are the entities which are both representative and discriminative to the target dataset besides being human-readable and hence more informative.
In this paper we propose a methodology for visual event summarization by extracting mid-level visual elements from images associated with social media events on Twitter (#VisualHashtags). The key research question is Which elements can visually capture the essence of a viral event?, hence explain its virality, and summarize it. Compared to the existing approaches of visual event summarization on social media data, we aim to discover #VisualHashtags, i.e., meaningful patches that can become the visual analog of a regular text hashtag that Twitter generates. Our algorithm incorporates a multi-stage filtering process and social popularity based ranking to discover mid-level visual elements, which overcomes the challenges faced by direct application of the existing methods.
We evaluate our approach on a recently collected social media event dataset, comprising of 20,084 images. We evaluate the quality of #VisualHashtags extracted by conducting a user-centered evaluation where users are asked to rate the relevance of the resultant patches w.r.t. the event and the quality of the patch in terms of how meaningful it is. We also do a quantitative evaluation on the results. We show a high search space reduction of 93% in images and 99% in patches after summarization. Further, we get a 83% of purity in the resultant patches with a data coverage of 18%
- Omar Alonso and Kyle Shiells. 2013. Timelines as summaries of popular scheduled events Proceedings of the 22nd International Conference on World Wide Web. ACM, 1037--1044. Google Scholar
Digital Library
- Jingwen Bian, Yang Yang, and Tat-Seng Chua. 2013. Multimedia summarization for trending topics in microblogs Proceedings of the 22nd ACM international conference on Conference on information & knowledge management. ACM, 1807--1812. Google ScholarDigital Library
- Jingwen Bian, Yang Yang, Hanwang Zhang, and Tat-Seng Chua. 2015. Multimedia summarization for social events in microblog stream. IEEE Transactions on Multimedia Vol. 17, 2 (2015), 216--228.Google ScholarDigital Library
- Paulo Cavalin, Flavio Figueiredo, Maıra de Bayser, and Claudio Pinhanez. 2016. Organizing Images from Social Media to Monitor Real-World Events. (october. 2016). http://gibis.unifesp.br/sibgrapi16Google Scholar
- Deepayan Chakrabarti and Kunal Punera. 2011. Event Summarization Using Tweets. ICWSM Vol. 11 (2011), 66--73.Google Scholar
- Freddy Chong Tat Chua and Sitaram Asur. 2013. Automatic Summarization of Events from Social Media. ICWSM.Google Scholar
- Veronika Coltheart. 1999. Fleeting memories: Cognition of brief visual stimuli. Mit Press.Google Scholar
- Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, Vol. Vol. 1. IEEE, 886--893. Google ScholarDigital Library
- Carl Doersch, Abhinav Gupta, and Alexei A Efros. 2013. Mid-level visual element discovery as discriminative mode seeking Advances in neural information processing systems. 494--502. Google ScholarDigital Library
- Carl Doersch, Saurabh Singh, Abhinav Gupta, Josef Sivic, and Alexei Efros. 2012. What makes paris look like paris? ACM Transactions on Graphics Vol. 31, 4 (2012). Google ScholarDigital Library
- Yao Li, Lingqiao Liu, Chunhua Shen, and Anton van den Hengel. 2016. Mining mid-level visual patterns with deep CNN activations. International Journal of Computer Vision (2016), 1--21. Google ScholarDigital Library
- Philip J McParlane, Andrew James McMinn, and Joemon M Jose. 2014. Picture the scene...;: Visually Summarising Social Media Events Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. ACM, 1459--1468. Google ScholarDigital Library
- Vishal Monga and Brian L Evans. 2006. Perceptual image hashing via feature points: performance evaluation and tradeoffs. IEEE Transactions on Image Processing Vol. 15, 11 (2006), 3452--3465. Google ScholarDigital Library
- Jeffrey Nichols, Jalal Mahmud, and Clemens Drews. 2012. Summarizing sporting events using twitter. In Proceedings of the 2012 ACM international conference on Intelligent User Interfaces. ACM, 189--198. Google ScholarDigital Library
- Pedro O Pinheiro, Ronan Collobert, and Piotr Dollar. 2015. Learning to segment object candidates. In Advances in Neural Information Processing Systems. 1990--1998. Google ScholarDigital Library
- Konstantinos Rematas, Basura Fernando, Frank Dellaert, and Tinne Tuytelaars. 2015. Dataset fingerprints: Exploring image collections through data mining Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4867--4875.Google Scholar
- Zhaochun Ren, Shangsong Liang, Edgar Meij, and Maarten de Rijke. 2013. Personalized time-aware tweets summarization. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. ACM, 513--522. Google ScholarDigital Library
- Manos Schinas, Symeon Papadopoulos, Yiannis Kompatsiaris, and Pericles A Mitkas. 2015. Visual event summarization on social media using topic modelling and graph-based ranking algorithms. In Proceedings of the 5th ACM on International Conference on Multimedia Retrieval. ACM, 203--210. Google ScholarDigital Library
- Beaux P Sharifi, David I Inouye, and Jugal K Kalita. 2013. Summarization of twitter microblogs. Comput. J. (2013), bxt109.Google Scholar
- Chao Shen, Fei Liu, Fuliang Weng, and Tao Li. 2013. A Participant-based Approach for Event Summarization Using Twitter Streams. HLT-NAACL. 1152--1162.Google Scholar
- LiMin Wang, Yu Qiao, and Xiaoou Tang. 2013. Motionlets: Mid-level 3d parts for human motion recognition Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2674--2681. Google ScholarDigital Library
- Arkaitz Zubiaga, Damiano Spina, Enrique Amigó, and Julio Gonzalo. 2012. Towards real-time summarization of scheduled events from twitter streams Proceedings of the 23rd ACM conference on Hypertext and social media. ACM, 319--320. Google ScholarDigital Library
Index Terms
- #VisualHashtags: Visual Summarization of Social Media Events Using Mid-Level Visual Elements
Recommendations
Seeing the Big Picture from Microblogs: Harnessing Social Signals for Visual Event Summarization
IUI '15: Proceedings of the 20th International Conference on Intelligent User InterfacesWe propose an approach to automatically select a set of representative images to generate a concise visual summary of a real-world event from the Tumblr microblogging platform. Central to our approach is a unified graph model with heterogeneous nodes ...
SentiStory: multi-grained sentiment analysis and event summarization with crowdsourced social media data
The massive social media data bring timely, multi-dimensional and rich information. Recently, many researchers have worked on event summarization with crowdsourced social media data. While existing works mostly focus on text-based summary, they only ...
Social sentiment sensor: a visualization system for topic detection and topic sentiment analysis on microblog
As a new form of social media, microblogging provides platform sharing, wherein users can share their feelings and ideas on certain topics. Bursty topics from microblogs are the results of the emerging issues that instantly attract more followers and ...
Comments