Donghyeon Won
ABSTRACT
We develop a novel visual model which can recognize protesters, describe their activities by visual attributes and estimate the level of perceived violence in an image. Studies of social media and protests use natural language processing to track how individuals use hashtags and links, often with a focus on those items' diffusion. These approaches, however, may not be effective in fully characterizing actual real-world protests (e.g., violent or peaceful) or estimating the demographics of participants (e.g., age, gender, and race) and their emotions. Our system characterizes protests along these dimensions. We have collected geotagged tweets and their images from 2013-2017 and analyzed multiple major protest events in that period. A multi-task convolutional neural network is employed in order to automatically classify the presence of protesters in an image and predict its visual attributes, perceived violence and exhibited emotions. We also release the UCLA Protest Image Dataset, our novel dataset of 40,764 images (11,659 protest images and hard negatives) with various annotations of visual attributes and sentiments. Using this dataset, we train our model and demonstrate its effectiveness. We also present experimental results from various analysis on geotagged image data in several prevalent protest events. Our dataset will be made accessible at https://www.sscnet.ucla.edu/comm/jjoo/mm-protest/.
- Brandon Amos, Bartosz Ludwiczuk, and Mahadev Satyanarayanan. 2016. Open-Face: A general-purpose face recognition library with mobile applications. Technical Report. Carnegie Mellon University-CS-16--118, Carnegie Mellon University School of Computer Science.Google Scholar
- Lefteris Anastasopoulos and Jake Williams. 2016. Identifying violent protest activity with scalable machine learning *. (2016). http://scholar.harvard.edu/janastasGoogle Scholar
- Pablo Barberá, Ning Wang, Richard Bonneau, John T. Jost, Jonathan Nagler, Joshua Tucker, and Sandra González-Bailón. 2015. The Critical Periphery in the Growth of Social Protests. PloS ONE 10, 11 (2015), 1--15.Google ScholarCross Ref
- Marco Bastos, Raquel Recuero, and Gabriela Zago. 2014. Taking tweets to the streets: A spatial analysis of the Vinegar Protests in Brazil. First Monday 19, 3 (2014), 1--27.Google ScholarCross Ref
- Ralph Allan Bradley and Milton E Terry. 1952. Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika 39, 3/4 (1952), 324--345.Google ScholarCross Ref
- Markus Brenner and Ebroul Izquierdo. 2012. Social event detection and retrieval in collaborative photo collections. In Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. ACM, 21. Google ScholarDigital Library
- Liang-Hua Chen, Hsi-Wen Hsu, Li-Yun Wang, and Chih-Wen Su. 2011. Violence detection in movies. In Computer Graphics, Imaging and Visualization (CGIV), 2011 Eighth International Conference on. IEEE, 119--124. Google ScholarDigital Library
- Fillipe DM De Souza, Guillermo C Chavez, Eduardo A do Valle Jr, and Arnaldo de A Araújo. 2010. Violence detection in video using spatio-temporal features. In Graphics, Patterns and Images (SIBGRAPI), 2010 23rd SIBGRAPI Conference on. IEEE, 224--230. Google ScholarDigital Library
- Jesse Driscoll and Zachary C. Steinert-Threlkeld. 2017. Structure, Agency, Hege- mony, and Action: Ukrainian Nationalism in East Ukraine. (2017).Google Scholar
- Ruben Enikolopov, Alexey Makarin, and Maria Petrova. 2016. Social Media and Protest Participation: Evidence from Russia. (2016).Google Scholar
- Matthew Feinberga, Robb Willer, and Chlose Kovacheff. 2017. Extreme Protest Tactics Reduce Popular Support for Social Movements. (2017).Google Scholar
- Dana R. Fisher. 2014. Studying Large-Scale Protest: Understanding Mobilization and Participation at the People's Climate March. (2014).Google Scholar
- Debashis Ganguly, Mohammad H Mofrad, and Adriana Kovashka. 2017. Detecting Sexually Provocative Images. In Winter Conference on Applications of Computer Vision (WACV). IEEE, 660--668.Google Scholar
- CJ Hutto Eric Gilbert. 2014. VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text. In Eighth International Conference on Weblogs and Social Media (ICWSM-14). Available at (20/04/16) http://comp.social.gatech.edu/papers/icwsm14.vader.hutto. pdf.Google Scholar
- Sandra Gonzalez-Bailon, Javier Borge-Holthoefer, and Yamir Moreno. 2013. Broadcasters and Hidden Influentials in Online Protest Diffusion. American Behavioral Scientist 57, 7 (mar 2013), 943--965.Google ScholarCross Ref
- Helmut Grabner, Fabian Nater, Michel Druey, and Luc Van Gool. 2013. Visual interestingness in image sequences. In Proceedings of the 21st ACM international conference on Multimedia. ACM, 1017--1026. Google ScholarDigital Library
- Tal Hassner, Yossi Itcher, and Orit Kliper-Gross. 2012. Violent flows: Real-time detection of violent crowd behavior. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on. IEEE, 1--6.Google ScholarCross Ref
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770--778.Google ScholarCross Ref
- Phillip Isola, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. 2011. What makes an image memorable?. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 145--152. Google ScholarDigital Library
- Jungseock Joo, Weixin Li, Francis Steen, and Song-Chun Zhu. 2014. Visual Persuasion: Inferring Communicative Intents of Images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 216--223. Google ScholarDigital Library
- Jungseock Joo, Francis F Steen, and Song-Chun Zhu. 2015. Automated facial trait judgment and election outcome prediction: Social dimensions of face. In Proceedings of the IEEE International Conference on Computer Vision. 3712--3720. Google ScholarDigital Library
- Adriana Kovashka, Devi Parikh, and Kristen Grauman. 2012. Whittlesearch: Image search with relative attribute feedback. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 2973--2980. Google ScholarDigital Library
- Timur Kuran. 1989. Sparks and Prairie Fires: A Theory of Unanticipated Political Revolution. Public Choice 61, 1 (1989), 41--74.Google ScholarCross Ref
- Kalev H. Leetaru, Shaowen Wang, Guofeng Cao, Anand Padmanabhan, and Eric Shook. 2013. Mapping the global Twitter heartbeat: The geography of Twitter. First Monday 18, 5--6 (2013), 1--33.Google ScholarCross Ref
- Andrew T. Little. 2015. Communication Technology and Protest. Journal of Politics 78, 1 (2015), 152--166.Google ScholarCross Ref
- Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015. Deep learning face attributes in the wild. In Proceedings of the IEEE International Conference on Computer Vision. 3730--3738. Google ScholarDigital Library
- Susanne Lohmann. 1994. The Dynamics of Informational Cascades: The Monday Demonstrations in Leipzig, East Germany, 1989--91. World Politics 47, 1 (1994), 42--101.Google ScholarCross Ref
- Virginia Lopez and Jonathan Watts. 2017. Deaths and injuries reported amid "mother of all marches" in Venezuela. (apr 2017).Google Scholar
- Doug McAdam. 1986. Recruitment to High-Risk Activism: The Case of Freedom Summer. Amer. J. Sociology 92, 1 (1986), 64--90.Google ScholarCross Ref
- Jonathan Mercer. 2010. Emotional Beliefs. International Organization 64, 01 (jan 2010), 1.Google ScholarCross Ref
- Edward N. Muller and Karl-Dieter Opp. 1986. Rational Choice and Rebellious Collective Action. The American Political Science Review 80, 2 (1986), 471--488.Google ScholarCross Ref
- Enrique Bermejo Nievas, Oscar Deniz Suarez, Gloria Bueno García, and Rahul Sukthankar. 2011. Violence detection in video using computer vision techniques. In International conference on Computer analysis of images and patterns. Springer, 332--339. Google ScholarDigital Library
- Devi Parikh and Kristen Grauman. 2011. Relative attributes. In Computer Vision (ICCV), 2011 IEEE International Conference on. IEEE, 503--510. Google ScholarDigital Library
- Wendy Pearlman. 2013. Emotions and the Microfoundations of the Arab Uprisings. Perspectives on Politics 11, 02 (may 2013), 387--409.Google ScholarCross Ref
- Georgios Petkos, Symeon Papadopoulos, and Yiannis Kompatsiaris. 2012. Social event detection using multimodal clustering and integrating supervisory signals. In Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. ACM, 23. Google ScholarDigital Library
- Georgios Petkos, Symeon Papadopoulos, Emmanouil Schinas, and Yiannis Kompatsiaris. 2014. Graph-based multimodal clustering for social event detection in large collections of images. In International Conference on Multimedia Modeling. Springer, 146--158. Google ScholarDigital Library
- Shengsheng Qian, Tianzhu Zhang, Changsheng Xu, and M Shamim Hossain. 2015. Social event classification via boosted multimodal supervised latent dirichlet allocation. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 11, 2 (2015), 27. Google ScholarDigital Library
- Miriam Redi, Neil O'Hare, Rossano Schifanella, Michele Trevisiol, and Alejandro Jaimes. 2014. 6 seconds of sound and vision: Creativity in micro-videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4272--4279. Google ScholarDigital Library
- Timo Reuter, Symeon Papadopoulos, Giorgos Petkos, Vasileios Mezaris, Yiannis Kompatsiaris, Philipp Cimiano, Christopher de Vries, and Shlomo Geva. 2013. Social event detection at mediaeval 2013: Challenges, datasets, and evaluation. In Proceedings of the MediaEval 2013 Multimedia Benchmark Workshop Barcelona, Spain, October 18-19, 2013.Google Scholar
- Jacob N Shapiro and David A Siegel. 2015. Coordination and security: How mobile communications affect insurgency. Journal of Peace Research 52, 3 (feb 2015), 1--11.Google ScholarCross Ref
- Stuart Soroka, Peter Loewen, Patrick Fournier, and Daniel Rubenson. 2016. The Impact of News Photos on Support for Military Action. Political Communication 33, 4 (2016), 563--582.Google ScholarCross Ref
- Zachary C. Steinert-Threlkeld. 2017. Spontaneous Collective Action: Peripheral Mobilization During the Arab Spring. American Political Science Review 111, 02 (2017).Google ScholarCross Ref
- Zachary C. Steinert-Threlkeld, Delia Mocanu, Alessandro Vespignani, and James Fowler. 2015. Online social networks and offline protest. EPJ Data Science 4, 1 (2015), 19.Google ScholarCross Ref
- Zeynep Tufekci. 2014. Big Questions for Social Media Big Data: Representative- ness, Validity and Other Methodological Pitfalls Pre-print. In Proceedings of the 8th International AAAI Conference on Weblogs and Social Media. Ann Arbor.Google Scholar
- Gordon Tullock. 1971. The Paradox of Revolution. Public Choice 11 (1971), 89--99.Google ScholarCross Ref
- Yu Wang, Yuncheng Li, and Jiebo Luo. 2016. Deciphering the 2016 US Presidential Campaign in the Twitter Sphere: A Comparison of the Trumpists and Clintonists. In Tenth International AAAI Conference on Web and Social Media.Google Scholar
- Guobin Yang. 2000. Achieving Emotions in Collective Action: Emotional Processes and Movement Mobilization in the 1989 Chinese Student Movement. The Sociological Quarterly 41, 4 (sep 2000), 593--614.Google ScholarCross Ref
- Xiaoshan Yang, Tianzhu Zhang, and Changsheng Xu. 2015. Cross-domain feature learning in multimedia. IEEE Transactions on Multimedia 17, 1 (2015), 64--78.Google ScholarCross Ref
- Quanzeng You, Liangliang Cao, Yang Cong, Xianchao Zhang, and Jiebo Luo. 2015. A multifaceted approach to social multimedia-based prediction of elections. IEEE Transactions on Multimedia 17, 12 (2015), 2271--2280.Google ScholarDigital Library
Index Terms
- Protest Activity Detection and Perceived Violence Estimation from Social Media Images
Recommendations
Social and Political Event Analysis based on Rich Media
MM '18: Proceedings of the 26th ACM international conference on MultimediaThis tutorial aims to provide a comprehensive overview on the applications of rich social media data for real world social and political event analysis, which is a new emerging topic in multimedia research. We will discuss the recent evolution of social ...
Mainstream media behavior analysis on Twitter: a case study on UK general election
HT '13: Proceedings of the 24th ACM Conference on Hypertext and Social MediaWith the development of social media tools such as Facebook and Twitter, mainstream media organizations including newspapers and TV media have played an active role in engaging with their audience and strengthening their influence on the recently ...
Social media signal detection using tweets volume, hashtag, and sentiment analysis
Social Media is a well-known platform for users to create, share and check the new information. The world becomes a global village because of the utilization of internet and social media. The data present on Twitter contains information of great ...
Comments