skip to main content
survey

A Survey on Assessment and Ranking Methodologies for User-Generated Content on the Web

Authors Info & Claims
Published:22 December 2015Publication History
Skip Abstract Section

Abstract

User-generated content (UGC) on the Web, especially on social media platforms, facilitates the association of additional information with digital resources; thus, it can provide valuable supplementary content. However, UGC varies in quality and, consequently, raises the challenge of how to maximize its utility for a variety of end-users. This study aims to provide researchers and Web data curators with comprehensive answers to the following questions: What are the existing approaches and methods for assessing and ranking UGC? What features and metrics have been used successfully to assess and predict UGC value across a range of application domains? What methods can be effectively employed to maximize that value? This survey is composed of a systematic review of approaches for assessing and ranking UGC: results are obtained by identifying and comparing methodologies within the context of short text-based UGC on the Web. Existing assessment and ranking approaches adopt one of four framework types: the community-based framework takes into consideration the value assigned to content by a crowd of humans, the end-user--based framework adapts and personalizes the assessment and ranking process with respect to a single end-user, the designer-based framework encodes the software designer’s values in the assessment and ranking method, and the hybrid framework employs methods from more than one of these types. This survey suggests a need for further experimentation and encourages the development of new approaches for the assessment and ranking of UGC.

References

  1. Fabian Abel, Ilknur Celik, Geert-Jan Houben, and Patrick Siehndel. 2011. Leveraging the semantics of tweets for adaptive faceted search on Twitter. In Proceedings of the 10th International Conference on The Semantic Web, Volume Part I (ISWC’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Eugene Agichtein, Carlos Castillo, Debora Donato, Aristides Gionis, Gilad Mishne, Eugene Agichtein, Carlos Castillo, Debora Donato, Aristides Gionis, and Gilad Mishne. 2008. Finding high-quality content in social media with an application to community-based question answering. In Proceedings of the International Conference on Web Search and Data Mining (WSDM’08). Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Omar Alonso, Catherine C. Marshall, and Marc Najork. 2013. Are some tweets more interesting than others? #Hardquestion. In Proceedings of the Symposium on Human-Computer Interaction and Information Retrieval (HCIR’13). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Jisun An, Meeyoung Cha, Krishna Gummadi, Jon Crowcroft, and Daniele Quercia. 2012. Visualizing media bias through Twitter. In Proceedings of the 6th International AAAI Conference on Weblog and Social Media (ICWSM’12).Google ScholarGoogle Scholar
  5. Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2013. Steering user behavior with badges. In Proceedings of the 22nd International Conference on World Wide Web (WWW’13). Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Marlene Asselin, Teresa Dobson, Eric M. Meyers, Cristina Teixiera, and Linda Ham. 2011. Learning from YouTube: An analysis of information literacy in user discourse. In Proceedings of the 2011 iConference (iConference’11). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Hila Becker, Dan Iter, Mor Naaman, and Luis Gravano. 2012. Identifying content for planned events across social media sites. In Proceedings of the 5th ACM International Conference on Web Search and Data Mining (WSDM’12). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Hila Becker, Mor Naaman, and Luis Gravano. 2011a. Beyond trending topics: Real-world event identification on Twitter. In Proceedings of the 5th International AAAI Conference on Weblogs and Social Media (ICWSM’11).Google ScholarGoogle Scholar
  9. Hila Becker, Mor Naaman, and Luis Gravano. 2011b. Selecting quality Twitter content for events. In Proceedings of the 5th International AAAI Conference on Weblogs and Social Media (ICWSM’11).Google ScholarGoogle Scholar
  10. Gerard Beenen, Kimberly Ling, Xiaoqing Wang, Klarissa Chang, Dan Frankowski, Paul Resnick, and Robert E. Kraut. 2004. Using social psychology to motivate contributions to online communities. In Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work (CSCW’04). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Michael S. Bernstein, Bongwon Suh, Lichan Hong, Jilin Chen, Sanjay Kairam, and Ed H. Chi. 2010. Eddi: Interactive topic-based browsing of social status streams. In Proceedings of the 23rd Annual ACM Symposium on User Interface Software and Technology (UIST’10). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Jiang Bian, Yandong Liu, Eugene Agichtein, and Hongyuan Zha. 2008. Finding the right facts in the crowd: Factoid question answering over social media. In Proceedings of the 17th International Conference on World Wide Web (WWW’08). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Jiang Bian, Yandong Liu, Ding Zhou, Eugene Agichtein, and Hongyuan Zha. 2009. Learning to recognize reliable users and content in social media with coupled mutual reinforcement. In Proceedings of the 18th International Conference on World Wide Web (WWW’09). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Avrim Blum and Tom Mitchell. 1998. Combining labeled and unlabeled data with co-training. In Proceedings of the 11th Annual Conference on Computational Learning Theory (COLT’98). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Matthew Burgess, Alessandra Mazzia, Eytan Adar, and Michael Cafarella. 2013. Leveraging noisy lists for social feed ranking. In Proceedings of the 7th International AAAI Conference on Weblog and Social Media (ICWSM’13).Google ScholarGoogle Scholar
  16. Moira Burke, Elisabeth Joyce, Tackjin Kim, Vivek An, and Robert Kraut. 2007. Introductions and requests: Rhetorical strategies that elicit response in online communities. In Proceedings of the 3rd International Conference on Communities and Technologies (C&T’’07).Google ScholarGoogle ScholarCross RefCross Ref
  17. Kevin R. Canini, Bongwon Suh, and Peter L. Pirolli. 2011. Finding credible information sources in social networks based on content and social structure. In Proceedings of the 2011 IEEE 3rd International Conference on Privacy, Security, Risk, and Trust and the 2011 IEEE 3rd International Conference on Social Computing. IEEE, Los Alamitos, CA.Google ScholarGoogle Scholar
  18. Carlos Castillo, Marcelo Mendoza, and Barbara Poblete. 2011. Information credibility on Twitter. In Proceedings of the 20th International Conference on World Wide Web (WWW’11). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Ruggiero Cavallo and Shaili Jain. 2012. Efficient crowdsourcing contests. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems, Volume 2 (AAMAS’12). Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Jilin Chen, Rowan Nairn, and Ed Chi. 2011. Speak little and well: Recommending conversations in online social streams. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’11). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Jilin Chen, Rowan Nairn, Les Nelson, Michael Bernstein, and Ed Chi. 2010. Short and tweet: Experiments on recommending content from information streams. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’10). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Munmun De Choudhury, Scott Counts, and Michael Gamon. 2012. Not all moods are created equal! Exploring human emotional states in social media. In Proceedings of the 6th International AAAI Conference on Weblogs and Social Media.Google ScholarGoogle Scholar
  23. Cristian Danescu-Niculescu-Mizil, Gueorgi Kossinets, Jon Kleinberg, and Lillian Lee. 2009. How opinions are received by online communities: A case study on Amazon.com helpfulness votes. In Proceedings of the 18th International Conference on World Wide Web (WWW’09). Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Nicholas Diakopoulos. 2015. The editor’s eye: Curation and comment relevance on the New York Times. In Proceedings of the ACM 2015 Conference on Computer Supported Cooperative Work (CSCW’15). Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Nicholas Diakopoulos, Munmun De Choudhury, and Mor Naaman. 2012. Finding and assessing social media information sources in the context of journalism. In Proceedings of the 2012 ACM Annual conference on Human Factors in Computing Systems (CHI’12). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Nicholas Diakopoulos and Mor Naaman. 2011. Towards quality discourse in online news comments. In Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work (CSCW’11). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Siamak Faridani, Ephrat Bitton, Kimiko Ryokai, and Ken Goldberg. 2010. Opinion space: A scalable tool for browsing online comments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’10). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Deen G. Freelon, Travis Kriplean, Jonathan Morgan, W. Lance Bennett, and Alan Borning. 2012. Facilitating diverse political engagement with the Living Voters Guide. Journal of Information Technology and Politics 9, 3, 279--297.Google ScholarGoogle ScholarCross RefCross Ref
  29. Anindya Ghose and Panagiotis G. Ipeirotis. 2007. Designing novel review ranking systems: Predicting the usefulness and impact of reviews. In Proceedings of the 9th International Conference on Electronic Commerce (ICEC’07). Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Anindya Ghose and Panagiotis G. Ipeirotis. 2011. Estimating the helpfulness and economic impact of product reviews: Mining text and reviewer characteristics. IEEE Transactions on Knowledge and Data Engineering 23, 10, 1498--1512. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Arpita Ghosh. 2012. Social computing and user-generated content: A game-theoretic approach. In Proceedings of ACM SIGecom Exchanges (SIGecom’12). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Arpita Ghosh and Patrick Hummel. 2011. A game-theoretic analysis of rank-order mechanisms for user-generated content. In Proceedings of the 12th ACM Conference on Electronic Commerce (EC’11). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Arpita Ghosh and Preston McAfee. 2011. Incentivizing high-quality user-generated content. In Proceedings of the 20th International Conference on World Wide Web (WWW’11). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Arpita Ghosh and Preston McAfee. 2012. Crowdsourcing with endogenous entry. In Proceedings of the 21st International Conference on World Wide Web (WWW’12). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Giorgos Giannopoulos, Ingmar Weber, Alejandro Jaimes, and Timos Sellis. 2012. Diversifying user comments on news articles. In Web Information Systems Engineering—WISE 2012. Lecture Notes in Computer Science, Vol. 7651. Springer, 100--113. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Eric Gilbert and Karrie Karahalios. 2010. Understanding deja reviewers. In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work (CSCW’10). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Manish Gupta, Rui Li, Zhijun Yin, and Jiawei Han. 2010. Survey on social tagging techniques. ACM SIGKDD Explorations Newsletter 12, 1, 58--72. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Catherine E. Hall and Michael A. Zarro. 2011. What do you call it? A comparison of library-created and user-created tags. In Proceedings of the 11th Annual International ACM/IEEE Joint Conference on Digital Libraries (JCDL’11). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. F. Maxwell Harper, Daniel Moy, and Joseph A. Konstan. 2009. Facts or friends? Distinguishing informational and conversational questions in social Q&A sites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’’09). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. F. Maxwell Harper, Daphne Raban, Sheizaf Rafaeli, and Joseph A. Konstan. 2008. Predictors of answer quality in online Q&A sites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’’08). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Liangjie Hong, Ron Bekkerman, Joseph Adler, and Brian D. Davison. 2012. Learning to rank social update streams. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’12). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Liangjie Hong, Ovidiu Dan, and Brian D. Davison. 2011. Predicting popular messages in Twitter. In Proceedings of the 20th International Conference Companion on World Wide Web (WWW’11). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Chiao-Fang Hsu, Elham Khabiri, and James Caverlee. 2009. Ranking comments on the social Web. In Proceedings of the 2009 International Conference on Computational Science and Engineering, Volume 4 (CSE’09). IEEE, Los Alamitos, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Yuheng Hu, Shelly D. Farnham, and Andrés Monroy-Hernández. 2013. Whoo.Ly: Facilitating information seeking for hyperlocal communities using social media. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Bernardo A. Huberman, Daniel M. Romero, and Fang Wu. 2009. Crowdsourcing, attention and productivity. Journal of Information Science 35, 6, 758--765. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Jessica Hullman, Nicholas Diakopoulos, Elaheh Momeni, and Eytan Adar. 2015. Content, context, and critique: Commenting on a data visualization blog. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW’15). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Shaili Jain, Yiling Chen, and David C. Parkes. 2009. Designing incentives for online question and answer forums. In Proceedings of the 10th ACM Conference on Electronic Commerce (EC’09). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Jiwoon Jeon, W. Bruce Croft, Joon Ho Lee, and Soyeon Park. 2006. A framework to predict the quality of answers with non-textual features. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’06). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Nitin Jindal and Bing Liu. 2007. Analyzing and detecting review spam. In Proceedings of the 2007 7th IEEE International Conference on Data Mining (ICDM’07). Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Nitin Jindal and Bing Liu. 2008. Opinion spam and analysis. In Proceedings of the 2008 International Conference on Web Search and Data Mining (WSDM’08). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Soo-Min Kim, Patrick Pantel, Tim Chklovski, and Marco Pennacchiotti. 2006. Automatically assessing review helpfulness. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP’06). Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Barbara Kitchenham. 2004. Procedures for Performing Systematic Reviews. Technical Report TR/SE-0401. Keele University, Keele, England.Google ScholarGoogle Scholar
  53. Ralf Krestel, Peter Fankhauser, and Wolfgang Nejdl. 2009. Latent Dirichlet allocation for tag recommendation. In Proceedings of the 3rd ACM Conference on Recommender Systems (RecSys’09). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Travis Kriplean, Jonathan Morgan, Deen Freelon, Alan Borning, and Lance Bennett. 2012a. Supporting reflective public thought with ConsiderIt. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (CSCW’12). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Travis Kriplean, Michael Toomim, Jonathan Morgan, Alan Borning, and Andrew Ko. 2012b. Is this what you meant? Promoting listening on the Web with Reflect. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Cliff Lampe and Paul Resnick. 2004. Slash(dot) and burn: Distributed moderation in a large online conversation space. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’04). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Cliff A.C. Lampe, Erik Johnston, and Paul Resnick. 2007. Follow the reader: Filtering comments on Slashdot. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’07). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. David Laniado and Peter Mika. 2010. Making sense of Twitter. In Proceedings of the 9th International Semantic Web Conference on the Semantic Web, Volume Part I (ISWC’10). Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Baoli Li, Yandong Liu, and Eugene Agichtein. 2008. CoCQA: Co-training over questions and answers with an application to predicting question subjectivity orientation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’08). Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Jiwei Li, Myle Ott, Claire Cardie, and Eduard Hovy. 2014. Towards a general rule for identifying deceptive opinion spam. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.Google ScholarGoogle ScholarCross RefCross Ref
  61. Xirong Li, Cees G. M. Snoek, and Marcel Worring. 2009. Learning social tag relevance by neighbor voting. IEEE Transactions on Multimedia 11, 7, 1310--1322. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Q. Vera Liao and Wai-Tat Fu. 2013. Beyond the filter bubble: Interactive effects of perceived threat and topic involvement on selective exposure to information. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Dong Liu, Xian-Sheng Hua, Linjun Yang, Meng Wang, and Hong-Jiang Zhang. 2009. Tag ranking. In Proceedings of the 18th International Conference on World Wide Web (WWW’09). ACM, New York, NY.Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Jingjing Liu, Yunbo Cao, Chin Y. Lin, Yalou Huang, and Ming Zhou. 2007. Low-quality product review detection in opinion summarization. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL’07).Google ScholarGoogle Scholar
  65. Yandong Liu, Jiang Bian, and Eugene Agichtein. 2008. Predicting information seeker satisfaction in community question answering. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Yue Lu, Panayiotis Tsaparas, Alexandros Ntoulas, and Livia Polanyi. 2010. Exploiting social context for review quality prediction. In Proceedings of the 19th International Conference on World Wide Web (WWW’10). Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Lena Mamykina, Bella Manoim, Manas Mittal, George Hripcsak, and Björn Hartmann. 2011. Design lessons from the fastest Q&A site in the West. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’’11). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Elaheh Momeni. 2014. Adaptive Moderation of User-Generated Content on Web. Ph.D. Dissertation. Retrieved November 16, 2015, from http://eprints.cs.univie.ac.at/4317/.Google ScholarGoogle Scholar
  69. Elaheh Momeni, Simon Braendle, and Eytan Adar. 2015. Adaptive faceted ranking for social media comments. In Advances in Information Retrieval (ECIR’15). Springer.Google ScholarGoogle Scholar
  70. Elaheh Momeni, Claire Cardie, and Myle Ott. 2013a. Properties, prediction, and prevalence of useful user-generated comments for descriptive annotation of social media objects. In Proceedings of the 7th International AAAI Conference on Weblog and Social Media (ICWSM’13).Google ScholarGoogle Scholar
  71. Elaheh Momeni, Ke Tao, Bernhard Haslhofer, and Geert-Jan Houben. 2013b. Identification of useful user comments in social media: A case study on Flickr commons. In Proceedings of the 13th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL’13). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Meredith Ringel Morris, Scott Counts, Asta Roseway, Aaron Hoff, and Julia Schwarz. 2012. Tweeting is believing? Understanding microblog credibility perceptions. In Proceedings of the 2012 ACM Conference on Computer Supported Cooperative Work (CSCW’12). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. Lev Muchnik, Sinan Aral, and Sean J. Taylor. 2013. Social influence bias: A randomized experiment. Science 341, 6146, 647--651.Google ScholarGoogle Scholar
  74. Sean A. Munson, Stephanie Y. Lee, and Paul Resnick. 2013. Encouraging reading of diverse political viewpoints with a browser widget. In Proceedings of the 7th International AAAI Conference on Weblog and Social Media (ICWSM’13).Google ScholarGoogle Scholar
  75. Sean A. Munson and Paul Resnick. 2010. Presenting diverse political opinions: How and how much. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’10). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. Kevin Kyung Nam, Mark S. Ackerman, and Lada A. Adamic. 2009. Questions in, knowledge in? A study of Naver’s question answering community. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’09). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. Syavash Nobarany, Louise Oram, Vasanth Kumar Rajendran, Chi-Hsiang Chen, Joanna McGrenere, and Tamara Munzner. 2012. The design space of opinion measurement interfaces: Exploring recall support for rating and ranking. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  78. Oded Nov, Mor Naaman, and Chen Ye. 2008. What drives content tagging: The case of photos on Flickr. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’08). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  79. Michael P. O’Mahony and Barry Smyth. 2009. Learning to recommend helpful hotel reviews. In Proceedings of the 3rd ACM Conference on Recommender Systems (RecSys’09). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  80. Myle Ott, Claire Cardie, and Jeff Hancock. 2012. Estimating the prevalence of deception in online review communities. In Proceedings of the 21st International Conference on World Wide Web (WWW’12). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  81. Myle Ott, Yejin Choi, Claire Cardie, and Jeff Hancock. 2011. Finding deceptive opinion spam by any stretch of the imagination. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (ACL’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  82. Tim Paek, Michael Gamon, Scott Counts, David Maxwell Chickering, and Aman Dhesi. 2010. Predicting the importance of newsfeed posts and social network friends. In Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI’10).Google ScholarGoogle Scholar
  83. Souneil Park, Seungwoo Kang, Sangyoung Chung, and Junehwa Song. 2009. NewsCube: Delivering multiple aspects of news to mitigate media bias. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’09). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  84. Dragomir R. Radev. 2004. LexRank: Graph-based lexical centrality as salience in text summarization. Journal of Artificial Intelligence Research 22, 1, 457--479. Google ScholarGoogle ScholarDigital LibraryDigital Library
  85. Daniel Ramage, Susan T. Dumais, and Daniel J. Liebling. 2010. Characterizing microblogs with topic models. In Proceedings of the 4th International AAAI Conference on Weblog and Social Media (ICWSM’10).Google ScholarGoogle Scholar
  86. Huzefa Rangwala and Salman Jamali. 2010. Defining a coparticipation network using comments on Digg. IEEE Intelligent Systems 25, 4, 36--45. Google ScholarGoogle ScholarDigital LibraryDigital Library
  87. Stephen Robertson, Milan Vojnovic, and Ingmar Weber. 2009. Rethinking the ESP game. In Extended Abstracts on Human Factors in Computing Systems (CHI’09). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  88. Dana Rotman, Jennifer Golbeck, and Jennifer Preece. 2009. The community is where the rapport is—on sense and structure in the YouTube community. In Proceedings of the 4th International Conference on Communities and Technologies (C&T’’09). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  89. Matthew Rowe, Sofia Angeletou, and Harith Alani. 2011. Predicting discussions on the social Semantic Web. In Proceedings of the 8th Extended Semantic Web Conference on the Semantic Web: Research and Applications, Volume Part II (ESWC’11). 405--420. Google ScholarGoogle ScholarDigital LibraryDigital Library
  90. Shilad Sen, F. Maxwell Harper, Adam LaPitz, and John Riedl. 2007. The quest for quality tags. In Proceedings of the 2007 International ACM Conference on Supporting Group Work (GROUP’07). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  91. Stefan Siersdorfer, Sergiu Chelaru, Wolfgang Nejdl, and Jose San Pedro. 2010. How useful are your comments? Analyzing and predicting YouTube comments and comment ratings. In Proceedings of the 19th International Conference on World Wide Web (WWW’10). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  92. Börkur Sigurbjörnsson and Roelof van Zwol. 2008. Flickr tag recommendation based on collective knowledge. In Proceedings of the 17th International Conference on World Wide Web (WWW’08). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  93. Ruben Sipos, Arpita Ghosh, and Thorsten Joachims. 2014. Was this review helpful to you? It depends! Context and voting patterns in online content. In Proceedings of the 23rd International Conference on World Wide Web (WWW’14). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  94. Bharath Sriram, Dave Fuhry, Engin Demir, Hakan Ferhatosmanoglu, and Murat Demirbas. 2010. Short text classification in Twitter to improve information filtering. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’10). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  95. Gabor Szabo and Bernardo A. Huberman. 2010. Predicting the popularity of online content. Communications of the ACM 53, 8, 80--88. Google ScholarGoogle ScholarDigital LibraryDigital Library
  96. Ke Tao, Fabian Abel, Claudia Hauff, and Geert-Jan Houben. 2012. What makes a tweet relevant for a topic? In Making Sense of Microposts (#MSM2012).Google ScholarGoogle Scholar
  97. Oren Tsur and Ari Rappoport. 2009. RevRank: A fully unsupervised algorithm for selecting the most helpful book reviews. In Proceedings of the International Conference on Weblogs and Social Media (ICWSM’09).Google ScholarGoogle Scholar
  98. Don Turnbull. 2007. Rating, voting & ranking: Designing for collaboration & consensus. In Proceedings of CHI’& consensus. In Proceedings of CHI’’07 Extended Abstracts on Human Factors in Computing Systems (CHI EA’07). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  99. Ibrahim Uysal and W. Bruce Croft. 2011. User oriented tweet ranking: A filtering approach to microblogs.. In Proceedings of the Conference on Information and Knowledge Management (CIKM’11). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  100. Adriano Veloso, Wagner Meira Jr., Tiago Macambira, Dorgival Guedes, and Hélio Almeida. 2007. Automatic moderation of comments in a large on-line journalistic environment. In Proceedings of the International AAAI Conference on Weblog and Social Media (ICWSM’07).Google ScholarGoogle Scholar
  101. Claudia Wagner, Matthew Rowe, Markus Strohmaier, and Harith Alani. 2012a. Ignorance isn’t bliss: An empirical analysis of attention patterns in online communities. In Proceedings of the IEEE International Conference on Social Computing. Google ScholarGoogle ScholarDigital LibraryDigital Library
  102. Claudia Wagner, Matthew Rowe, Markus Strohmaier, and Harith Alani. 2012b. What catches your attention? An empirical study of attention patterns in community forums. In Proceedings of the 6th International AAAI Conference on Weblogs and Social Media (ICWSM’12).Google ScholarGoogle Scholar
  103. Meng Wang, Bingbing Ni, Xian-Sheng Hua, and Tat-Seng Chua. 2012. Assistive tagging: A survey of multimedia tagging with human-computer joint exploration. ACM Computing Surveys 44, 4, Article No. 25. Google ScholarGoogle ScholarDigital LibraryDigital Library
  104. Markus Weimer, Iryna Gurevych, and Max Mühlhäuser. 2007. Automatically assessing the post quality in online discussions on software. In Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions (ACL’07). Google ScholarGoogle ScholarDigital LibraryDigital Library
  105. Kilian Quirin Weinberger, Malcolm Slaney, and Roelof van Zwol. 2008. Resolving tag ambiguity. In Proceedings of the 16th ACM International Conference on Multimedia (MM’08). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  106. Robert West, Ashwin Paranjape, and Jure Leskovec. 2015. Mining missing hyperlinks from human navigation traces: A case study of Wikipedia. In Proceedings of the 24th International Conference on World Wide Web (WWW’15). Google ScholarGoogle ScholarDigital LibraryDigital Library
  107. Jiang Yang, Mark S. Ackerman, and Lada A. Adamic. 2011. Virtual gifts and guanxi: Supporting social exchange in a Chinese online community. In Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work (CSCW’11). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  108. Kyung Hyan Yoo and Ulrike Gretzel. 2009. Comparison of deceptive and truthful travel reviews. In Proceedings of Information and Communication Technologies in Tourism (ENTER’09). 37--47.Google ScholarGoogle ScholarCross RefCross Ref
  109. Hong Yu and Vasileios Hatzivassiloglou. 2003. Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing (EMNLP’03). Google ScholarGoogle ScholarDigital LibraryDigital Library
  110. Zhu Zhang and Balaji Varadarajan. 2006. Utility scoring of product reviews. In Proceedings of the 15th ACM International Conference on Information and Knowledge Management (CIKM’06). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A Survey on Assessment and Ranking Methodologies for User-Generated Content on the Web

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            • Published in

              cover image ACM Computing Surveys
              ACM Computing Surveys  Volume 48, Issue 3
              February 2016
              619 pages
              ISSN:0360-0300
              EISSN:1557-7341
              DOI:10.1145/2856149
              • Editor:
              • Sartaj Sahni
              Issue’s Table of Contents

              Copyright © 2015 ACM

              Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

              Publisher

              Association for Computing Machinery

              New York, NY, United States

              Publication History

              • Published: 22 December 2015
              • Revised: 1 August 2015
              • Accepted: 1 August 2015
              • Received: 1 March 2014
              Published in csur Volume 48, Issue 3

              Permissions

              Request permissions about this article.

              Request Permissions

              Check for updates

              Qualifiers

              • survey
              • Research
              • Refereed

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader