B. Thomas Adler
ABSTRACT
We consider the problem of measuring user contributions to versioned, collaborative bodies of information, such as wikis. Measuring the contributions of individual authors can be used to divide revenue, to recognize merit, to award status promotions, and to choose the order of authors when citing the content. In the context of the Wikipedia, previous works on author contribution estimation have focused on two criteria: the total text created, and the total number of edits performed. We show that neither of these criteria work well: both techniques are vulnerable to manipulation, and the total-text criterion fails to reward people who polish or re-arrange the content.
We consider and compare various alternative criteria that take into account the quality of a contribution, in addition to the quantity, and we analyze how the criteria differ in the way they rank authors according to their contributions. As an outcome of this study, we propose to adopt total edit longevity as a measure of author contribution. Edit longevity is resistant to simple attacks, since edits are counted towards an author's contribution only if other authors accept the contribution. Edit longevity equally rewards people who create content, and people who rearrange or polish the content. Finally, edit longevity distinguishes the people who contribute little (who have contribution close to zero) from spammers or vandals, whose contribution quickly grows negative.
- B. T. Adler, J. Benterou, K. Chatterjee, L. de Alfaro, I. Pye, and V. Raman. Assigning trust to wikipedia content. Technical Report UCSC-CRL-07-09, School of Engineering, University of California, Santa Cruz, CA, USA, 2007.Google Scholar
- B. T. Adler and L. de Alfaro. A content-driven reputation system for the Wikipedia. In Proc. of the 16th Intl. World Wide Web Conf. (WWW 2007). ACM Press, 2007. Google ScholarDigital Library
- M. Burke and R. Kraut. Taking up the mop: identifying future Wikipedia administrators. In CHI '08: CHI '08 extended abstracts on Human factors in computing systems, pages 3441--3446, New York, NY, USA, 2008. ACM. Google ScholarDigital Library
- G. Cormode and S. Muthukrishnan. The string edit distance matching problem with moves. ACM Trans. Algorithms, 3(1):2, 2007. Google ScholarDigital Library
- W. Cunningham and B. Leuf. The Wiki Way. Quick Collaboration on the Web. Addison-Wesley, 2001. Google ScholarDigital Library
- R. E. Park et al. Software size measurement: A framework for counting source statements. Technical Report CMU/SEI-92-TR-020, Carnegie Mellon University, September 1992.Google ScholarCross Ref
- C. L. Giles and I. G. Councill. Who gets acknowledged: Measuring scientific contributions through automatic acknowledgement indexing. Proc. of the National Academy of Sciences, 101(51):17599--17604, 2004.Google ScholarCross Ref
- A. Kittur, E. Chi, B. A. Pendleton, B. Suh, and T. Mytkowicz. Power of the Few vs. Wisdom of the Crowd: Wikipedia and the rise of the Bourgeoisie. Alt. CHI, 2007.Google Scholar
- N. Korfiatis, M. Poulos, and G. Bokos. Evaluating authoritative source using social networks: an insight from Wikipedia. Online Information Review, 30(3):252--262, 2006.Google ScholarCross Ref
- V. I. Levenshtein. Binary codes capable of correcting insertions and reversals. Sov. Phys. Dokl., 10:707--710, 1966.Google Scholar
- D. L. McGuinness, H. Zeng, P. P. da Silva, L. Ding, D. Narayanan, and M. Bhaowal. Investigation into trust for collaborative information repositories: A Wikipedia case study. In Proceedings of the Workshop on Models of Trust for the Web, 2006.Google Scholar
- F. Ortega and J. M. G. Barahona. Quantitative analysis of the Wikipedia community of users. In WikiSym '07: Proceedings of the 2007 international symposium on Wikis, pages 75--86, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- L. Page, S. Brin, R. Motwani, and T. Winograd. The PageRank citation ranking: Bringing order to the web. Technical report, Stanford Digital Library Technologies Project, 1998.Google Scholar
- R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2007. ISBN 3-900051-07-0.Google Scholar
- H. P. Schultz. Software management metrics. Technical Report AD-A196 916, MITRE, May 1988.Google ScholarCross Ref
- C. Shirky. Gin, television, and social surplus. http://www.herecomeseverybody.org/2008/04/looking-for-the-mouse.html, April 2008. (Retrieved on 9-May-2008.).Google Scholar
- K. Stein and C. Hess. Does it matter who contributes: a study on featured articles in the german wikipedia. In HT '07: Proceedings of the 18th conference on Hypertext and hypermedia, pages 171--174, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- B. Suh, E. H. Chi, A. Kittur, and B. A. Pendleton. Lifting the veil: improving accountability and social transparency in Wikipedia with wikidashboard. In CHI '08: Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, pages 1037--1040, New York, NY, USA, 2008. ACM. Google ScholarDigital Library
- A. Swartz. Who writes Wikipedia? http://www.aaronsw.com/weblog/whowriteswikipedia, September 2006. (Retrieved on 9-May-2008.).Google Scholar
- W. F. Tichy. The string-to-string correction problem with block move. ACM Trans. on Computer Systems, 2(4), 1984. Google ScholarDigital Library
- J. Voß. Measuring wikipedia. In Proc. of the 10th Intl. Conf. of the ISSI, 2005.Google Scholar
- Robert A. Wagner and Michael J. Fischer. The string-to-string correction problem. J. ACM, 21(1):168--173, 1974. Google ScholarDigital Library
- J. Wales. Wikipedia, emergence, and the wisdom of crowds. http://lists.wikimedia.org/pipermail/wikipedia-l/2005-May/021764.html, May 2005. (Retrieved 9-May-2008.).Google Scholar
- D. M. Wilkinson and B. A. Huberman. Cooperation and quality in wikipedia. In WikiSym '07: Proceedings of the 2007 international symposium on Wikis, pages 157--164, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
Index Terms
- Measuring author contributions to the Wikipedia
Recommendations
The Effect of Emotional Cues from the NFL on Wikipedia Contributions
Exploiting evidence that sporting results affect fans' mood, we analyze whether National Football League game outcomes can affect the contributions of Wikipedia editors who identify as fans of a specific team. We find that the day after a team loses, ...
Estimating similarity among collaboration contributions
K-CAP '05: Proceedings of the 3rd international conference on Knowledge captureThe need for collaboration arises in many activities required for effective problem solving and decision making. We are developing Angler, a web-services tool that supports collaboration among participants on some focus topic. Angler overcomes some ...
Management of community contributions: A case study on the Android and Linux software ecosystems
AbstractIn recent years, many companies have realized that collaboration with a thriving user or developer community is a major factor in creating innovative technology driven by market demand. As a result, businesses have sought ways to stimulate ...
Comments