ABSTRACT
The "wisdom of crowds" refers to the phenomenon that aggregated predictions from a large group of people can rival or even beat the accuracy of experts. In domains with substantial stochastic elements, such as stock picking, crowd strategies (e.g. indexing) are difficult to beat. However, in domains in which some crowd members have demonstrably more skill than others, smart sub-crowds could possibly outperform the whole. The central question this work addresses is whether such smart subsets of a crowd can be identified a priori in a large-scale prediction contest that has substantial skill and luck components. We study this question with data obtained from fantasy soccer, a game in which millions of people choose professional players from the English Premier League to be on their fantasy soccer teams. The better the professional players do in real life games, the more points fantasy teams earn. Fantasy soccer is ideally suited to this investigation because it comprises millions of individual-level, within-subject predictions, past performance indicators, and the ability to test the effectiveness of arbitrary player-selection strategies. We find that smaller, smarter crowds can be identified in advance and that they beat the wisdom of the larger crowd. We also show that many players would do better by simply imitating the strategy of a player who has done well in the past. Finally, we provide a theoretical model that explains the results we see from our empirical analyses.
- BRUCE, R. S. 1935-'1936. Group judgements in the fields of lifted weights and visual discrimination. Journal of Psychology 1, 117'-121.Google Scholar
- FAMA, E. F. AND FRENCH, K. R. 2010. Luck versus skill in the cross-section of mutual fund returns. The Journal of Finance LXV, 5, 1915-'1947.Google ScholarCross Ref
- GALTON, F. 1907a. Letters to the editor. Nature 75, 1952.Google Scholar
- GALTON, F. 1907b. Vox populi. Nature 75, 1949, 450--451.Google ScholarCross Ref
- GIGERENZER, G. 2008. Why heuristics work. Perspectives on Psychological Science 3, 1, 20-'29.Google ScholarCross Ref
- GORDON, K. 1924. Group judgements in the field of lifted weights. Journal of Experimental Psychology 7, 389-'400.Google ScholarCross Ref
- HERZOG, S.M. AND HERTWIG, R. 2009. The wisdom of many in one mind: Improving individual judgments with dialectical bootstrapping. Psychological Science 20, 231--237.Google ScholarCross Ref
- KLUGMAN, S. F. 1945. Group judgement for familiar and unfamiliar materials. Journal of Genetic Psychology 32, 103-'110.Google ScholarCross Ref
- KNIGHT, H. 1921. A comparison of the reliability of group and individual judgements. M.S. thesis, Columbia University. unpublished.Google Scholar
- LEVITT, S. D.,MILES, T. J., AND ROSENFIELD, A. M. 2012. Is texas hold'em a game of chance' a legal and economic analysis. The Georgetown Law Journal 101, 581-'636.Google Scholar
- LORENZ, J., RAUHUT, H., SCHWEITZER, F., AND HELBING, D. 2011. How social influence can undermine the wisdom of crowd effect. Proceedings of the National Academy of Sciences 108, 22, 9020'-9025.Google ScholarCross Ref
- LORGE, I., FOX, D., DAVITZ, J., AND BRENNER, M. 1958. A survey of studies contrasting the quality of group performance and individual performance, 1920--1957. Psychological Bulletin 55, 6, 337-'372.Google ScholarCross Ref
- MANNES, A. E., SOLL, J. B., AND LARRICK, R. P. 2013. The wisdom of small crowds. Manuscript in Preparation.Google Scholar
- MCMURRAY, J. C. 2013. Aggregating information by voting: The wisdom of the experts versus the wisdom of the masses. The Review of Economic Studies 80, 1, 277-'312.Google ScholarCross Ref
- MUCHNIK, L., ARAL, S., AND TAYLOR, S. J. 2013. Social influence bias: A randomized experiment. Science 341, 6146, 647'-651.Google Scholar
- SIMMONS, J. P., NELSON, L. D., GALAK, J., AND FREDERICK, S. 2011. Intuitive biases in choice vs. estimation: Implications for the wisdom of crowds. Journal of Consumer Research 38, 1, 1-'15.Google ScholarCross Ref
- SUROWIECKI, J. 2005. The Wisdom of Crowds. Anchor. Google ScholarDigital Library
- TREYNOR, J. L. 1987. Market efficiency and the bean jar experiment. Financial Analysts Journal 43, 3, 50-'53.Google ScholarCross Ref
Index Terms
- The wisdom of smaller, smarter crowds
Recommendations
The wisdom of ad-hoc crowds
HT '14: Proceedings of the 25th ACM conference on Hypertext and social mediaIn this keynote we give an introduction to wisdom of crowds in the Web, the long tail of web content, and the bias involved in the generation of user generated content (UGC). This bias creates the wisdom of ad-hoc crowds or the wisdom of a few. Although ...
The Wisdom of Multiple Guesses
EC '15: Proceedings of the Sixteenth ACM Conference on Economics and ComputationThe "wisdom of crowds" dictates that aggregate predictions from a large crowd can be surprisingly accurate, rivaling predictions by experts. Crowds, meanwhile, are highly heterogeneous in their expertise. In this work, we study how the heterogeneous ...
Analysis of Recycling Capabilities of Individuals and Crowds to Encourage and Educate People to Separate Their Garbage Playfully
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing SystemsSorting garbage is a relevant topic in many countries as it contributes to environmental protection. Empirical evidence suggests that not all people separate waste, potentially because they do not know how to do it correctly or are simply not motivated ...
Comments