Lilly C. Irani
ABSTRACT
As HCI researchers have explored the possibilities of human computation, they have paid less attention to ethics and values of crowdwork. This paper offers an analysis of Amazon Mechanical Turk, a popular human computation system, as a site of technically mediated worker-employer relations. We argue that human computation currently relies on worker invisibility. We then present Turkopticon, an activist system that allows workers to publicize and evaluate their relationships with employers. As a common infrastructure, Turkopticon also enables workers to engage one another in mutual aid. We conclude by discussing the potentials and challenges of sustaining activist technologies that intervene in large, existing socio-technical systems.
- Ackerman, M. 2000. The intellectual challenge of CSCW: The gap between social requirements and technical feasibility. Human-computer interaction, 15(2), 179--203. Google Scholar
Digital Library
- Bardzell, S. 2010. Feminist HCI: Taking Stock and Outlining an Agenda for Design, Proc. CHI, 1301--1310. Google ScholarDigital Library
- Barley, S. R. and Kunda, G. 2004. Gurus, hired guns, and warm bodies: itinerant experts in a knowledge economy. Princeton University Press. Google ScholarDigital Library
- Berg, M. 1998. The politics of technology: On bringing social theory into technological design. ST&HV, 23(4), 456--490.Google ScholarCross Ref
- Bezos, J. 2006. Opening Keynote. MIT Emerging Technologies Conference. Accessed: http://video.mit.edu/watch/opening-keynote-and-keynote-interview-with-jeff-bezos-9197/Google Scholar
- Biewald, L. 2009. "Turkopticon." Crowdflower Blog. Accessed: http://blog.crowdflower.com/2009/02/turkopticon/Google Scholar
- Borning, A. and Muller, M. 2012. Next steps for value sensitive design. Proc. CHI, 1125--1134. Google ScholarDigital Library
- Bowker, G. & Star, S. L. 2000. Sorting Things Out: Classification and Its Consequences. MIT Press. Google ScholarDigital Library
- Bruckman, A. and Hudson, J. M. 2005. Using Empirical Data to Reason about Internet Research Ethics. Proc. ECSCW, 287--306. Google ScholarDigital Library
- Brynjarsdottir, H., Håkansson, M., Pierce, J., Baumer, E., DiSalvo, C., and Sengers, P. 2012. Sustainably unpersuaded. Proc. CHI, 947--956. Google ScholarDigital Library
- Crenshaw, K. 1991. Mapping the Margins: Intersectionality, Identity Politics, and Violence Against Women of Color. Stanford Law Review, 43(6), 1241--1299.Google ScholarCross Ref
- Crowdsourced data analysis with Clockwork Raven. Accessed at http://engineering.twitter.com/2012/08/crowdsourceddata-analysis-with.htmlGoogle Scholar
- da Costa, B. and Philip, K., eds. 2008. Tactical Biopolitics: Art, Activism, Technoscience. MIT Press.Google Scholar
- DiSalvo, C. 2009. Design and the Construction of Publics. Design Issues, 25(1), 48--63.Google ScholarCross Ref
- DiSalvo, C. 2012. Adversarial Design. MIT Press. Google ScholarDigital Library
- Dourish, P. 2010. HCI and environmental sustainability: the Design of Politics and Politics of Design. Proc. DIS 2010, 1--10. Google ScholarDigital Library
- Dourish, P. and Mainwaring, S. 2012. Ubicomp's Colonial Impulse. Proc. Ubicomp. Google ScholarDigital Library
- Felstiner, A. 2010. Working the Crowd: Employment and Labor Law in the Crowdsourcing Industry. Berkeley Journal of Employment & Labor Law, 32 (1), 143--204.Google Scholar
- Garcia, D. and Lovink, G. 1997. The ABC of tactical media. nettime listserv. Accessed: http://www.nettime.org/Lists-Archives/nettime-l-9705/msg00096.htmlGoogle Scholar
- Haraway, D. J. 1990. Simians, Cyborgs, and Women: The Reinvention of Nature. Routledge.Google Scholar
- Hirsch, T. 2009. Learning from activists. Interactions, 16(3), 31--33. Google ScholarDigital Library
- Ipeirotis, P. 2010. Demographics of Mechanical Turk.. NYU Working Paper No. CEDER-10-01Google Scholar
- Ipeirotis, P. 2010. Analyzing the Mechanical Turk Marketplace. XRDS, 17(2), 16--21. Google ScholarDigital Library
- Ipeirotis, P. 2008. Why People Participate in Mechanical Turk? Accessed: http://www.behind-the-enemylines.com/2008/03/why-people-participate-on-mechanical.htmlGoogle Scholar
- Jackson, S., Pompe, A., and Krieshok, G. 2011. Things fall apart: maintenance, repair, and technology for education initiatives in rural Namibia. Proc iConference, 83--90. Google ScholarDigital Library
- Kelty, C. 2012. Two Bits: The Cultural Significance of Free Software. Duke University Press. Google ScholarDigital Library
- Lampe, C. and Resnick, P. 2004. Slash(dot) and burn. Proc SIGCHI 2004, 543--550. Google ScholarDigital Library
- Law, E. and Von Ahn, L. 2011. Human Computation. Morgan Claypool Publishers. Google ScholarDigital Library
- Law, J. 2004. After method: mess in social science research. Routledge.Google Scholar
- LeDantec, C. 2012. Participation and Publics: Supporting Community Engagement, Proc. CHI 2012, 1351--1360. Google ScholarDigital Library
- Light, A. 2011. Digital interdependence and how to design for it. Interactions, 18 (2), 34. Google ScholarDigital Library
- Orlikowski, W. J. 1992. The Duality of Technology: Rethinking the Concept of Technology in Organizations. Organization Science, 3(3), 398--427.Google ScholarDigital Library
- Ross, J., Irani, L., Silberman, M. S., et al. 2010. Who are the crowdworkers?, EA CHI 2010 (alt.chi), 2863--2872. Google ScholarDigital Library
- Ruhleder, K. and Star, S. L. 2001. Steps Toward an Ecology of Infrastructure: Design and Access for Large Information Spaces. In J. Yates and J. Van Maanen, eds., Information Technology and Organizational Transformation: History, Rhetoric, and Practice. Sage, 305--343.Google Scholar
- Smith, V. 1997. New Forms of Work Organization. Annual Review of Sociology, 23, 315--339.Google ScholarCross Ref
- Snow, R., O'Connor, B., Jurafsky, D., & Ng, A. Y. 2008. Cheap and fast - but is it good?: Evaluating non-expert annotations for natural language tasks. Proc. EMNLP, 254--263. Google ScholarDigital Library
- Star, S. L. and Strauss, A. 1999. Layers of Silence, Arenas of Voice: The Ecology of Visible and Invisible Work. Proc. CSCW, 8, 9--30. Google ScholarDigital Library
- Sood, S., Antin, J., and Churchill, E. 2012. Profanity use in online communities. Proc. CHI 2012, 1481--1490. Google ScholarDigital Library
- Suchman, L. 1995. Making Work Visible. CACM, 38(9). Google ScholarDigital Library
- Suchman, L. 2006. Human Machine Reconfigurations, Cambridge Univ. Press. Google ScholarDigital Library
- Taylor, A. 2011. Out There. Proc CHI, 685--694. Google ScholarDigital Library
- TurkWork. http://turkwork.differenceengines.comGoogle Scholar
Index Terms
- Turkopticon: interrupting worker invisibility in amazon mechanical turk
Recommendations
We Are Dynamo: Overcoming Stalling and Friction in Collective Action for Crowd Workers
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing SystemsBy lowering the costs of communication, the web promises to enable distributed collectives to act around shared issues. However, many collective action efforts never succeed: while the web's affordances make it easy to gather, these same decentralizing ...
How many crowdsourced workers should a requester hire?
Recent years have seen an increased interest in crowdsourcing as a way of obtaining information from a potentially large group of workers at a reduced cost. The crowdsourcing process, as we consider in this paper, is as follows: a requester hires a ...
Mechanical turk as an ontology engineer?: using microtasks as a component of an ontology-engineering workflow
WebSci '13: Proceedings of the 5th Annual ACM Web Science ConferenceOntology evaluation has proven to be one of the more difficult problems in ontology engineering. Researchers proposed numerous methods to evaluate logical correctness of an ontology, its structure, or coverage of a domain represented by a corpus. ...
Comments