Shruti Sannon
Dan Cosley
ABSTRACT
Tasks on crowdsourcing platforms such as Amazon Mechanical Turk often request workers' personal information, raising privacy risks that may be exacerbated by requester-worker power dynamics. We interviewed 14 workers to understand how they navigate these risks. We found that Turkers' decisions to provide personal information during tasks were based on evaluations of the pay rate, the requester, the purpose, and the perceived sensitivity of the request. Participants also engaged in multiple privacy-protective behaviors, such as abandoning tasks or providing inaccurate data, though there were costs associated with these behaviors, such as wasted time and risk of rejection. Finally, their privacy concerns and practices evolved as they learned about both the platform and worker-designed tools and forums. These findings deepen our understanding of both privacy decision-making and invisible labor in paid crowdsourcing, and emphasize a general need to understand how privacy stances change over time.
- Ali Alkhatib, Michael S Bernstein, and Margaret Levi. 2017. Examining Crowd Work and Gig Work through the Historical Lens of Piecework. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 4599--4616. Google Scholar
Digital Library
- Ann Cavoukian. 2011. Privacy by Design in Law, Policy and Practice. A white paper for regulators, decision-makers and policy-makers (2011).Google Scholar
- Kathy Charmaz. 2001. Qualitative Interviewing and Grounded Theory Analysis. 675--694.Google Scholar
- Lydia B Chilton, John J Horton, Robert C Miller, and Shiri Azenkot. 2010. Task Search in a Human Computation Market. In Proceedings of the ACM SIGKDD Workshop on Human Computation. ACM, 1--9. Google ScholarDigital Library
- Mary J Culnan and Pamela K Armstrong. 1999. Information Privacy Concerns, Procedural Fairness, and Impersonal Trust: An Empirical Investigation. Organization Science 10, 1 (1999), 104--115. Google ScholarDigital Library
- Djellel Difallah, Elena Filatova, and Panos Ipeirotis. 2018. Demographics and dynamics of Mechanical Turk Workers. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. ACM, 135--143. Google ScholarDigital Library
- Djellel Eddine Difallah, Michele Catasta, Gianluca Demartini, Panagiotis G Ipeirotis, and Philippe Cudré-Mauroux. 2015. The Dynamics of Micro-task Crowdsourcing: The Case of Amazon MTurk. In Proceedings of the 24th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 238--247. Google ScholarDigital Library
- Alek Felstiner. 2011. Working the Crowd: Employment and Labor Law in the Crowdsourcing Industry. Berkeley J. Emp. & Lab. L. 32 (2011), 143.Google Scholar
- Ujwal Gadiraju, Ricardo Kawase, and Stefan Dietze. 2014. A Taxonomy of Microtasks on the Web. In Proceedings of the 25th ACM Conference on Hypertext and Social Media. ACM, 218--223. Google ScholarDigital Library
- Barney G Glaser and Anselm L Strauss. 1967. The Discovery of Grounded Theory: Strategies for Qualitative Theory. New Brunswick: Aldine Transaction (1967).Google Scholar
- Mary L Gray, Siddharth Suri, Syed Shoaib Ali, and Deepti Kulkarni. 2016. The Crowd is a Collaborative Network. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. ACM, 134--147. Google ScholarDigital Library
- Kotaro Hara, Abigail Adams, Kristy Milland, Saiph Savage, Chris Callison-Burch, and Jeffrey P Bigham. 2018. A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 449. Google ScholarDigital Library
- Lilly C Irani and M Silberman. 2013. Turkopticon: Interrupting Worker Invisibility in Amazon Mechanical Turk. In Proceedings of the 2013 CHI Conference on Human Factors in Computing Systems. ACM, 611--620. Google ScholarDigital Library
- Thivya Kandappu, Vijay Sivaraman, Arik Friedman, and Roksana Boreli. 2013. Exposing and Mitigating Privacy Loss in Crowdsourced Survey Platforms. In Proceedings of the 2013 Workshop on Student Workhop (CoNEXT Student Workhop '13). ACM, New York, NY, USA, 13--16. Google ScholarDigital Library
- Ruogu Kang, Stephanie Brown, Laura Dabbish, and Sara Kiesler. 2014. Privacy Attitudes of Mechanical Turk Workers and the US Public. In Symposium on Usable Privacy and Security (SOUPS), Vol. 4. 1. Google ScholarDigital Library
- Nicolas Kaufmann, Thimo Schulze, and Daniel Veit. 2011. More than Fun and Money: Worker Motivation in Crowdsourcing -- A Study on Mechanical Turk. In AMCIS, Vol. 11. 1--11.Google Scholar
- Aniket Kittur, Jeffrey V Nickerson, Michael Bernstein, Elizabeth Gerber, Aaron Shaw, John Zimmerman, Matt Lease, and John Horton. 2013. The Future of Crowd Work. In Proceedings of the 2013 Conference on Computer-Supported Cooperative Work. ACM, 1301--1318. Google ScholarDigital Library
- Walter S Lasecki, Jeffrey M Rzeszotarski, Adam Marcus, and Jeffrey P Bigham. 2015. The Effects of Sequence and Delay on Crowd Work. In Proceedings of the 2015 ACM Conference on Human Factors in Computing Systems. ACM, 1375--1378. Google ScholarDigital Library
- Walter S Lasecki, Jaime Teevan, and Ece Kamar. 2014. Information extraction and manipulation threats in crowd-powered systems. In Proceedings of the 17th ACM Conference on Computer-Supported Cooperative Work & Social Computing. ACM, 248--256. Google ScholarDigital Library
- Matthew Lease, Jessica Hullman, Jeffrey Bigham, Michael Bernstein, Juho Kim, Walter Lasecki, Saeideh Bakhshi, Tanushree Mitra, and Robert Miller. 2013. Mechanical Turk is not Anonymous. (2013).Google Scholar
- J Lofland, D Snow, L Anderson, and LH Lofland. 2006. Analyzing Social Settings: A Guide to Qualitative Observation and Analysis. Wadsworth/Thomson Learning.Google Scholar
- Jennifer Marlow and Laura A Dabbish. 2014. Who's the Boss?: Requester Transparency and Motivation in a Microtask Marketplace. In CHI'14 Extended Abstracts on Human Factors in Computing Systems. ACM, 2533--2538. Google ScholarDigital Library
- David Martin, Benjamin V Hanrahan, Jacki O'Neill, and Neha Gupta. 2014. Being a Turker. In Proceedings of the 17th ACM Conference on Computer-Supported Cooperative Work & Social Computing. ACM, 224-- 235. Google ScholarDigital Library
- Alice E Marwick and danah boyd. 2018. Understanding Privacy at the Margins -- Introduction. International Journal of Communication 12 (2018), 9.Google Scholar
- Brian McInnis, Dan Cosley, Chaebong Nam, and Gilly Leshed. 2016. Taking a HIT: Designing around Rejection, Mistrust, Risk, and Workers' Experiences in Amazon Mechanical Turk. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 2271-- 2282. Google ScholarDigital Library
- Helen Nissenbaum. 2004. Privacy as Contextual Integrity. Washington Law Review 79 (2004), 119.Google Scholar
- Sandra Petronio. 2012. Boundaries of Privacy: Dialectics of Disclosure. SUNY Press.Google Scholar
- Shruti Sannon, Natalya N Bazarova, and Dan Cosley. 2018. Privacy Lies: Understanding How, When, and Why People Lie to Protect their Privacy in Multiple Online Contexts. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 52. Google ScholarDigital Library
- Shruti Sannon and Dan Cosley. 2018. It was a shady HIT: Navigating Work-Related Privacy Concerns on MTurk. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, LBW507. Google ScholarDigital Library
- Trebor Scholz. 2012. Digital labor: The Internet as Playground and Factory. Routledge. Google ScholarDigital Library
- Aaron Smith. 2016. Gig work, online selling and home sharing. Pew Research Center 17 (2016).Google Scholar
- Jai-Yeol Son and Sung S Kim. 2008. Internet Users' Information Privacyprotective Responses: A Taxonomy and a Nomological Model. MIS Quarterly (2008), 503--529. Google ScholarDigital Library
- Susan Leigh Star and Anselm Strauss. 1999. Layers of Silence, Arenas of Voice: The Ecology of Visible and Invisible Work. Computer-Supported Cooperative Work (CSCW) 8, 1--2 (1999), 9--30. Google ScholarDigital Library
- Latanya Sweeney. 2000. Simple demographics often identify people uniquely. Carnegie Mellon, Data Privacy Working Paper 3. Pittsburgh (2000), 1--34.Google Scholar
- Amazon Mechanical Turk. 2018. Acceptable Use Policy. (2018). https: //www.mturk.com/worker/acceptable-use-policyGoogle Scholar
- Huichuan Xia, Yang Wang, Yun Huang, and Anuj Shah. 2017. "Our privacy needs to be protected at all costs": Crowd workers' privacy experiences on Amazon Mechanical Turk. Proc. ACM Hum.-Comput. Interact 1 (2017). Google ScholarDigital Library
- Jie Yang, Carlo van der Valk, Tobias Hossfeld, Judith Redi, and Alessandro Bozzon. 2018. How do Crowdworker Communities and Microtask Markets Influence Each Other? A Data-driven Study on Amazon Mechanical Turk. Technical Report. Université de Fribourg.Google Scholar
- Ming Yin, Mary L Gray, Siddharth Suri, and Jennifer Wortman Vaughan. 2016. The Communication Network Within the Crowd. In Proceedings of the 25th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 1293--1303. Google ScholarDigital Library
Index Terms
- Privacy, Power, and Invisible Labor on Amazon Mechanical Turk
Recommendations
Quantifying the Invisible Labor in Crowd Work
CSCW2Crowdsourcing markets provide workers with a centralized place to find paid work. What may not be obvious at first glance is that, in addition to the work they do for pay, crowd workers also have to shoulder a variety of unpaid invisible labor in these ...
A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsA growing number of people are working as part of on-line crowd work. Crowd work is often thought to be low wage work. However, we know little about the wage distribution in practice and what causes low/high earnings in this setting. We recorded 2,676 ...
"Our Privacy Needs to be Protected at All Costs": Crowd Workers' Privacy Experiences on Amazon Mechanical Turk
Crowdsourcing platforms such as Amazon Mechanical Turk (MTurk) are widely used by organizations, researchers, and individuals to outsource a broad range of tasks to crowd workers. Prior research has shown that crowdsourcing can pose privacy risks (e.g., ...
Comments