Top

Published in:

2016 | OriginalPaper | Chapter

Crowdsourcing User and Design Research

Authors : Vassilis Javed Khan, Gurjot Dhillon, Maarten Piso, Kimberly Schelle

Published in: Collaboration in Creative Design

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Crowdsourcing can be defined as a task, which is usually performed by an employee, that is given out as an open call to a crowd of users to be completed. Although crowdsourcing has been growing in recent years, its application to design research and education has only scratched the surface of its potential. In this chapter we first introduce the different types of crowdsourcing. Then, following the typical design cycle we present examples from literature and cases from an educational setting of how crowdsourcing can support designers. Based on these examples we provide a list of tips for utilizing crowdsourcing for design and user research activities.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Combining User Needs and Stakeholder Requirements: The Value Design Method

next chapter Cardboard Modeling: Exploring, Experiencing and Communicating

Kittur A, Chi EH, Suh B (2008) Crowdsourcing user studies with Mechanical Turk. In: Proceedings of the SIGCHI conference on human factors in computing systems, April. ACM, New York, USA, pp 453–456

The authors describe an experiment in which they investigate Amazon’s Mechanical Turk and how they can get quality responses from workers

Mason W, Suri S (2012) Conducting behavioral research on Amazon’s Mechanical Turk. Behav Res Methods 44(1):1–23CrossRef

The paper demonstrates how to use Amazon’s Mechanical Turk website for conducting behavioral research and tries to lower the entry barrier for researchers who could benefit from this platform

McDonald P, Mohebbi M, Slatkin B (2012) Comparing Google consumer surveys to existing probability and non-probability based internet surveys. Google Whitepaper. Retrieved from http://www.google.com/insights/consumersurveys/static/consumer_surveys_whitepaper.pdf

Description of how Google Consumer Surveys is superior to current probability based Internet panels by using what is known as a “surveywall” to attract respondents

Aitamurto T, Leiponen A, Tee R (2013). The promise of idea crowdsourcing: benefits, contexts, limitations, September 25. Retrieved from http://www.academia.edu/963662/The_Promise_of_Idea_Crowdsourcing_Benefits_Contexts_Limitations

The authors review crowdsourcing for idea generation (‘idea crowdsourcing’) both from the perspective of academic literature and actual cases from businesses to understand how and when to use crowdsourcing and with which benefits and costs

4 Reasons you should consider crowdsourced design for your next big project (n.d.) Retrieved September 30, 2014, from http://michaelhyatt.com/crowd-sourced-design.html

Adepetu A, Ahmed KA, Al Abd, Y, Al Zaabi A, Svetinovic D (2012) CrowdREquire: a requirements engineering crowdsourcing platform. In: AAAI spring symposium: wisdom of the crowd, March

Albert W, Tullis T, Tedesco D (2009) Beyond the usability lab: conducting large-scale online user experience studies. Morgan Kaufmann

Bayus BL (2013) Crowdsourcing new product ideas over time: an analysis of the dell IdeaStorm community. Manag Sci 59(1):226–244, http://doi.org/10.1287/mnsc.1120.1599 CrossRef

Bjelland O, Wood R (2008) An inside view of IBM’s ‘innovation jam. MIT Sloan Manag Rev 50(1):32–40

Blattberg E (2011) Crowdsourcing industry landscape. Retrieved from http://www.crowdsourcing.org/editorial/november-2011-crowdsourcing-industry-landscape-infographic/7680

Brabham DC (2008) Crowdsourcing as a model for problem solving: an introduction and cases. Converg Int J Res New Media Technol 14(1):75–90, http://doi.org/10.1177/1354856507084420 CrossRef

Corney JR, Sanchez CT, Jagadeesan AP, Regli WC (2009) Outsourcing labour to the cloud. Int J Innov Sustain Dev 4(4):294, http://doi.org/10.1504/IJISD.2009.033083 CrossRef

Crowdsourcing your brand design: the math just doesn’t work out (n.d.) Retrieved from http://forty.co/crowdsourcing-your-brand-design-the-math-just-doesnt-work-out

Dow S, Gerber E, Wong A (2013) A pilot study of using crowds in the classroom. ACM Press, New York, USA, p227, http://doi.org/10.1145/2470654.2470686

Felstiner A (2011) Working the crowd: employment and labor law in the crowdsourcing industry (SSRN scholarly paper No. ID 1593853). Social Science Research Network, Rochester. Retrieved from http://papers.ssrn.com/abstract=1593853

Finke RA (1990) Creative imagery: discoveries and inventions in visualization. L. Erlbaum Associates, Hillsdale

Geiger D, Rosemann M, Fielt E, Schader M (2012) Crowdsourcing information systems – definition, typology, and design. In: ICIS 2012 proceedings. Retrieved from http://aisel.aisnet.org/icis2012/proceedings/ResearchInProgress/53

Glinz M, Wieringa RJ (2007) Guest editors’ introduction: stakeholders in requirements engineering. Software IEEE 24(2):18–20CrossRef

Grier DA (2011) Foundational issues in human computing and crowdsourcing. In: CHI 2011

Heer J, Bostock M (2010). Crowdsourcing graphical perception: using mechanical turk to assess visualization design. In: Proceedings of the SIGCHI conference on human factors in computing systems, March. ACM, pp 203–212

Hosseini M, Phalp K, Taylor J, Ali R (2013) Towards crowdsourcing for requirements engineering. In: Joint proceedings of REFSQ-2014 workshops, doctoral symposium, empirical track, and posters, co-located with the 20th international conference on requirements engineering: foundation for software quality (REFSQ 2014). Retrieved from http://ceur-ws.org/Vol-1138/et2.pdf

Howe J (2006) The rise of crowdsourcing. North 14:1–5, http://doi.org/10.1086/599595

Ipeirotis PG (2010) Analyzing the Amazon Mechanical Turk marketplace. XRDS Crossroads ACM Magaz Stud 17(2):16. http://doi.org/10.1145/1869086.1869094

Ko AJ, Chilana PK (2010) How power users help and hinder open bug reporting. ACM Press, New York, USA, p1665. http://doi.org/10.1145/1753326.1753576

Lewis JR (2014) Usability: lessons learned … yet to be learned. Int J Hum Comput Interact 30(9):663–684, http://doi.org/10.1080/10447318.2014.930311 CrossRef

Lim SL, Quercia D, Finkelstein A (2010a) StakeNet: using social networks to analyse the stakeholders of large-scale software projects. In: Proceedings of the 32Nd ACM/IEEE international conference on software engineering – Volume 1. ACM, New York, pp 295–304. http://doi.org/10.1145/1806799.1806844

Lim SL, Quercia D, Finkelstein A (2010b) StakeSource: harnessing the power of crowdsourcing and social networks in stakeholder analysis, vol 2. ACM Press, p 239. http://doi.org/10.1145/1810295.1810340

Liu D, Bias RG, Lease M, Kuipers R (2012) Crowdsourcing for usability testing. Proc Am Soc Inf Sci Technol 49(1):1–10, http://doi.org/10.1002/meet.14504901100

Li Z (2011) Research of crowdsourcing model based on case study. Manag Sci 1–5. http://doi.org/10.1109/ICSSSM.2011.5959456

Malone TW, Laubacher RJ (1999) The dawn of the E-Lance economy. In: Nüttgens M, Scheer A-W (eds) Electronic business engineering. Physica-Verlag HD, Heidelberg, pp 13–24. Retrieved from http://link.springer.com/10.1007/978-3-642-58663-7_2

Mason W, Watts DJ (2009) Financial incentives and the "performance of crowds". In: Bennett P, Chandrasekar R, Chickering M, Ipeirotis P, Law E, Mityagin A, Provost F, von Ahn L (eds) Proceedings of the ACM SIGKDD workshop on human computation (HCOMP '09). ACM, New York, pp 77–85, http://dx.doi.org/10.1145/1600150.1600175

Nelson ET, Stavrou A (2011) Advantages and disadvantages of remote asynchronous usability testing using Amazon Mechanical Turk. Proc Hum Fact Ergon Soc Annu Meet 55(1):1080–1084, http://doi.org/10.1177/1071181311551226 CrossRef

Poetz MK, Schreier M (2012) The value of crowdsourcing: can users really compete with professionals in generating new product ideas?: the value of crowdsourcing. J Prod Innov Manag 29(2):245–256, http://doi.org/10.1111/j.1540-5885.2011.00893.x CrossRef

Quinn AJ, Bederson BB (2011) Human computation: a survey and taxonomy of a growing field. ACM Press, New York, USA, p 1403. http://doi.org/10.1145/1978942.1979148

Rogers Y, Sharp H, Preece J (2011) Interaction design: beyond human computer interaction, 3rd edn. Wiley

Ross J, Irani L, Silberman MS, Zaldivar A, Tomlinson B (2010) Who are the crowdworkers?: shifting demographics in mechanical turk. ACM Press, p 2863. http://doi.org/10.1145/1753846.1753873

Schön DA (1983) The reflective practitioner: how professionals think in action. Basic Books, New York

Simula H, Ahola T (2014) A network perspective on idea and innovation crowdsourcing in industrial firms. Ind Mark Manag 43(3):400–408, http://doi.org/10.1016/j.indmarman.2013.12.008 CrossRef

Standish Group (1999) Chaos: a recipe for success. Standish Group International

Surowiecki J (2005) The wisdom of crowds. Anchor Books, New York

Tates K, Zwaanswijk M, Otten R, van Dulmen S, Hoogerbrugge PM, Kamps WA, Bensing JM (2009) Online focus groups as a tool to collect data in hard-to-include populations: examples from paediatric oncology. BMC Med Res Methodol 9(1):15, http://doi.org/10.1186/1471-2288-9-15 CrossRef

Taylor A (2000) IT projects: sink or swim. Comput Bull January, 24–26

Viskovic D, Varga M, Curko K (2008) Bad practices in complex IT projects. In: ITI 2008 – 30th international conference on information technology interfaces, p. 301

von Ahn L, Maurer B, McMillen C, Abraham D, Blum M (2008) reCAPTCHA: human-based character recognition via web security measures. Science 321(5895):1465–1468, http://doi.org/10.1126/science.1160379 MathSciNetCrossRefMATH

Why Designers Hate Crowdsourcing (n.d.) Retrieved September 30, 2014, from http://www.forbes.com/2010/07/09/99designs-spec-graphic-technology-future-design-crowdsourcing.html

Yu L, Nickerson JV (2013) An internet-scale idea generation system. ACM Trans Interact Intell Syst 3(1):1–24, http://doi.org/10.1145/2448116.2448118 CrossRef

Title: Crowdsourcing User and Design Research
Authors: Vassilis Javed Khan
Gurjot Dhillon
Maarten Piso
Kimberly Schelle
Publisher: Springer International Publishing
Book: Collaboration in Creative Design
Print ISBN: 978-3-319-29153-6

Electronic ISBN: 978-3-319-29155-0

Copyright Year: 2016
DOI: https://doi.org/10.1007/978-3-319-29155-0_7

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"