Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Cover of the book

2020 | OriginalPaper | Chapter

Software Crowdsourcing Design: An Experiment on the Relationship Between Task Design and Crowdsourcing Performance

Authors : Turki Alelyani, Paul T. Grogan, Yla Tausczik, Ye Yang

Published in: HCI International 2020 – Late Breaking Papers: Interaction, Knowledge and Social Media

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Software crowdsourcing platforms allow for wide task accessibility and self-selection, which require participants to interact with a wide range of task options. This increased accessibility enables crowd workers to freely choose tasks based on their skills, experience and interests. However, the inconsistencies in crowd workers’ comprehension and interpretation of the designs of different tasks and the platform interaction strategies may introduce task uncertainty into the selection process. This may result in low performance, which can diminish the quality of the crowdsourcing platform. Previous studies show that the quality of the outcome of these systems depends on different design parameters in both social and technical contexts. Therefore, through the lens of Socio-Technical Systems, our experimental study quantifies and compares 30 participants’ behavior and performance throughout their interaction and selection of different software tasks via an online-based tool named “SoftCrowd,” using different design strategies and task features. Results showed a significant relationship between task submission rate and platform design, including interaction strategies, task ambiguity and task instruction design. These findings suggest that our proposed design factors may have unanticipated effects on the quality of crowdsourcing, calling for more studies of crowdsourcing design to inform better strategies for various aspects of platform design.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
next chapter Implementation of Descriptive Similarity for Decision Making in Smart Cities
Footnotes
Literature
1.
Alelyani, T., Mao, K., Yang, Y.: Context-centric pricing: early pricing models for software crowdsourcing tasks. In: Proceedings of the 13th International Conference on Predictive Models and Data Analytics in Software Engineering (2017)
2.
Alelyani, T., Yang, Y.: Software crowdsourcing reliability: an empirical study on developers behavior. In: Proceedings of the 2nd International Workshop on Software Analytics, pp. 36–42 (2016)
3.
Archak, N.: Money, glory and cheap talk: analyzing strategic behavior of contestants in simultaneous crowdsourcing contests on TopCoder.com. In: Proceedings of the 19th International Conference on World Wide Web, pp. 21–30 (2010)
4.
Archak, N., Sundararajan, A.: Optimal design of crowdsourcing contests. In: ICIS 2009 Proceedings, p. 200 (2009)
5.
Begel, A., Herbsleb, J.D., Storey, M.A.: The future of collaborative software development. In: Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work Companion, pp. 17–18 (2012)
6.
7.
Bostrom, R.P., Heinen, J.S.: MIS problems and failures: a socio-technical perspective, Part II: the application of socio-technical theory. MIS Q. 1, 11–28 (1977)CrossRef
8.
9.
Dow, S., Kulkarni, A., Klemmer, S., Hartmann, B.: Shepherding the crowd yields better work. In: Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, pp. 1013–1022 (2012)
10.
Drommi, A., Ulferts, G.W., Shoemaker, D.: Interface design: a focus on cognitive science. In: The Proceedings of ISECON 2001, vol. 18 (2001)
11.
Finnerty, A., Kucherbaev, P., Tranquillini, S., Convertino, G.: Keep it simple: reward and task design in crowdsourcing (2013)
12.
Gefen, D., Rigdon, E.E., Straub, D.: Editor’s comments: an update and extension to SEM guidelines for administrative and social science research. MIS Q. 35, iii–xiv (2011)
13.
Haas, D., Greenstein, M., Kamalov, K., Marcus, A., Olszewski, M., Piette, M.: Reducing error in context-sensitive crowdsourced tasks. In: First AAAI Conference on Human Computation and Crowdsourcing (2013)
14.
Hackman, J.R., Lorsch, J.: Handbook of Organizational Behavior. Prentice-Hall, Englewood Cliffs (1987)
15.
Kittur, A., et al.: The future of crowd work. In: Proceedings of the 2013 Conference on Computer Supported Cooperative Work, pp. 1301–1318 (2013)
16.
Liu, S., Xia, F., Zhang, J., Wang, L.: How crowdsourcing risks affect performance: an exploratory model. Manage. Decis. 54, 2235–2255 (2016)CrossRef
17.
Malone, T.W., Crowston, K.: The interdisciplinary study of coordination. ACM Comput. Surv. (CSUR) 26(1), 87–119 (1994)CrossRef
18.
Mao, K., Yang, Y., Wang, Q., Jia, Y., Harman, M.: Developer recommendation for crowdsourced software development tasks. In: 2015 IEEE Symposium on Service-Oriented System Engineering, pp. 347–356. IEEE (2015)
19.
Mason, W., Watts, D.J.: Financial incentives and the “performance of crowds”. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, pp. 77–85 (2009)
20.
Moussawi, S., Koufaris, M.: Working on low-paid micro-task crowdsourcing platforms: an existence, relatedness and growth view (2015)
21.
Retelny, D., Bernstein, M.S., Valentine, M.A.: No workflow can ever be enough: how crowdsourcing workflows constrain complex work. Proc. ACM Hum.-Comput. Interact. CSCCSCW(1), 1–23 (2017)
22.
Rogstadius, J., Kostakos, V., Kittur, A., Smus, B., Laredo, J., Vukovic, M.: An assessment of intrinsic and extrinsic motivation on task performance in crowdsourcing markets. In: Fifth International AAAI Conference on Weblogs and Social Media (2011)
23.
Rymill, S.J., Dodgson, N.A.: A psychologically-based simulation of human behaviour. In: TPCG, pp. 35–42. Citeseer (2005)
24.
Scacchi, W.: Understanding the requirements for developing open source software systems. IEE Proc. Softw. 149(1), 24–39 (2002)
25.
Schulze, T., Krug, S., Schader, M.: Workers’ task choice in crowdsourcing and human computation markets (2012)
26.
Schulze, T., Seedorf, S., Geiger, D., Kaufmann, N., Schader, M.: Exploring task properties in crowdsourcing-an empirical study on mechanical turk (2011)
27.
Shah, U.S., Jinwala, D.C.: Resolving ambiguities in natural language software requirements: a comprehensive survey. ACM SIGSOFT Softw. Eng. Notes 40(5), 1–7 (2015)CrossRef
28.
Slivkins, A., Vaughan, J.W.: Online decision making in crowdsourcing markets: theoretical challenges. ACM SIGecom Exchanges 12(2), 4–23 (2014)CrossRef
29.
Stewart, O., Lubensky, D., Huerta, J.M.: Crowdsourcing participation inequality: a scout model for the enterprise domain. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, pp. 30–33 (2010)
30.
Stol, K.J., Fitzgerald, B.: Two’s company, three’s a crowd: a case study of crowdsourcing software development. In: Proceedings of the 36th International Conference on Software Engineering, pp. 187–198 (2014)
31.
32.
Trist, E.: The evolution of socio-technical systems: a conceptual framework and an action research program. Ontario Ministry of Labour (1981)
33.
Vaish, R., Organisciak, P., Hara, K., Bigham, J.P., Zhang, H.: Low effort crowdsourcing: Leveraging peripheral attention for crowd work. In: Second AAAI Conference on Human Computation and Crowdsourcing (2014)
34.
Wu, H., Corney, J., Grant, M.: Relationship between quality and payment in crowdsourced design. In: Proceedings of the 2014 IEEE 18th International Conference on Computer Supported Cooperative Work in Design (CSCWD), pp. 499–504. IEEE (2014)
35.
Xia, F., Liu, S., Zhang, J.: How social subsystem and technical subsystem risks influence crowdsourcing performance. In: PACIS, p. 222 (2015)
36.
37.
Zhao, Y.C., Zhu, Q.: Effects of extrinsic and intrinsic motivation on participation in crowdsourcing contest. Online Inf. Rev. 38, 896–917 (2014)
38.
Zheng, H., Li, D., Hou, W.: Task design, motivation, and participation in crowdsourcing contests. Int. J. Electron. Commer. 15(4), 57–88 (2011)CrossRef
39.
Zhu, H., Djurjagina, K., Leker, J.: Innovative behaviour types and their influence on individual crowdsourcing performances. Int. J. Innov. Manage. 18(06), 1440015 (2014)CrossRef
Metadata
Title
Software Crowdsourcing Design: An Experiment on the Relationship Between Task Design and Crowdsourcing Performance
Authors
Turki Alelyani
Paul T. Grogan
Yla Tausczik
Ye Yang
Copyright Year
2020
Book
HCI International 2020 – Late Breaking Papers: Interaction, Knowledge and Social Media

Print ISBN: 978-3-030-60151-5

Electronic ISBN: 978-3-030-60152-2

Copyright Year: 2020

DOI
https://doi.org/10.1007/978-3-030-60152-2_1