nach oben

Empirical Software Engineering

Erschienen in:

01.01.2023

A controlled experiment on the impact of microtasking on programming

verfasst von: Emad Aghayi, Thomas D. LaToza

Erschienen in: Empirical Software Engineering | Ausgabe 1/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

In microtask programming, developers complete short self-contained microtasks through the use of a specialized programming environment. For example, given only a short description of the purpose of a function and a partially complete implementation, a developer might be asked to identify, test, and implement an additional behavior in the function. Adopting a microtask approach to programming tasks has been envisioned to offer a number of potential benefits, including reducing the onboarding time necessary for new developers to contribute to a project and achieving higher project velocity by enabling larger project teams and greater parallelism. To investigate the potential benefits and drawbacks of microtask programming we conducted a controlled experiment. We focused our investigation on the context in which microtasking is most widely used, implementing and debugging function bodies, and investigated the impact of microtasking with respect to onboarding, project velocity, code quality, and developer productivity. 28 developers worked to implement microservice endpoints, either in the form of traditional programming tasks described in an issue tracker or as programming microtasks. Our study did not examine the design effort necessary to prepare for microtask programming or consider how microtask programming might be applied to maintenance tasks. We found that, compared to traditional programming, microtask programming reduced onboarding time by a factor of 3.7, increased project velocity by a factor of 5.76 by increasing team size by a factor of 7, and decreased individual developer productivity by a factor of 1.3. The quality of code did not significantly differ. Through qualitative analysis of how developers worked, we explore potential explanations of these differences. These findings offer evidence for the potential benefits that adopting microtask programming might bring, particularly in cases where increasing project velocity is paramount.

Vorheriger Artikel Automatic prediction of rejected edits in Stack Overflow

Nächster Artikel An empirical study on a single company’s cost estimations of 338 software projects

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

https://www.utest.com

https://expressjs.com/

https://firebase.google.com/

https://doi.org/10.5281/zenodo.4922866

Aberdour M (2007) Achieving quality in open-source software. IEEE Softw, 58–64

Aghayi E (2020) Large-scale microtask programming. In: Symposium on visual languages and human-centric computing, pp 1–2

Aghayi E, LaToza TD Replication package of this study. https://doi.org/10.5281/zenodo.4922866

Aghayi E, LaToza TD, Surendra P, Abolghasemi S (2021) Crowdsourced behavior-driven development. J Syst Softw 171:110840CrossRef

Ammann P, Offutt J (2016) Introduction to software testing. Cambridge University Press

Basili VR (2001) Open source and empirical software engineering. Empir Softw Eng, 193–194

Bavota G, Qusef A, Oliveto R, De Lucia A, Binkley D (2012) An empirical analysis of the distribution of unit test smells and their impact on software maintenance. In: International conference on software maintenance, pp 56–65

Begel A, Simon B (2008) Novice software developers, all over again. In: International workshop on computing education research, pp 3–14

Bernstein M, Little G, Miller RC, Hartmann B, Ackerman MS, Karger DR, Crowell D, Panovich K (2010) Soylent: a word processor with a crowd inside. In: Symposium on user interface software and technology, pp 313–322

Britto R, Smite D, Damm LO, Börstler J (2020) Evaluating and strategizing the onboarding of software developers in large-scale globally distributed projects. J Syst Softw, 169

Chen Y, Lee SW, Xie Y, Yang Y, Lasecki WS, Oney S (2017) Codeon: on-demand software development assistance. In: Conference on human factors in computing systems, pp 6220–6231

Chen Y, Oney S, Lasecki WS (2016) Towards providing on-demand expert support for software developers. In: Conference on human factors in computing systems, pp 3192–3203

Cheng J, Teevan J, Iqbal S, Bernstein M (2015) Break it down: a comparison of macro-and microtasks. In: Conference on human factors in computing systems, pp 4061–4064

Cutrell EB, Czerwinski M, Horvitz E (2000) Effects of instant messaging interruptions on computing tasks. In: Extended abstracts on human factors in computing systems, pp 99–100

Czerwinski M, Horvitz E, Wilhite S (2004) A diary study of task switching and interruptions. In: Conference on human factors in computing systems, pp 175–182

Doan A, Ramakrishnan R, Halevy AY (2011) Crowdsourcing systems on the world-wide web. Commun ACM 54:86–96CrossRef

Espinosa A, Kraut R, Lerch J, Slaughter S, Herbsleb J, Mockus A (2001) Shared mental models and coordination in large-scale, distributed software development. International Conference on Information Systems, 64

Fagerholm F, Guinea AS, Borenstein J, Münch J (2014) Onboarding in open source projects. IEEE Softw, 54–61

Fagerholm F, Johnson P, Guinea AS, Borenstein J, Münch J (2013) Onboarding in open source software projects: a preliminary analysis. In: International conference on global software engineering workshops, pp 5–10. IEEE

Filippova A, Cho H (2016) The effects and antecedents of conflict in free and open source software development. In: Conference on computer-supported cooperative work and social computing, pp 705–716

Goldman M (2012) Software development with real-time collaborative editing. Ph.D. thesis Massachusetts Institute of Technology

Goldman M, Little G, Miller RC (2011) Real-time collaborative coding in a web ide. In: Symposium on user interface software and technology, pp 155–164

Goldman M, et al. (2012) Software development with real-time collaborative editing. Ph.D. thesis Massachusetts Institute of Technology

Hoseini M, Saghafi F, Aghayi E (2018) A multidimensional model of knowledge sharing behavior in mobile social networks Kybernetes

Iqbal S, Bailey BP (2008) Effects of intelligent notification management on users and their tasks. In: Conference on human factors in computing systems, pp 93–102

Iqbal S, Teevan J, Liebling D, Thompson AL (2018) Multitasking with play write, a mobile microproductivity writing tool. In: Symposium on user interface software and technology, pp 411–422

Jensen RW (2014) Improving software development productivity: Effective leadership and quantitative methods in software management. Pearson Education

Jergensen C, Sarma A, Wagstrom P (2011) The onion patch: migration in open source ecosystems. In: Special interest group on software engineering symposium and the european conference on foundations of software engineering, pp 70–80

Jiang H, Matsubara S (2014) Efficient task decomposition in crowdsourcing. In: International conference on principles and practice of multi-agent systems, pp 65–73

Johnson M, Senges M (2010) Learning to be a programmer in a complex organization: A case study on practice-based learning during the onboarding process at google. Journal of Workplace Learning

Kamma D, Jalote P (2013) Effect of task processes on programmer productivity in model-based testing. In: India software engineering conference, pp 23–28

Kersten M, Murphy GC (2006) Using task context to improve programmer productivity. In: International symposium on foundations of software engineering, pp 1–11

Kim J, Sterman S, Cohen AAB, Bernstein M (2017) Mechanical novel: crowdsourcing complex work through reflection and revision. In: Conference on computer supported cooperative work and social computing

Kittur A, Nickerson JV, Bernstein M, Gerber E, Shaw A, Zimmerman J, Lease M, Horton J (2013) The future of crowd work. In: Conference on computer supported cooperative work, pp 1301–1318

Kittur A, Smus B, Khamkar S, Kraut RE (2011) Crowdforge: crowdsourcing complex work. In: Symposium on user interface software and technology, pp 43–52

Kochhar PS, Bissyandé TF, Lo D, Jiang L (2013) An empirical study of adoption of software testing in open source projects. In: International conference on quality software, pp 103–112

Lakhani KR, Garvin DA, Lonstein E (2010) Topcoder (a): developing software through crowdsourcing Harvard Business School General Management Unit Case

Lasecki WS, Kim J, Rafter N, Sen O, Bigham JP, Bernstein M (2015) Apparition: crowdsourced user interfaces that come to life as you sketch them. In: Human factors in computing systems, pp 1925–1934

LaToza TD, Chiquillo E, Towne WB, Adriano C, van der Hoek A (2013) Crowdcode: a platform for crowd development. In: CrowdConf

LaToza TD, Di Lecce A, Ricci F, Towne B, Van der Hoek A (2018) Microtask programming. Transactions on Software Engineering, 1–20

LaToza TD, van der Hoek A (2016) Crowdsourcing in software engineering: models, motivations, and challenges. IEEE software, 74–80

LaToza TD, Lecce AD, Ricci F, Towne WB, van der Hoek A (2015) Ask the crowd: scaffolding coordination and knowledge sharing in microtask programming. In: Symposium on visual languages and human-centric computing, pp 23–27

LaToza TD, Towne WB, Adriano CM, Van Der Hoek A (2014) Microtask programming: building software with a crowd. In: Symposium on user interface software and technology, pp 43–54

LaToza TD, Van Der Hoek A (2015) Crowdsourcing in software engineering: models, motivations, and challenges. IEEE Softw, 74–80

Lee SW, Krosnick R, Park SY, Keelean B, Vaidya S, O’Keefe SD, Lasecki WS (2018) Exploring real-time collaboration in crowd-powered systems through a ui design tool. Computer-Supported Cooperative Work and Social Computing, 1–23

Mao K, Capra L, Harman M, Jia Y (2017) A survey of the use of crowdsourcing in software engineering. J Syst Softw, 57–84

Mark G, Gonzalez VM, Harris J (2005) No task left behind? Examining the nature of fragmented work. In: Conference on human factors in computing systems, pp 321–330

Martin J, McClure CL (1983) Software maintenance: the problems and its solutions prentice hall professional technical reference

Maxwell KD (2003) Software development productivity. Adv Comput 58:1–46CrossRef

Meyer AN, Fritz T, Murphy GC, Zimmermann T (2014) Software developers’ perceptions of productivity. In: International symposium on foundations of software engineering, pp 19–29

Nebeling M, Leone S, Norrie MC (2012) Crowdsourced web engineering and design. In: International conference on web engineering, pp 31–45

Oliveira E, Viana D, Cristo M, Conte T (2017) How have software engineering researchers been measuring software productivity?-a systematic mapping study. In: International conference on enterprise information systems, pp 76–87

Perry DE, Siy HP, Votta LG (2001) Parallel changes in large-scale software development: an observational case study. Transactions on Software Engineering and Methodology, 308–337

Peterson B (2017) Travis kalanick lasted in his role for 6.5 years—five times longer than the average uber employee. Business Insider

Ramakrishnan R, Baptist A, Ercegovac V, Hanselman M, Kabra N, Marathe A, Shaft U (2004) Mass collaboration: a case study. In: International database engineering and applications symposium, pp 133–146

Retelny D, Bernstein M, Valentine MA (2017) No workflow can ever be enough: how crowdsourcing workflows constrain complex work. In: Conference on Computer-Supported Cooperative Work and Social Computing, 1–23

Sadowski C, Zimmermann T (2019) Rethinking productivity in software engineering. Springer Nature

Saengkhattiya M, Sevandersson M, Vallejo U (2012) Quality in crowdsourcing-how software quality is ensured in software crowdsourcing. Master’s thesis, Department of Informatics Lund University

Saito S, Iimura Y, Aghayi E, LaToza TD (2020) Can microtask programming work in industry?. In: European software engineering conference and symposium on the foundations of software engineering, pp 1263–1273

Schiller TW, Ernst MD (2012) Reducing the barriers to writing verified specifications. Special Interest Group on Programming Languages Notices, 95–112

Sim SE, Holt RC (1998) The ramp-up problem in software projects: a case study of how software immigrants naturalize. In: International conference on software engineering, pp 361–370

Stamelos I, Angelis L, Oikonomou A, Bleris GL (2002) Code quality analysis in open source software development. Inf Syst J, 43–60

Steinmacher I, Conte TU, Treude C, Gerosa MA (2016) Overcoming open source project entry barriers with a portal for newcomers. In: International conference on software engineering, pp 273–284

Steinmacher I, Silva MAG, Gerosa MA, Redmiles DF (2015) A systematic literature review on the barriers faced by newcomers to open source software projects. Inf Softw Technol, 67–85

Stol KJ, Fitzgerald B (2014) Two’s company, three’s a crowd: a case study of crowdsourcing software development. In: International conference on software engineering, pp 187–198

Teevan J, Iqbal S, Cai CJ, Bigham JP, Bernstein M, Gerber EM (2016) Productivity decomposed: getting big things done with little microtasks. In: Conference extended abstracts on human factors in computing systems, pp 3500–3507

Tian J (2005) Software quality engineering: testing, quality assurance, and quantifiable improvement. Wiley

Trafton JG, Altmann EM, Brock DP, Mintz FE (2003) Preparing to resume an interrupted task: effects of prospective goal encoding and retrospective rehearsal. International Journal of Human-Computer Studies, 583–603

Von Ahn L, Dabbish L (2004) Labeling images with a computer game. In: Conference on human factors in computing systems, pp 319–326

Von Krogh G, Spaeth S, Lakhani KR (2003) Community, joining, and specialization in open source software innovation: a case study. Res Policy, 1217–1241

Wang J, Sarma A (2011) Which bug should i fix: helping new developers onboard a new project. In: International workshop on cooperative and human aspects of software engineering, pp 76–79

Warner J, Guo PJ (2017) Codepilot: scaffolding end-to-end collaborative software development for novice programmers. In: Conference on human factors in computing systems, pp 1136–1141

Weidema ER, Lopez C, Nayebaziz S, Spanghero F, van der Hoek A (2016) Toward microtask crowdsourcing software design work. In: International workshop on crowdsourcing in software engineering, pp 41–44

Weld DS, Wu F, Adar E, Amershi S, Fogarty J, Hoffmann R, Patel K, Skinner M (2008) Intelligence in wikipedia. In: AAAI, vol 8, pp 1609–1614

Williams A, Kaur H, Iqbal S, White RW, Teevan J, Fourney A (2019) Mercury: empowering programmers’ mobile work practices with microproductivity. In: Symposium on user interface software and technology

Witkower Z, Tracy JL, Cheng JT, Henrich J (2020) Two signals of social rank: prestige and dominance are associated with distinct nonverbal displays. J Pers Soc Psychol 118:89CrossRef

Zanatta AL, Machado L, Steinmacher I (2018) Competence, collaboration, and time management: barriers and recommendations for crowdworkers. In: Workshop on crowd sourcing in software engineering, pp 9–16

Zhao L, Elbaum S (2000) A survey on quality related activities in open source. SIGSOFT Software Engineering Notes, 54–57

Zhou M, Mockus A (2010) Developer fluency: achieving true mastery in software projects. In: International symposium on foundations of software engineering, pp 137–146

Titel: A controlled experiment on the impact of microtasking on programming
verfasst von: Emad Aghayi
Thomas D. LaToza
Publikationsdatum: 01.01.2023
Verlag: Springer US
Erschienen in: Empirical Software Engineering / Ausgabe 1/2023
Print ISSN: 1382-3256
Elektronische ISSN: 1573-7616
DOI: https://doi.org/10.1007/s10664-022-10226-2

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Weitere Artikel der Ausgabe 1/2023

Workflow analysis of data science code in public GitHub repositories

A systematic literature review on trust in the software ecosystem

Smells in system user interactive tests

Excluding code from test coverage: practices, motivations, and impact

A longitudinal explanatory case study of coordination in a very large development programme: the impact of transitioning from a first- to a second-generation large-scale agile development method

CyberSAGE: The cyber security argument graph evaluation tool

Premium Partner