Georgios Gousios
Margaret-Anne Storey
Alberto Bacchelli
ABSTRACT
The pull-based development model is an emerging way of contributing to distributed software projects that is gaining enormous popularity within the open source software (OSS) world. Previous work has examined this model by focusing on projects and their owners---we complement it by examining the work practices of project contributors and the challenges they face.
We conducted a survey with 645 top contributors to active OSS projects using the pull-based model on GitHub, the prevalent social coding site. We also analyzed traces extracted from corresponding GitHub repositories. Our research shows that: contributors have a strong interest in maintaining awareness of project status to get inspiration and avoid duplicating work, but they do not actively propagate information; communication within pull requests is reportedly limited to low-level concerns and contributors often use communication channels external to pull requests; challenges are mostly social in nature, with most reporting poor responsiveness from integrators; and the increased transparency of this setting is a confirmed motivation to contribute. Based on these findings, we present recommendations for practitioners to streamline the contribution process and discuss potential future research directions.
- GitHub is your new resume. https://news.ycombinator.com/item?id=2763182. Accessed 2016/02/15.Google Scholar
- A. Bacchelli and C. Bird. Expectations, outcomes, and challenges of modern code review. In Proceedings of the 2013 International Conference on Software Engineering, ICSE '13, pages 712--721, Piscataway, NJ, USA, 2013. IEEE Press. Google ScholarDigital Library
- O. Baysal, O. Kononenko, R. Holmes, and M. Godfrey. The secret life of patches: A firefox case study. In Reverse Engineering (WCRE), 2012 19th Working Conference on, pages 447--455, Oct 2012. Google ScholarDigital Library
- C. Bird, A. Gourley, and P. Devanbu. Detecting patch submission and acceptance in oss projects. In Proceedings of the Fourth International Workshop on Mining Software Repositories, MSR '07, pages 26--, Washington, DC, USA, 2007. IEEE Computer Society. Google ScholarDigital Library
- C. Bird and T. Zimmermann. Assessing the value of branches with what-if analysis. In Proceedings of the 20th International Symposium on Foundations of Software Engineering, November 2012. Google ScholarDigital Library
- Bitbucket. http://bitbucket.org/. Accessed 2016/02/15.Google Scholar
- B. Bleikamp. How we use pull requests to build GitHub. https://github.com/blog/1124-how-we-use-pull-requests-to-build-github. Accessed 2016/02/15.Google Scholar
- J. Coglan. Why GitHub is not your cv. https://blog.jcoglan.com/2013/11/15/why-github-is-not-your-cv/, November 2013. Accessed 2016/02/15.Google Scholar
- J. M. Corbin and A. Strauss. Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative Sociology, 13(1):3--21, 1990.Google ScholarCross Ref
- J. W. Creswell. Research design: Qualitative, quantitative, and mixed methods approaches. Sage Publications, 3rd edition, 2009.Google Scholar
- L. Dabbish, C. Stuart, J. Tsay, and J. Herbsleb. Social coding in Github: transparency and collaboration in an open software repository. In Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work, CSCW '12, pages 1277--1286, New York, NY, USA, 2012. ACM. Google ScholarDigital Library
- L. Dabbish, C. Stuart, J. Tsay, and J. Herbsleb. Leveraging transparency. IEEE Software, 30(1):37--43, 2013. Google ScholarDigital Library
- M. Dias, A. Bacchelli, G. Gousios, D. Cassou, and S. Ducasse. Untangling fine-grained code changes. In Proceedings of the 22nd International Conference on Software Analysis, Evolution, and Reengineering, SANER 2015, pages 341--350. IEEE Computer Society, 2015.Google ScholarCross Ref
- P. Dourish and V. Bellotti. Awareness and coordination in shared workspaces. In Proceedings of the ACM conference on Computer-supported cooperative work, pages 107--114. ACM, 1992. Google ScholarDigital Library
- N. Ducheneaut. Socialization in an open source software community: A socio-technical analysis. Computer Supported Cooperative Work (CSCW), 14(4):323--368, 2005. Google ScholarDigital Library
- U. Flick. An introduction to qualitative research. SAGE Publications, 5th edition, 2014.Google Scholar
- K. Fogel. Producing Open Source Software. O'Reilly Media, first edition, 2005. Google ScholarDigital Library
- A. Furnham. Response bias, social desirability and dissimulation. Personality and Individual Differences, 7(3):385--400, 1986.Google ScholarCross Ref
- E. Giger, M. Pinzger, and H. Gall. Predicting the fix time of bugs. In Proceedings of the 2nd International Workshop on Recommendation Systems for Software Engineering, pages 52--56. ACM, 2010. Google ScholarDigital Library
- GitHub. https://github.com/. Accessed 2016/02/15.Google Scholar
- Gitorious. http://gitorious.org/. Accessed 2016/02/15.Google Scholar
- D. Goode and S. Agarwal. https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-facebook/. Accessed 2016/02/15.Google Scholar
- G. Gousios. http://ghtorrent.org/pullreq-perf/. Accessed 2016/02/15.Google Scholar
- G. Gousios. The GHTorrent dataset and tool suite. In Proceedings of the 10th Working Conference on Mining Software Repositories, MSR '13, pages 233--236, Piscataway, NJ, USA, 2013. IEEE Press. Google ScholarDigital Library
- G. Gousios, M. Pinzger, and A. van Deursen. An exploratory study of the pull-based software development model. In Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, pages 345--355, New York, NY, USA, 2014. ACM. Google ScholarDigital Library
- G. Gousios, M.-A. Storey, and A. Bacchelli. Pull request contributors analysis dataset. http://dx.doi.org/10.5281/zenodo.46063, Feb. 2016.Google Scholar
- G. Gousios, A. Zaidman, M.-A. Storey, and A. v. Deursen. Work practices and challenges in pull-based development: The integrator's perspective. In Proceedings of the 37th International Conference on Software Engineering, ICSE 2015, 2015. Google ScholarDigital Library
- A. Guzzi, A. Bacchelli, M. Lanza, M. Pinzger, and A. van Deursen. Communication in open source software development mailing lists. In Proceedings of MSR 2013 (10th IEEE Working Conference on Mining Software Repositories), pages 277--286, 2013. Google ScholarDigital Library
- K. Herzig and A. Zeller. The impact of tangled code changes. In Proceedings of 10th Conference on Mining Software Repositories, pages 121--130. IEEE, 2013. Google ScholarDigital Library
- Y. Jiang, B. Adams, and D. M. German. Will my patch make it? and how fast?: case study on the linux kernel. In Proceedings of the 10th Working Conference on Mining Software Repositories, pages 101--110. IEEE Press, 2013. Google ScholarDigital Library
- E. Kalliamvakou, G. Gousios, K. Blincoe, L. Singer, D. M. German, and D. Damian. The promises and perils of mining github. In Proceedings of the 11th Working Conference on Mining Software Repositories, pages 92--101, 2014. Google ScholarDigital Library
- A. Kumar. http://www.infoq.com/presentations/Development-at-Google. Accessed 2016/02/15.Google Scholar
- J. Lerner and J. Tirole. Some simple economics of open source. The journal of industrial economics, 50(2):197--234, 2002.Google Scholar
- J. Marlow, L. Dabbish, and J. Herbsleb. Impression formation in online peer production: activity traces and personal profiles in GitHub. In Proceedings of the 2013 conference on Computer supported cooperative work, CSCW '13, pages 117--128, New York, NY, USA, 2013. ACM. Google ScholarDigital Library
- N. McDonald and S. Goggins. Performance and participation in open source software on GitHub. In Extended Abstracts on Human Factors in Computing Systems, CHI EA '13, pages 139--144, New York, NY, USA, 2013. ACM. Google ScholarDigital Library
- A. Mockus, R. T. Fielding, and J. D. Herbsleb. Two case studies of open source software development: Apache and Mozilla. ACM Trans. Softw. Eng. Methodol., 11(3):309--346, 2002. Google ScholarDigital Library
- J. Nielsen. 10 usability heuristics for user interface design. http://www.nngroup.com/articles/ten-usability-heuristics/, January 1995.Google Scholar
- R. Pham, L. Singer, O. Liskin, F. Figueira Filho, and K. Schneider. Creating a shared understanding of testing culture on a social coding site. In Proceedings of the 2013 International Conference on Software Engineering, ICSE '13, pages 112--121, Piscataway, NJ, USA, 2013. IEEE Press. Google ScholarDigital Library
- T. Punter, M. Ciolkowski, B. Freimut, and I. John. Conducting on-line surveys in software engineering. In International Symposium on Empirical Software Engineering, ISESE'03, pages 80--88. IEEE, 2003. Google ScholarDigital Library
- P. C. Rigby and C. Bird. Convergent contemporary software peer review practices. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2013, pages 202--212, New York, NY, USA, 2013. ACM. Google ScholarDigital Library
- P. C. Rigby and M.-A. Storey. Understanding broadcast based peer review on open source software projects. In Proceedings of the 33rd International Conference on Software Engineering, ICSE '11, pages 541--550, New York, NY, USA, 2011. ACM. Google ScholarDigital Library
- J. Sheehy. There is no now. ACM Queue, 13(3):1--8, 2015. Google ScholarDigital Library
- L. Sigelaman. Question-order effects on presidential popularity. Public Opinion Quarterly, 45(2):199--207, 1981.Google ScholarCross Ref
- I. Steinmacher, T. U. Conte, M. Gerosa, and D. Redmiles. Social barriers faced by newcomers placing their first contribution in open source software projects. In Proceedings of the 18th ACM conference on Computer supported cooperative work & social computing, pages 1379--1392, 2015. Google ScholarDigital Library
- J. Tsay, L. Dabbish, and J. Herbsleb. Influence of social and technical factors for evaluating contribution in GitHub. In Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, pages 356--366, New York, NY, USA, 2014. ACM. Google ScholarDigital Library
- J. Tsay, L. Dabbish, and J. Herbsleb. Let's talk about it: Evaluating contributions through discussion in github. In Proceedings of the 22Nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2014, pages 144--154, New York, NY, USA, 2014. ACM. Google ScholarDigital Library
- E. van der Veen, G. Gousios, and A. Zaidman. Automatically prioritizing pull requests. In Proceedings of the 2015 International Working Conference on Mining Software Repositories, pages 357--361, 2015. Google ScholarDigital Library
- B. Vasilescu, Y. Yu, H. Wang, P. Devanbu, and V. Filkov. Quality and productivity outcomes relating to continuous integration in GitHub. In 10th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC/FSE, page accepted. IEEE, 2015. Google ScholarDigital Library
- G. von Krogh, S. Spaeth, and K. R. Lakhani. Community, joining, and specialization in open source software innovation: a case study. Research Policy, 32(7):1217--1241, 2003. Open Source Software Development.Google ScholarCross Ref
- B. Weiss. GitHub is your resume now. http://anti-pattern.com/github-is-your-resume-now, June 2012. Accessed 2016/02/15.Google Scholar
- P. Weissgerber, D. Neu, and S. Diehl. Small patches get in! In Proceedings of the 2008 International Working Conference on Mining Software Repositories, MSR '08, pages 67--76, New York, NY, USA, 2008. ACM. Google ScholarDigital Library
- Y. Ye and K. Kishida. Toward an understanding of the motivation of open source software developers. In Proceedings of the 25th International Conference on Software Engineering, 2003, pages 419--429, May 2003. Google ScholarDigital Library
- H. Zhang, L. Gong, and S. Versteeg. Predicting bug-fixing time: an empirical study of commercial software projects. In Proceedings of the 2013 International Conference on Software Engineering, pages 1042--1051. IEEE Press, 2013. Google ScholarDigital Library
- M. Zhou and A. Mockus. Who will stay in the FLOSS community? modeling participant's initial behavior. IEEE Transactions on Software Engineering, 41(1):82--99, 2015.Google ScholarDigital Library
Index Terms
- Work practices and challenges in pull-based development: the contributor's perspective
Recommendations
An exploratory study of the pull-based software development model
ICSE 2014: Proceedings of the 36th International Conference on Software EngineeringThe advent of distributed version control systems has led to the development of a new paradigm for distributed software development; instead of pushing changes to a central repository, developers pull them from other repositories and merge them ...
A dataset for pull-based development research
MSR 2014: Proceedings of the 11th Working Conference on Mining Software RepositoriesPull requests form a new method for collaborating in distributed software development. To study the pull request distributed development model, we constructed a dataset of almost 900 projects and 350,000 pull requests, including some of the largest ...
Developers assignment for analyzing pull requests
SAC '15: Proceedings of the 30th Annual ACM Symposium on Applied ComputingA new collaboration approach is becoming increasingly common in open-source projects: the pull request model. In this kind of collaboration, developers that do not belong to the core team of a project can submit contributions to the core team. In ...
Comments