Top

Empirical Software Engineering

Published in:

01-12-2022

Pitfalls and guidelines for using time-based Git data

Authors: Samuel W. Flint, Jigyasa Chauhan, Robert Dyer

Published in: Empirical Software Engineering | Issue 7/2022

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Many software engineering research papers rely on time-based data (e.g., commit timestamps, issue report creation/update/close dates, release dates). Like most real-world data however, time-based data is often dirty. To date, there are no studies that quantify how frequently such data is used by the software engineering research community, or investigate sources of and quantify how often such data is dirty. Depending on the research task and method used, including such dirty data could affect the research results. This paper presents an extended survey of papers that utilize time-based data, published in the Mining Software Repositories (MSR) conference series. Out of the 754 technical track and data papers published in MSR 2004–2021, we saw at least 290 (38%) papers utilized time-based data. We also observed that most time-based data used in research papers comes in the form of Git commits, often from GitHub. Based on those results, we then used the Boa and Software Heritage infrastructures to help identify and quantify several sources of dirty Git timestamp data. Finally we provide guidelines/best practices for researchers utilizing time-based data from Git repositories.

previous article DiverGet: a Search-Based Software Testing approach for Deep Neural Network Quantization assessment

next article Developers’ perception matters: machine learning to detect developer-sensitive smells

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

All authors brainstormed potential keywords and helped create the final list.

https://github.com/MetricsGrimoire/CVSAnalY

Note that: “While Subversion automatically attaches properties (svn:date, svn:author, svn:log, and so on) to revisions, it does not presume thereafter the existence of those properties, and neither should you or the tools you use to interact with your repository.” https://svnbook.red-bean.com/en/1.7/svn.advanced.props.html

The Kotlin dataset contains some projects which may exist in the Java dataset.

The SF.net dataset contained Subversion projects, which store commit IDs as integers and thus are not unique across projects and can not be easily deduplicated.

https://archive.softwareheritage.org/browse/origin/log/?origin_url=https://github.com/KevinHoward/Irony&timestamp=2015-07-29T09:07:18Z

https://archive.softwareheritage.org/browse/origin/log/?origin_url=https://github.com/maodouzi/PY&timestamp=2015-08-07T07:29:54Z

https://www.nltk.org/

https://stackoverflow.com/questions/633353

https://stackoverflow.com/questions/52507279

https://stackoverflow.com/questions/16259105

https://gerritcodereview.com

https://www.mercurial-scm.org/wiki/HgGit

https://opensource.google/projects/moe

https://github.com/scrapy/scrapy

http://boa.cs.iastate.edu/boa/?q=content/dataset-notes-october-2019

Ahasanuzzaman M, Asaduzzaman M, Roy CK, Schneider KA (2016) Mining duplicate questions in stack overflow. In: Proceedings of the 13th international conference on mining software repositories, MSR ’16. Association for Computing Machinery, New York, pp 402–412. https://doi.org/10.1145/2901739.2901770

Antoniol G, Rollo VF, Venturi G (2005) Linear predictive coding and cepstrum coefficients for mining time variant information from software repositories. In: Proceedings of the 2005 international workshop on mining software repositories, MSR ’05, vol 2005. Association for Computing Machinery, New York, pp 1–5. https://doi.org/10.1145/1083142.1083156

Baysal O, Holmes R, Godfrey MW (2012) Mining usage data and development artifacts. In: 2012 9th IEEE working conference on mining software repositories (MSR), pp 98–107. https://doi.org/10.1109/MSR.2012.6224305

Bird C, Rigby PC, Barr ET, Hamilton DJ, German DM, Devanbu P (2009) The promises and perils of mining Git. In: 2009 6th IEEE international working conference on mining software repositories, pp 1–10. https://doi.org/10.1109/MSR.2009.5069475

Cito J, Schermann G, Wittern JE, Leitner P, Zumberi S, Gall HC (2017) An empirical analysis of the Docker container ecosystem on GitHub. In: 2017 IEEE/ACM 14th international conference on mining software repositories (MSR). IEEE. https://doi.org/10.1109/msr.2017.67

Claes M, Mäntylä MV (2020) 20-MAD: 20 years of issues and commits of Mozilla and Apache development. In: Proceedings of the 17th international conference on mining software repositories, MSR ’20. Association for Computing Machinery, New York, pp 503–507. https://doi.org/10.1145/3379597.3387487

Cosentino V, Izquierdo JLC, Cabot J (2016) Findings from GitHub: methods, datasets and limitations. In: 2016 IEEE/ACM 13th working conference on mining software repositories (MSR), pp 137–141

Cosmo RD, Zacchiroli S (2017) Software Heritage: why and how to preserve software source code. In: iPRES 2017: 14th international conference on digital preservation. Kyoto, Japan

D’Ambros M, Lanza M, Robbes R (2010) An extensive comparison of bug prediction approaches. In: 2010 7th IEEE working conference on mining software repositories (MSR 2010), pp 31–41. https://doi.org/10.1109/MSR.2010.5463279

Demeyer S, Murgia A, Wyckmans K, Lamkanfi A (2013) Happy birthday! A trend analysis on past MSR papers. In: Proceedings of the 10th working conference on mining software repositories, MSR ’13. IEEE Press, pp 353–362

Durieux T, Le Goues C, Hilton M, Abreu R (2020) Empirical study of restarted and flaky builds on Travis CI. In: Proceedings of the 17th international conference on mining software repositories, MSR ’20. Association for Computing Machinery, New York, pp 254–264. https://doi.org/10.1145/3379597.3387460

Dyer R, Nguyen HA, Rajan H, Nguyen TN (2013) Boa: a language and infrastructure for analyzing ultra-large-scale software repositories. In: Proceedings of the international conference on software engineering, ICSE ’13, vol 2013. IEEE Press, pp 422–431. https://doi.org/10.5555/2486788.2486844

Dyer R, Nguyen HA, Rajan H, Nguyen TN (2021) Boa: Mining ultra-large-scale software repositories. http://boa.cs.iastate.edu/boa/. Accessed 14 Oct 2021

Flint SW, Chauhan J, Dyer R (2021a) Escaping the time pit: pitfalls and guidelines for using time-based Git data. In: 2021 IEEE/ACM 18th international conference on mining software repositories (MSR), pp 85–96. https://doi.org/10.1109/MSR52588.2021.00022

Flint SW, Chauhan J, Dyer R (2021b) Replication package for “Pitfalls and Guidelines for Using Time-Based GitData From Java, Kotlin, and Python Projects”. https://doi.org/10.5281/zenodo.5558291

Gasser L, Ripoche G, Sandusky RJ (2004) Research infrastructure for empirical science of F/OSS. In: Proceedings of the 1st international workshop on mining software repositories

Ghezzi G, Gall HC (2013) Replicating mining studies with SOFAS. In: Proceedings of the 10th working conference on mining software repositories, MSR ’13. IEEE Press, pp 363–372

Goeminne M, Claes M, Mens T (2013) A historical dataset for the Gnome ecosystem. In: 2013 10th working conference on mining software repositories (MSR), pp 225–228. https://doi.org/10.1109/MSR.2013.6624032

Gonzalez-Barahona JM, Robles G, Izquierdo-Cortazar D (2015) The MetricsGrimoire database collection. In: Proceedings of the 12th working conference on mining software repositories, MSR ’15. IEEE Press, pp 478–481

Hayashi J, Higo Y, Matsumoto S, Kusumoto S (2019) Impacts of daylight saving time on software development. In: Proceedings of the 16th international conference on mining software repositories, MSR ’19. IEEE Press, pp 502–506. https://doi.org/10.1109/MSR.2019.00076

Hemmati H, Nadi S, Baysal O, Kononenko O, Wang W, Holmes R, Godfrey MW (2013) The MSR cookbook: mining a decade of research. In: Proceedings of the 10th working conference on mining software repositories, MSR ’13. IEEE Press, pp 343–352

Kagdi H, Yusuf S, Maletic JI (2006) Mining sequences of changed-files from version histories. In: Proceedings of the 2006 international workshop on mining software repositories, MSR ’06. Association for Computing Machinery, New York, pp 47–53. https://doi.org/10.1145/1137983.1137996

Kalliamvakou E, Gousios G, Blincoe K, Singer L, German DM, Damian D (2014) The promises and perils of mining GitHub. In: Proceedings of the 11th working conference on mining software repositories, MSR 2014. Association for Computing Machinery, New York, pp 92–101. https://doi.org/10.1145/2597073.2597074

Kalliamvakou E, Gousios G, Blincoe K, Singer L, German DM, Damian D (2016) An in-depth study of the promises and perils of mining GitHub. Empirical Softw Engg 21(5):2035–2071. https://doi.org/10.1007/s10664-015-9393-5CrossRef

Karampatsis RM, Sutton C (2020) How often do single-statement bugs occur?. In: Proceedings of the 17th international conference on mining software repositories. ACM. https://doi.org/10.1145/3379597.3387491

Kikas R, Dumas M, Pfahl D (2016) Using dynamic and contextual features to predict issue lifetime in GitHub projects. In: Proceedings of the 13th international conference on mining software repositories, MSR ’16. Association for Computing Machinery, New York, pp 291–302. https://doi.org/10.1145/2901739.2901751

Kotti Z, Spinellis D (2019) Standing on shoulders or feet? The usage of the MSR data papers. In: Proceedings of the 16th international conference on mining software repositories, MSR ’19. IEEE Press, pp 565–576. https://doi.org/10.1109/MSR.2019.00085

Liu Y, Lin J, Cleland-Huang J (2020) Traceability support for multi-lingual software projects. In: Proceedings of the 17th international conference on mining software repositories, MSR ’20. Association for Computing Machinery, New York, pp 443–454. https://doi.org/10.1145/3379597.3387440

Pietri A, Rousseau G, Zacchiroli S (2020) Forking without clicking: on how to identify software repository forks. In: Proceedings of the 17th international conference on mining software repositories. Association for Computing Machinery, New York, pp 277–287

Pimentel JaF, Murta L, Braganholo V, Freire J (2019) A large-scale study about quality and reproducibility of Jupyter notebooks. In: Proceedings of the 16th international conference on mining software repositories, MSR ’19. IEEE Press, pp 507–517. https://doi.org/10.1109/MSR.2019.00077

Robles G (2010) Replicating MSR: A study of the potential replicability of papers published in the Mining Software Repositories proceedings. In: 7th IEEE working conference on mining software repositories, MSR ’10, pp 171–180. https://doi.org/10.1109/MSR.2010.5463348

Robles G, González-Barahona JM, Cervigón C, Capiluppi A, Izquierdo-Cortázar D (2014) Estimating development effort in free/open source software projects by mining software repositories: a case study of OpenStack. In: Proceedings of the 11th working conference on mining software repositories, MSR 2014. Association for Computing Machinery, New York, pp 222–231. https://doi.org/10.1145/2597073.2597107

Sadowski C, Lewis C, Lin Z, Zhu X, Whitehead EJ (2011) An empirical analysis of the FixCache algorithm. In: Proceedings of the 8th working conference on mining software repositories, MSR ’11. Association for Computing Machinery, New York, pp 219–222. https://doi.org/10.1145/1985441.1985475

Software Heritage developers (2020) Software Heritage archive. https://archive.softwareheritage.org/. Accessed 28 Dec 2020

Steff M, Russo B (2012) Co-evolution of logical couplings and commits for defect estimation. In: Proceedings of the 9th IEEE working conference on mining software repositories, MSR ’12. IEEE Press, pp 213–216

Walker RJ, Holmes R, Hedgeland I, Kapur P, Smith A (2006) A lightweight approach to technical risk estimation via probabilistic impact analysis. In: Proceedings of the 2006 international workshop on mining software repositories, MSR ’06. Association for Computing Machinery, New York, pp 98–104. https://doi.org/10.1145/1137983.1138008

Wang P, Brown C, Jennings JA, Stolee KT (2020) An empirical study on regular expression bugs. In: Proceedings of the 17th international conference on mining software repositories. ACM. https://doi.org/10.1145/3379597.3387464

Xu Y, Zhou M (2018) A multi-level dataset of Linux kernel patchwork. In: Proceedings of the 15th international conference on mining software repositories, MSR ’18. Association for Computing Machinery, New York, pp 54–57. https://doi.org/10.1145/3196398.3196475

Zhu J, Wei J (2019) An empirical study of multiple names and email addresses in OSS version control repositories. In: 2019 IEEE/ACM 16th international conference on mining software repositories (MSR). IEEE. https://doi.org/10.1109/msr.2019.00068

Zimmermann T, Weißgerber P (2004) Preprocessing CVS data for fine-grained analysis. In: Proceedings of the 1st international workshop on mining software repositories, MSR ’04, pp 2–6

Title: Pitfalls and guidelines for using time-based Git data
Authors: Samuel W. Flint
Jigyasa Chauhan
Robert Dyer
Publication date: 01-12-2022
Publisher: Springer US
Published in: Empirical Software Engineering / Issue 7/2022
Print ISSN: 1382-3256
Electronic ISSN: 1573-7616
DOI: https://doi.org/10.1007/s10664-022-10200-y

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 7/2022

Developing research questions in conversation with the literature: operationalization & tool support

Exploring the relationship between performance metrics and cost saving potential of defect prediction models

The human experience of comprehending source code in virtual reality

A large-scale empirical study of commit message generation: models, datasets and evaluation

A controlled experiment of different code representations for learning-based program repair

Static detection of equivalent mutants in real-time model-based mutation testing

Premium Partner