nach oben

Empirical Software Engineering

Erschienen in:

01.07.2021

Evaluating network embedding techniques’ performances in software bug prediction

verfasst von: Yu Qu, Heng Yin

Erschienen in: Empirical Software Engineering | Ausgabe 4/2021

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Software bug prediction techniques can be very helpful in testing and code inspection. Over the past decade, network measures have been successfully used in bug prediction. Following the same intuition, recently, researchers started using network embedding techniques in bug prediction. However, existing studies only evaluated the Skip-gram and CBOW models with random walk. Considering network embedding is a fast-developing research direction, it is important to evaluate other network embedding techniques’ performances in bug prediction. Moreover, existing studies have not investigated the application and performance of network embedding in effort-aware bug prediction, which is thought to be a more realistic scenario that evaluates the cost effectiveness of bug prediction models. In this paper, we conduct an extensive empirical study to evaluate network embedding algorithms in bug prediction by utilizing and extending node2defect, a newly proposed bug prediction model that combines the embedded vectors with traditional software engineering metrics through concatenation. Experiments are conducted based on seven network embedding algorithms, two effort-aware models, and 13 open-source Java systems. Experimental results show that node2defect outperforms traditional metrics by + 14.64% in terms of MCC score, and by + 7.51% to + 16.57% in effort-aware bug prediction. More interestingly, when combined with CBS + , the embedded vectors alone can achieve the best performance. Among different network embedding algorithms, the newly proposed algorithm ProNE has the best performance.

Vorheriger Artikel Out of sight, out of mind? How vulnerable dependencies affect open-source projects

Nächster Artikel Embedding API dependency graph for neural code generation

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Throughout the paper, we use the terms “network embedding techniques” and “network embedding algorithms” interchangeably.

http://bug.inf.usi.ch/index.php

https://scikit-learn.org/stable/

https://scikit-learn.org/stable/modules/grid_search.html

https://github.com/thunlp/OpenNE

https://github.com/THUDM/ProNE

https://github.com/benedekrozemberczki/walklets

https://github.com/clowee/OpenSZZ

https://issues.apache.org/jira/

https://pmd.github.io/

https://pmd.github.io/latest/pmd_rules_java.html

Agrawal A, Menzies T (2018) Is better data better than better data miners?: on the benefits of tuning smote for defect prediction. In: Proceedings of the 40th international conference on software engineering. ACM, pp 1050–1061

Agrawal A, Fu W, Chen D, Shen X, Menzies T (2019) How to “dodge” complex software analytics. IEEE Trans Softw Eng:1–1. https://doi.org/10.1109/TSE.2019.2945020

Belkin M, Niyogi P (2001) Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Proceedings of the 14th international conference on neural information processing systems: natural and synthetic, pp 585–591

Bennin KE, Keung J, Phannachitta P, Monden A, Mensah S (2017) Mahakil: diversity based oversampling approach to alleviate the class imbalance issue in software defect prediction. IEEE Trans Softw Eng 44(6):534–550CrossRef

Bowes D, Hall T, Harman M, Jia Y, Sarro F, Wu F (2016) Mutation-aware fault prediction. In: Proceedings of the 25th international symposium on software testing and analysis, pp 330–341

Breiman L (2001) Random forests. Mach Learn 45(1):5–32CrossRef

Cai H, Zheng VW, Chang KCC (2018) A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Trans Knowl Data Eng 30(9):1616–1637CrossRef

Cao S, Lu W, Xu Q (2015) Grarep: learning graph representations with global structural information. In: Proceedings of the 24th ACM international on conference on information and knowledge management. ACM, pp 891–900

Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) Smote: synthetic minority over-sampling technique. J Artif Intell Res 16(1):321–357CrossRef

Chen L, Ma W, Zhou Y, Xu L, Wang Z, Chen Z, Xu B (2016) Empirical analysis of network measures for predicting high severity software faults. Science China Information Sciences 59(12):122901CrossRef

Chidamber SR, Kemerer CF (1994) A metrics suite for object oriented design. IEEE Trans Softw Eng 20(6):476–493CrossRef

Dam HK, Tran T, Pham T, Ng SW, Grundy J, Ghose A (2021) Automatic feature learning for predicting vulnerable software components. IEEE Trans Softw Eng 47(1):67–85. https://doi.org/10.1109/TSE.2018.2881961CrossRef

D’Ambros M, Lanza M, Robbes R (2010) An extensive comparison of bug prediction approaches. In: 2010 7th IEEE working conference on mining software repositories (MSR 2010). IEEE, pp 31–41

Elkhail AA, Cerny T (2019) On relating code smells to security vulnerabilities. In: 2019 IEEE 5th intl conference on big data security on cloud (BigDataSecurity), IEEE intl conference on high performance and smart computing, (HPSC) and IEEE intl conference on intelligent data and security (IDS). IEEE, pp 7–12

Fu W, Menzies T (2017) Revisiting unsupervised learning for defect prediction. In: Proceedings of the 2017 11th joint meeting on foundations of software engineering. ACM, pp 72–83

Goyal P, Ferrara E (2018) Graph embedding techniques, applications, and performance: a survey. Knowl-Based Syst 151:78–94CrossRef

Grover A, Leskovec J (2016) node2vec: scalable feature learning for networks. In: Acm Sigkdd international conference on knowledge discovery & data mining, pp 855–864

Guo Y, Shepperd M, Li N (2018) Bridging effort-aware prediction and strong classification: a just-in-time software defect prediction study. In: Proceedings of the 40th international conference on software engineering: companion proceedings. ACM, pp 325–326

Hall T, Beecham S, Bowes D, Gray D, Counsell S (2011) A systematic literature review on fault prediction performance in software engineering. IEEE Trans Softw Eng 38(6):1276–1304CrossRef

Harris E (2002) Information gain versus gain ratio: a study of split method biases International symposium on artificial intelligence and mathematics, AI&m 2002, fort lauderdale, florida, USA, January, vol 2-4, p 2002

He Z, Shu F, Yang Y, Li M, Wang Q (2012) An investigation on the feasibility of cross-project defect prediction. Autom Softw Eng 19 (2):167–199CrossRef

Herbold S, Trautsch A, Grabowski J (2017) A comparative study to benchmark cross-project defect prediction approaches. IEEE Trans Softw Eng 44 (9):811–833CrossRef

Herzig K, Just S, Rau A, Zeller A (2013) Predicting defects using change genealogies. In: 2013 IEEE 24th international symposium on software reliability engineering (ISSRE). IEEE, pp 118–127

Hosmer DW Jr, Lemeshow S, Sturdivant RX (2013) Applied logistic regression, vol 398. Wiley, New YorkCrossRef

Huang Q, Xia X, Lo D (2017) Supervised vs unsupervised models: a holistic look at effort-aware just-in-time defect prediction. In: 2017 IEEE international conference on software maintenance and evolution (ICSME). IEEE, pp 159–170

Huang Q, Xia X, Lo D (2019) Revisiting supervised and unsupervised models for effort-aware just-in-time defect prediction. Empir Softw Eng 24 (5):2823–2862CrossRef

Jimenez M, Rwemalika R, Papadakis M, Sarro F, Le Traon Y, Harman M (2019) The importance of accounting for real-world labelling when predicting software vulnerabilities. In: Proceedings of the 2019 27th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering. ACM, pp 695–705

Jureczko M, Madeyski L (2010) Towards identifying software project clusters with regard to defect prediction. In: Proceedings of the 6th international conference on predictive models in software engineering, pp 1–10

Kamei Y, Matsumoto S, Monden A, Matsumoto K, Adams B, Hassan AE (2010) Revisiting common bug prediction findings using effort-aware models. In: 2010 IEEE international conference on software maintenance. IEEE, pp 1–10

Kamei Y, Shihab E, Adams B, Hassan AE, Mockus A, Sinha A, Ubayashi N (2013) A large-scale empirical study of just-in-time quality assurance. IEEE Trans Softw Eng 39(6):757–773CrossRef

Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: 5th international conference on learning representations, ICLR 2017, Toulon, France, April 24–26, 2017, Conference Track Proceedings. https://openreview.net/forum?id=SJU4ayYgl

Kong YX, Shi GY, Wu RJ, Zhang YC (2019) K-core: theories and applications. Phys Rep 832:1–32MathSciNetCrossRef

Lee JR, Gharan SO, Trevisan L (2014) Multiway spectral partitioning and higher-order cheeger inequalities. Journal of the ACM (JACM) 61(6):1–30MathSciNetCrossRef

Lenarduzzi V, Palomba F, Taibi D, Tamburri DA (2020) Openszz: a free, open-source, web-accessible implementation of the szz algorithm. In: Proceedings of the 28th international conference on program comprehension, pp 446–450

Lessmann S, Baesens B, Mues C, Pietsch S (2008) Benchmarking classification models for software defect prediction: a proposed framework and novel findings. IEEE Trans Softw Eng 34(4):485–496CrossRef

Loyola P, Matsuo Y (2017a) Learning feature representations from change dependency graphs for defect prediction. In: 2017 IEEE 28th international symposium on software reliability engineering (ISSRE). IEEE, pp 361–372

Loyola P, Matsuo Y (2017b) Learning graph representations for defect prediction. In: Proceedings of the 39th international conference on software engineering companion. IEEE Press, pp 265–267

Ma W, Chen L, Yang Y, Zhou Y, Xu B (2016) Empirical analysis of network measures for effort-aware fault-proneness prediction. Inf Softw Technol 69:50–70CrossRef

Matthews BW (1975) Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure 405(2):442–451CrossRef

Mende T, Koschke R (2009) Revisiting the evaluation of defect prediction models. In: Proceedings of the 5th international conference on predictor models in software engineering, pp 1–10

Mende T, Koschke R (2010) Effort-aware defect prediction models. In: 2010 14th European conference on software maintenance and reengineering (CSMR). IEEE, pp 107–116

Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. In: 1st international conference on learning representations, ICLR 2013, Scottsdale, Arizona, USA, May 2–4, 2013, Workshop Track Proceedings. 1301.3781

Monden A, Hayashi T, Shinoda S, Shirai K, Yoshida J, Barker M, Matsumoto K (2013) Assessing the cost effectiveness of fault prediction in acceptance testing. IEEE Trans Softw Eng 39(10):1345–1357CrossRef

Nguyen TH, Adams B, Hassan AE (2010) Studying the impact of dependency network measures on software quality. In: 2010 IEEE international conference on software maintenance (ICSM). IEEE, pp 1–10

Osman H, Ghafari M, Nierstrasz O, Lungu M (2017) An extensive analysis of efficient bug prediction configurations. In: Proceedings of the 13th international conference on predictive models and data analytics in software engineering. ACM, pp 107–116

Palomba F, Zanoni M, Fontana FA, De Lucia A, Oliveto R (2016) Smells like teen spirit: improving bug prediction performance using the intensity of code smells. In: 2016 IEEE international conference on software maintenance and evolution (ICSME). IEEE, pp 244–255

Pascarella L, Palomba F, Bacchelli A (2020) On the performance of method-level bug prediction: a negative result. J Syst Softw 161:110493CrossRef

Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 701–710

Perozzi B, Kulkarni V, Chen H, Skiena S (2017) Don’t walk, skip! online learning of multi-scale network embeddings. In: Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, vol 2017, pp 258–265

Pinzger M, Nagappan N, Murphy B (2008) Can developer-module networks predict failures?. In: Proceedings of the 16th ACM SIGSOFT international symposium on foundations of software engineering. ACM, pp 2–12

Premraj R, Herzig K (2011) Network versus code metrics to predict defects: a replication study. In: 2011 international symposium on empirical software engineering and measurement. IEEE, pp 215–224

Qu Y, Liu T, Chi J, Jin Y, Cui D, He A, Zheng Q (2018) node2defect: using network embedding to improve software defect prediction. In: Proceedings of the 33rd ACM/IEEE international conference on automated software engineering, ACM. pp 844–849

Qu Y, Zheng Q, Chi J, Jin Y, He A, Cui D, Zhang H, Liu T (2021) Using k-core decomposition on class dependency networks to improve bug prediction model’s practical performance. IEEE Trans Softw Eng 47 (2):348–366. https://doi.org/10.1109/TSE.2019.2892959CrossRef

Śliwerski J, Zimmermann T, Zeller A (2005) When do changes induce fixes? ACM Sigsoft Software Engineering Notes 30(4):1–5CrossRef

Šubelj L, Bajec M (2011) Community structure of complex software systems: analysis and applications. Physica A: Statistical Mechanics and its Applications 390(16):2968–2975CrossRef

Suykens JA, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Processing Letters 9(3):293–300CrossRef

Tang J, Qu M, Wang M, Zhang M, Yan J, Mei Q (2015) Line: large-scale information network embedding. In: Proceedings of the 24th international conference on world wide web. International World Wide Web Conferences Steering Committee, pp 1067–1077

Tantithamthavorn C, McIntosh S, Hassan AE, Matsumoto K (2019) The impact of automated parameter optimization on defect prediction models. IEEE Trans Softw Eng 45(7):683–711. https://doi.org/10.1109/TSE.2018.2794977CrossRef

Tosun A, Turhan B, Bener A (2009) Validation of network measures as indicators of defective modules in software systems. In: Proceedings of the 5th international conference on predictor models in software engineering, pp 1–9

Tu K, Cui P, Wang X, Yu PS, Zhu W (2018) Deep recursive network embedding with regular equivalence. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 2357–2366

Velickovic P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: 6th international conference on learning representations, ICLR 2018, Vancouver, BC, Canada, April 30–May 3, 2018. Conference Track Proceedings. https://openreview.net/forum?id=rJXMpikCZ

Wang D, Cui P, Zhu W (2016) Structural deep network embedding. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 1225–1234

Wattanakriengkrai S, Thongtanunam P, Tantithamthavorn C, Hata H, Matsumoto K (2020) Predicting defective lines using a model-agnostic technique. IEEE Trans Softw Eng:1–1. https://doi.org/10.1109/TSE.2020.3023177

Witten IH, Frank E, Hall MA, Pal CJ (2016) Data mining: practical machine learning tools and techniques. Morgan Kaufmann, San Mateo

Yang Y, Zhou Y, Lu H, Chen L, Chen Z, Xu B, Leung H, Zhang Z (2015) Are slice-based cohesion metrics actually useful in effort-aware post-release fault-proneness prediction? An empirical study. IEEE Trans Softw Eng 41 (4):331–357CrossRef

Yang Y, Zhou Y, Liu J, Zhao Y, Lu H, Xu L, Xu B, Leung H (2016) Effort-aware just-in-time defect prediction: simple unsupervised models could be better than supervised models. In: Proceedings of the 2016 24th ACM SIGSOFT international symposium on foundations of software engineering. ACM, pp 157–168

Zhang J, Dong Y, Wang Y, Tang J, Ding M (2019) Prone: fast and scalable network representation learning. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, pp 4278–4284. https://doi.org/10.24963/ijcai.2019/594

Zhao Y, Yang Y, Lu H, Liu J, Leung H, Wu Y, Zhou Y, Xu B (2017) Understanding the value of considering client usage context in package cohesion for fault-proneness prediction. Autom Softw Eng 24(2):393–453CrossRef

Zimmermann T, Nagappan N (2008) Predicting defects using network analysis on dependency graphs. In: Proceedings of the 30th international conference on Software engineering, pp 531–540

Titel: Evaluating network embedding techniques’ performances in software bug prediction
verfasst von: Yu Qu
Heng Yin
Publikationsdatum: 01.07.2021
Verlag: Springer US
Erschienen in: Empirical Software Engineering / Ausgabe 4/2021
Print ISSN: 1382-3256
Elektronische ISSN: 1573-7616
DOI: https://doi.org/10.1007/s10664-021-09965-5

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Weitere Artikel der Ausgabe 4/2021

A first look at Android applications in Google Play related to COVID-19

Assessing practitioner beliefs about software engineering

Analysing Time-Stamped Co-Editing Networks in Software Development Teams using git2net

Lessons Learnt on Reproducibility in Machine Learning Based Android Malware Detection

Exposed! A case study on the vulnerability-proneness of Google Play Apps

Genetic programming for feature model synthesis: a replication study

Premium Partner