Top

Annals of Data Science

Published in:

05-11-2021

Deep Learning System and It’s Automatic Testing: An Approach

Author: Rijwan Khan

Published in: Annals of Data Science | Issue 4/2023

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

The process of testing conventional programs is quite easy as compared to the programs using Deep Learning approach. The term Deep learning (DL) is used for a novel programming approach that is highly data centric and where the governing rules and logic are primarily dependent on the data used for training. Conventionally, Deep Learning models are evaluated by using a test dataset to evaluate their performance against set parameters. The difference in data and logic handling between programs using conventional methods and programs using the DL approach makes it difficult to apply the traditional approaches of testing directly to DL based programs. The accuracy of test data is currently the best measure of the adequacy of testing in the DL based systems. This poses a problem because of the difficulty in availability of test data that is of sufficient quality. This in turn restricts the level of confidence that can be established on the adequacy of testing of DL based systems. Unlike conventional applications, using the conventional programming approaches the lack of quality test data and the lack of interpretability makes the system analysis and detection of defects a difficult task in DL based systems. So testing of DL based models can be done automatically with a different approach compared to normal software.

previous article Genetic Algorithms in the Fields of Artificial Intelligence and Data Sciences

next article On Some Estimation Methods for the Inverse Pareto Distribution

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Ma L, Zhang F, Sun J, Xue M, Li B, Juefei-Xu F, Wang Y (2018) Deepmutation: mutation testing of deep learning systems. In: 2018 IEEE 29th international symposium on software reliability engineering (ISSRE), pp 100–111. https://doi.org/10.1109/ISSRE.2018.00021

Dhar V (2013) Data science and prediction. Commun ACM 56(12):64–73. https://doi.org/10.1145/2500499CrossRef

Van Der Aalst W (2016) Data science in action. Process mining. Springer, Berlin, pp 3–23. https://doi.org/10.1007/978-3-662-49851-4_1

Garousi V, Felderer M, Karapıçak ÇM, Yılmaz U (2018) Testing embedded software: a survey of the literature. Inf Softw Technol 104:14–45. https://doi.org/10.1016/j.infsof.2018.06.016CrossRef

Takanen A, Demott JD, Miller C, Kettunen A (2018) Fuzzing for software security testing and quality assurance. Artech House

Asplund F (2019) Exploratory testing: Do contextual factors influence software fault identification? Inf Softw Technol 107:101–111. https://doi.org/10.1016/j.infsof.2018.11.003CrossRef

Monperrus M (2018) Automatic software repair: a bibliography. ACM Comput Surv (CSUR) 51(1):1–24. https://doi.org/10.1145/3105906CrossRef

Felderer M, Russo B, Auer F (2019) On testing data-intensive software systems. In: Security and quality in cyber-physical systems engineering. Springer, Cham, pp 129–148. https://doi.org/10.1007/978-3-030-25312-7_6

Kim Y, Hong S, Ko B, Phan DL, Kim M (2018) Invasive software testing: mutating target programs to diversify test exploration for high test coverage. In: 2018 IEEE 11th international conference on software testing, verification and validation (ICST), pp 239–249. https://doi.org/10.1109/ICST.2018.00032

10.

Budgen D, Tomayko JE (2003) Norm Gibbs and his contribution to software engineering education through the SEI curriculum modules. In: Proceedings 16th conference on software engineering education and training, 2003 (CSEE&T 2003), pp 3–13. https://doi.org/10.1109/CSEE.2003.1191340

11.

Scatalon LP, Fioravanti ML, Prates JM, Garcia RE, Barbosa EF (2018) A survey on graduates’ curriculum-based knowledge gaps in software testing. In: 2018 IEEE frontiers in education conference (FIE), pp 1–8. https://doi.org/10.1109/FIE.2018.8658688

12.

Drave I, Hillemacher S, Greifenberg T, Rumpe B, Wortmann A, Markthaler M, Kriebel S (2018) Model-based testing of software-based system functions. In: 2018 44th Euromicro conference on software engineering and advanced applications (SEAA), pp 146–153. https://doi.org/10.1109/SEAA.2018.00032

13.

Böhme M, Paul S (2015) A probabilistic analysis of the efficiency of automated software testing. IEEE Trans Software Eng 42(4):345–360. https://doi.org/10.1109/TSE.2015.2487274CrossRef

14.

Chen TY, Kuo FC, Liu H, Poon PL, Towey D, Tse TH, Zhou ZQ (2018) Metamorphic testing: a review of challenges and opportunities. ACM Comput Surv (CSUR) 51(1):1–27. https://doi.org/10.1145/3143561CrossRef

15.

Kuchta T, Palikareva H, Cadar C (2018) Shadow symbolic execution for testing software patches. ACM Trans Software Eng Methodol (TOSEM) 27(3):1–32. https://doi.org/10.1145/3208952CrossRef

16.

Alghamdi AM, Eassa FE (2019) Software testing techniques for parallel systems: a survey. Int J Comput Sci Netw Secur 19(4):176–186

17.

Wu H, Petke J, Jia Y, Harman M (2018) An empirical comparison of combinatorial testing, random testing and adaptive random testing. IEEE Trans Software Eng. https://doi.org/10.1109/TSE.2018.2852744CrossRef

18.

Markthaler M, Kriebel S, Salman KS, Greifenberg T, Hillemacher S, Rumpe B, Richenhagen J (2018) Improving model- based testing in automotive software engineering. In: 2018 IEEE/ACM 40th international conference on software engineering: software engineering in practice track (ICSE-SEIP), pp 172–180

19.

Van Deursen A, Aniche M, Boone C, Cunha ML, Nadeem A (2019) Software quality and testing. Delft University of Technology

20.

Lemieux C, Sen K (2018) Fairfuzz: a targeted mutation strategy for increasing greybox fuzz testing coverage. In: Proceedings of the 33rd ACM/IEEE international conference on automated software engineering, pp 475–485. https://doi.org/10.1145/3238147.3238176

21.

Harman M, Jia Y, Zhang Y (2015) Achievements, open problems and challenges for search based software testing. In: 2015 IEEE 8th international conference on software testing, verification and validation (ICST), pp 1–12. https://doi.org/10.1109/ICST.2015.7102580

22.

Majumdar R (2010) Paul Ammann and Jeff Offutt Introduction to Software Testing. Cambridge University Press (2008). ISBN: 978-0-521-88038-1.£ 32.99. Hardcover. Comput J 53(5):615–615. https://doi.org/10.1093/comjnl/bxp017

23.

Maciel CP, Souza EF, Vijaykumar NL, Falbo RA, Meinerz GV, Felizardo KR (2018) An empirical study on the knowledge management practice in software testing. In: Experimental Software Engineering Latin American Workshop (ESELAW’18). XXI Ibero-American Conference on Software Engineering (CIBSE)

24.

Xie T, Tillmann N, Lakshman P (2016) Advances in unit testing: theory and practice. In: Proceedings of the 38th international conference on software engineering companion, pp 904–905. https://doi.org/10.1145/2889160.2891056

25.

Clegg BS, Rojas JM, Fraser G (2017) Teaching software testing concepts using a mutation testing game. In: 2017 IEEE/ACM 39th international conference on software engineering: software engineering education and training track (ICSE-SEET), pp 33–36. https://doi.org/10.1109/ICSE-SEET.2017.1

26.

Angell R, Johnson B, Brun Y, Meliou A (2018) Themis: automatically testing software for discrimination. In: Proceedings of the 2018 26th ACM Joint Meeting on European software engineering conference and symposium on the foundations of software engineering, pp 871–875. https://doi.org/10.1145/3236024.3264590

27.

Xiu M, Ming Z, Adams B (2019) An exploratory study on machine learning model stores. https://doi.org/10.1109/MS.2020.2975159

28.

Kamilaris A, Prenafeta-Boldú FX (2018) Deep learning in agriculture: a survey. Comput Electron Agric 147:70–90. https://doi.org/10.1016/j.compag.2018.02.016CrossRef

29.

Ferentinos KP (2018) Deep learning models for plant disease detection and diagnosis. Comput Electron Agric 145:311–318. https://doi.org/10.1016/j.compag.2018.01.009CrossRef

30.

Lusch B, Kutz JN, Brunton SL (2018) Deep learning for universal linear embeddings of nonlinear dynamics. Nat Commun 9(1):1–10. https://doi.org/10.1038/s41467-018-07210-0CrossRef

31.

Singh AK, Ganapathy subramanian B, Sarkar S, Singh A (2018) Deep learning for plant stress phenotyping: trends and future perspectives. Trends Plant Sci 23(10):883–898. https://doi.org/10.1016/j.tplants.2018.07.004CrossRef

32.

Raissi M, Wang Z, Triantafyllou MS, Karniadakis GE (2019) Deep learning of vortex-induced vibrations. J Fluid Mech 861:119–137. https://doi.org/10.1017/jfm.2018.872CrossRef

33.

Kulin M, Kazaz T, Moerman I, De Poorter E (2018) End-to-end learning from spectrum data: a deep learning approach for wireless signal identification in spectrum monitoring applications. IEEE Access 6:18484–18501. https://doi.org/10.1109/ACCESS.2018.2818794CrossRef

34.

Shallue CJ, Vanderburg A (2018) Identifying exoplanets with deep learning: a five-planet resonant chain around kepler-80 and an eighth planet around kepler-90. Astron J 155(2):94CrossRef

35.

Finlayson SG, Chung HW, Kohane IS, Beam AL (2018) Adversarial attacks against medical deep learning systems. arXiv preprint arXiv:1804.05296

36.

Stadie BC, Yang G, Houthooft R, Chen X, Duan Y, Wu Y, Sutskever I (2018) Some considerations on learning to explore via meta-reinforcement learning. arXiv preprint arXiv:1803.01118.

37.

Raissi M, Perdikaris P, Karniadakis GE (2018) Multistep neural networks for data-driven discovery of nonlinear dynamical systems. arXiv preprint arXiv:1801.01236.

38.

Gurovich Y, Hanani Y, Bar O, Nadav G, Fleischer N, Gelbman D, Bird LM (2019) Identifying facial phenotypes of genetic disorders using deep learning. Nat Med 25(1):60–64. https://doi.org/10.1038/s41591-018-0279-0CrossRef

39.

Ahmed M, Najmul Islam AKM (2020) Deep learning: hope or hype. Ann Data Sci 7:427–432. https://doi.org/10.1007/s40745-019-00237-0CrossRef

40.

Hassani H, Huang X, Ghodsi SE, M (2020) Deep learning and implementations in banking. Ann Data Sci 7:433–446. https://doi.org/10.1007/s40745-020-00300-1CrossRef

41.

Olson DL, Shi Y (2007) Introduction to business data mining. McGraw-Hill/Irwin, New York

42.

Shi Y, Tian YJ, Kou G, Peng Y, Li JP (2011) Optimization based data mining: theory and applications. Springer, BerlinCrossRef

43.

Tien JM (2017) Internet of things, real-time decision making, and artificial intelligence. Ann Data Sci 4(2):149–178. https://doi.org/10.1007/s40745-017-0112-5CrossRef

44.

Grano G, Titov TV, Panichella, S., & Gall, H. C. (2018, March). How high will it be? using machine learning models to predict branch coverage in automated testing. In 2018 IEEE workshop on machine learning techniques for software quality evaluation (MaLTeSQuE), pp 19–24. https://doi.org/10.1109/MALTESQUE.2018.8368454

45.

King TM, Arbon J, Santiago D, Adamo D, Chin W, Shanmugam R (2019) AI for testing today and tomorrow: industry perspectives. In: 2019 IEEE international conference on Artificial Intelligence Testing (AITest), pp 81–88. https://doi.org/10.1109/AITest.2019.000-3

46.

Marcus G (2018) Deep learning: a critical appraisal. arXiv preprint arXiv:1801.00631

47.

Ma L, Juefei-Xu F, Zhang F, Sun J, Xue M, Li B, Zhao J (2018) Deepgauge: multi-granularity testing criteria for deep learning systems. In: Proceedings of the 33rd ACM/IEEE international conference on automated software engineering, pp. 120–131. https://doi.org/10.1145/3238147.3238202

48.

Ma L, Zhang F, Xue M, Li B, Liu Y, Zhao J, Wang Y (2018) Combinatorial testing for deep learning systems. arXiv preprint arXiv:1806.07723

49.

Dwarakanath A, Ahuja M, Sikand S, Rao RM, Bose RJC, Dubash N, Podder S (2018) Identifying implementation bugs in machine learning based image classifiers using metamorphic testing. In: Proceedings of the 27th ACM SIGSOFT international symposium on software testing and analysis, pp 118–128. https://doi.org/10.1145/3213846.3213858

50.

Kim J, Feldt R, Yoo S (2019) Guiding deep learning system testing using surprise adequacy. In: 2019 IEEE/ACM 41st international conference on software engineering (ICSE), pp 1039–1049. https://doi.org/10.1109/ICSE.2019.00108

51.

Zhang JM, Harman M, Ma L, Liu Y (2019) Machine learning testing: survey, landscapes and horizons. https://doi.org/10.1109/TSE.2019.2962027

52.

Srisakaokul S, Wu Z, Astorga A, Alebiosu O, Xie T (2018) Multiple-implementation testing of supervised learning software. In: Workshops at the thirty-second AAAI conference on artificial intelligence

53.

Sun Y, Huang X, Kroening D (2018) Testing deep neural networks. arXiv preprint arXiv:1803.04792

54.

Al-Hajjaji M, Thüm T, Lochau M, Meinicke J, Saake G (2019) Effective product-line testing using similarity-based product prioritization. Softw Syst Model 18(1):499–521. https://doi.org/10.1007/s10270-016-0569-2CrossRef

Title: Deep Learning System and It’s Automatic Testing: An Approach
Author: Rijwan Khan
Publication date: 05-11-2021
Publisher: Springer Berlin Heidelberg
Published in: Annals of Data Science / Issue 4/2023
Print ISSN: 2198-5804
Electronic ISSN: 2198-5812
DOI: https://doi.org/10.1007/s40745-021-00361-w

Wind Speed Prediction of Central Region of Chhattisgarh (India) Using Artificial Neural Network and Multiple Linear Regression Technique: A Comparative Study

Springer Professional

Deep Learning System and It’s Automatic Testing: An Approach

Abstract

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 4/2023

Genetic Algorithms in the Fields of Artificial Intelligence and Data Sciences

A Machine Learning Approach and Methodology for Solar Radiation Assessment Using Multispectral Satellite Images

A Generalized-Alpha–Beta-Skew Normal Distribution with Applications

Wind Speed Prediction of Central Region of Chhattisgarh (India) Using Artificial Neural Network and Multiple Linear Regression Technique: A Comparative Study

The Discrete Analogue of the Weibull G Family: Properties, Different Applications, Bayesian and Non-Bayesian Estimation Methods

A Class of Bivariate Modified Weighted Distributions: Properties and Applications

Premium Partner

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Genetic Algorithms in the Fields of Artificial Intelligence and Data Sciences

A Machine Learning Approach and Methodology for Solar Radiation Assessment Using Multispectral Satellite Images

A Generalized-Alpha–Beta-Skew Normal Distribution with Applications

Wind Speed Prediction of Central Region of Chhattisgarh (India) Using Artificial Neural Network and Multiple Linear Regression Technique: A Comparative Study

The Discrete Analogue of the Weibull G Family: Properties, Different Applications, Bayesian and Non-Bayesian Estimation Methods

A Class of Bivariate Modified Weighted Distributions: Properties and Applications