Top

Published in:

2021 | OriginalPaper | Chapter

Software Quality for AI: Where We Are Now?

Authors : Valentina Lenarduzzi, Francesco Lomio, Sergio Moreschini, Davide Taibi, Damian Andrew Tamburri

Published in: Software Quality: Future Perspectives on Software Engineering Quality

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Artificial Intelligence is getting more and more popular, being adopted in a large number of applications and technology we use on a daily basis. However, a large number of Artificial Intelligence applications are produced by developers without proper training on software quality practices or processes, and in general, lack in-depth knowledge regarding software engineering processes. The main reason is due to the fact that the machine-learning engineer profession has been born very recently, and currently there is a very limited number of training or guidelines on issues (such as code quality or testing) for machine learning and applications using machine learning code. In this work, we aim at highlighting the main software quality issues of Artificial Intelligence systems, with a central focus on machine learning code, based on the experience of our four research groups. Moreover, we aim at defining a shared research road map, that we would like to discuss and to follow in collaboration with the workshop participants. As a result, the software quality of AI-enabled systems is often poorly tested and of very low quality.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Quality Assurance for AI-Based Systems: Overview and Challenges (Introduction to Interactive Session)

next chapter Hidden Feedback Loops in Machine Learning Systems: A Simulation Model and Preliminary Results

Informatics Europe https://www.informatics-europe.org.

ACM Europe Council https://europe.acm.org.

The Networked European Software and Services Initiative - NESSI http://www.nessi-europe.com.

https://twitter.com/vale_lenarduzzi/status/1295055334264975360. Last access: 28 August 2020.

TensorFlow version compatibility. https://www.tensorflow.org/guide/versions

Compatible Versions of PyTorch/Libtorch with Cuda 10.0 (2019). https://discuss.pytorch.org/t/compatible-versions-of-pytorch-libtorch-with-cuda-10-0/58506. Accessed 11 July 2020

Machine Learning Glossary, Google Developers (2019). https://developers.google.com/machine-learning/glossary. Accessed 28 Aug 2020

Pytorch Lightning. The lightweight PyTorch wrapper for ML researchers (2019). https://github.com/PyTorchLightning/pytorch-lightning. Accessed 11 July 2020

Tensorflow 1.11.0 incompatible with keras2.2.2? (2019). https://github.com/tensorflow/tensorflow/issues/22601. Accessed 11 July 2020

Avgeriou, P., Kruchten, P., Ozkaya, I., Seaman, C.: Managing technical debt in software engineering (Dagstuhl seminar 16162). Dagstuhl Reports 6 (2016)

Avgeriou, P., et al.: An overview and comparison of technical debt measurement tools. IEEE Softw. (2021)

Bradley, A.P.: The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recogn. 30(7), 1145–1159 (1997)CrossRef

Britten, N., Campbell, R., Pope, C., Donovan, J., Morgan, M., Pill, R.: Using meta ethnography to synthesise qualitative research: a worked example. J. Health Serv. Res. Policy 7(4), 209–215 (2002). http://www.ncbi.nlm.nih.gov/pubmed/12425780CrossRef

10.

Chen, T.Y.: Metamorphic testing: a simple method for alleviating the test oracle problem. In: Proceedings of the 10th International Workshop on Automation of Software Test, AST 2015, pp. 53–54. IEEE Press (2015)

11.

Cohen, G., Afshar, S., Tapson, J., Van Schaik, A.: EMNIST: extending MNIST to handwritten letters. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 2921–2926. IEEE (2017)

12.

Commission, E.: WHITE PAPER On Artificial Intelligence - A European approach to excellence and trust (2020). https://ec.europa.eu/info/sites/info/files/commission-white-paper-artificial-intelligence-feb2020_en.pdf?utm_source=CleverReach&utm_medium=email&utm_campaign=23-02-2020+Instituts-Journal+07%2F20%3A+Wo+waren+Sie%3F+Es+ging+um+Sie%21&utm_content=Mailing_11823061. Accessed 09 July 2020

13.

Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

14.

Kästner, C., Kang, E.: Teaching software engineering for AI-enabled systems. arXiv preprint arXiv:2001.06691 (2020)

15.

Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the 14th International Joint Conference on Artificial Intelligence, IJCAI 1995, vol. 2, pp. 1137–1143. Morgan Kaufmann Publishers Inc., San Francisco (1995)

16.

Larus, J., et al.: When computers decide: European recommendations on machine-learned automated decision making (2018)

17.

LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRef

18.

Lenarduzzi, V., Sillitti, A., Taibi, D.: A survey on code analysis tools for software maintenance prediction. In: Ciancarini, P., Mazzara, M., Messina, A., Sillitti, A., Succi, G. (eds.) SEDA 2018. AISC, vol. 925, pp. 165–175. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-14687-0_15CrossRef

19.

Lwakatare, L.E., Raj, A., Bosch, J., Olsson, H.H., Crnkovic, I.: A taxonomy of software engineering challenges for machine learning systems: an empirical investigation. In: Kruchten, P., Fraser, S., Coallier, F. (eds.) XP 2019. LNBIP, vol. 355, pp. 227–243. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-19034-7_14CrossRef

20.

Dhaval, M.: How to perform Quality Assurance for Machine Learning models? (2018). https://medium.com/datadriveninvestor/how-to-perform-quality-assurance-for-ml-models-cef77bbbcfb. Accessed 09 July 2020

21.

Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (2010). [u.a.]. http://www.amazon.com/Machine-Learning-Tom-M-Mitchell/dp/0070428077MATH

22.

Murphy, C., Kaiser, G.E., Arias, M.: A framework for quality assurance of machine learning applications. Columbia University Computer Science Technical reports, CUCS-034-06 (2006)

23.

NESSI: Software and Artificial Intelligence (2019). http://www.nessi-europe.com/files/NESSI%20-%20Software%20and%20AI%20-%20issue%201.pdf. Accessed 09 July 2020

24.

Radhakrishnan, V.: How to perform Quality Assurance for Machine Learning models? (2019). https://blog.sasken.com/quality-assurance-for-machine-learning-models-part-1-why-quality-assurance-is-critical-for-machine-learning-models. Accessed 09 July 2020

25.

van Rossum, G., Warsaw, B., Coghlan, N.: PEP 8 - Style Guide for Python Code. https://www.python.org/dev/peps/pep-0008/

26.

Rushby, J.: Quality measures and assurance for AI (artificial intelligence) software. Technical report (1988)

27.

Russell, S.J., Norvig, P.: Artificial Intelligence - A Modern Approach: The Intelligent Agent Book. Prentice Hall Series in Artificial Intelligence. Prentice Hall, Upper Saddle River (1995)MATH

28.

Sculley, D., et al.: Hidden technical debt in machine learning systems. In: Advances in Neural Information Processing Systems, pp. 2503–2511 (2015)

29.

Wang, J., Li, L., Zeller, A.: Better code, better sharing: on the need of analyzing Jupyter notebooks (2019)

30.

Zhang, J.M., Harman, M., Ma, L., Liu, Y.: Machine learning testing: survey, landscapes and horizons. IEEE Trans. Softw. Eng. PP, 1 (2020)

31.

Ören, T.I.: Quality assurance paradigms for artificial intelligence in modelling and simulation. Simulation 48(4), 149–151 (1987)CrossRef

Title: Software Quality for AI: Where We Are Now?
Authors: Valentina Lenarduzzi
Francesco Lomio
Sergio Moreschini
Davide Taibi
Damian Andrew Tamburri
Publisher: Springer International Publishing
Book: Software Quality: Future Perspectives on Software Engineering Quality
Print ISBN: 978-3-030-65853-3

Electronic ISBN: 978-3-030-65854-0

Copyright Year: 2021
DOI: https://doi.org/10.1007/978-3-030-65854-0_4

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner