Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Using Assurance Cases to Prevent Malicious Behaviour from Targeting Safety Vulnerabilities Read chapter Read first chapter

2023 | OriginalPaper | Chapter

AIMOS: Metamorphic Testing of AI - An Industrial Application

Authors : Augustin Lemesle, Aymeric Varasse, Zakaria Chihani, Dominique Tachet

Published in: Computer Safety, Reliability, and Security. SAFECOMP 2023 Workshops

Publisher: Springer Nature Switzerland

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In this paper, we present the AIMOS tool as well as the results of its application to industrial use cases. Relying on the widely used Metamorphic testing paradigm, we show how the process of verification and validation can benefit from the early testing of models’ robustness to perturbations stemming from the intended operational domain.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Conformal Prediction and Uncertainty Wrapper: What Statistical Guarantees Can You Get for Uncertainty Quantification in Machine Learning?
next chapter AERoS: Assurance of Emergent Behaviour in Autonomous Robotic Swarms
Literature
1.
2.
Chen, T.Y.: Metamorphic testing: a simple method for alleviating the test oracle problem. In: Proceedings of the 10th International Workshop on Automation of Software Test, AST 2015, pp. 53–54. IEEE Press (2015)
3.
Chen, T.Y., et al.: Metamorphic testing: a review of challenges and opportunities. ACM Comput. Surv. (CSUR) 51(1), 1–27 (2018)CrossRef
4.
Ding, J., Hu, X.H., Gudivada, V.: A machine learning based framework for verification and validation of massive scale image data. IEEE Trans. Big Data 7(2), 451–467 (2017)CrossRef
5.
Dwarakanath, A., et al.: Identifying implementation bugs in machine learning based image classifiers using metamorphic testing. In: Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis. ACM (2018). https://​doi.​org/​10.​1145/​3213846.​3213858
6.
7.
He, P., Meister, C., Su, Z.: Structure-invariant testing for machine translation (2020)
8.
9.
10.
Pullum, L.L., Ozmen, O.: Early results from metamorphic testing of epidemiological models. In: 2012 ASE/IEEE International Conference on BioMedical Computing (BioMedCom), pp. 62–67. IEEE (2012)
11.
Segura, S., Fraser, G., Sanchez, A.B., Ruiz-Cortés, A.: A survey on metamorphic testing. IEEE Trans. Softw. Eng. 42(9), 805–824 (2016)CrossRef
12.
Serge, D., Augustin, L., Zakaria, C., Caterina, U., François, T.: ReCIPH: relational coefficients for input partitioning heuristic. To be Presented at ICML’s Workshop on Formal Verification of Machine Learning (WFVML 2022) (2022)
13.
Sun, Z., Zhang, J.M., Harman, M., Papadakis, M., Zhang, L.: Automatic testing and improvement of machine translation (2019)
14.
Tian, Y., Pei, K., Jana, S., Ray, B.: DeepTest: automated testing of deep-neural-network-driven autonomous cars (2018)
15.
Tse, T., Yau, S.S.: Testing context-sensitive middleware-based software applications. In: Proceedings of the 28th Annual International Computer Software and Applications Conference, COMPSAC 2004, pp. 458–466. IEEE (2004)
16.
Zhang, M., Zhang, Y., Zhang, L., Liu, C., Khurshid, S.: DeepRoad: GAN-based metamorphic testing and input validation framework for autonomous driving systems. In: 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 132–142 (2018). https://​doi.​org/​10.​1145/​3238147.​3238187
17.
Zhou, Z.Q., Sun, L.: Metamorphic testing of driverless cars. Commun. ACM 62, 61–67 (2019)CrossRef
Metadata
Title
AIMOS: Metamorphic Testing of AI - An Industrial Application
Authors
Augustin Lemesle
Aymeric Varasse
Zakaria Chihani
Dominique Tachet
Copyright Year
2023
Book
Computer Safety, Reliability, and Security. SAFECOMP 2023 Workshops

Print ISBN: 978-3-031-40952-3

Electronic ISBN: 978-3-031-40953-0

Copyright Year: 2023

DOI
https://doi.org/10.1007/978-3-031-40953-0_27

Premium Partner

    Image Credits