Skip to main content
Erschienen in: Innovations in Systems and Software Engineering 4/2015

01.12.2015 | SI: QUATIC 2010

Bridging the gap between test cases and requirements by abstract testing

verfasst von: Florian Merz, Carsten Sinz, Hendrik Post, Thomas Gorges, Thomas Kropf

Erschienen in: Innovations in Systems and Software Engineering | Ausgabe 4/2015

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this article we propose a technique, called abstract testing, which replaces traditional test cases by abstract test cases. By doing so, fewer test cases are needed, and they are linked more closely to the requirements. Abstract tests can be considered as verification scenarios on the source code level which are derived from the requirements. Checking verification scenarios against the source code is done automatically using a software model checker. We also suggest a migration path from traditional tests to abstract test cases, which provides a smooth transition towards this new technique. Finally, we demonstrate feasibility of abstract testing by a case study from the automotive systems domain.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Fußnoten
1
We mainly have black-box unit testing of functional properties in mind when talking about a test case.
 
2
Inlining is done by software bounded model checking tools automatically as part of the preprocessing of the code.
 
3
MISRA-C is a software development standard developed by the The Motor Industry Software Reliability Association (MISRA) containing rules and guidelines for the development of safe, portable and reliably source code for embedded systems in the automotive industry [28]
 
4
The havoc() procedure call in the example sets some global variables to undefined (non-deterministic) values.
 
5
Due to a non-disclosure agreement we cannot publish the precise number of test cases created during software testing.
 
6
Available at http://​junit.​org. Accessed 24/09/2014).
 
Literatur
1.
Zurück zum Zitat Arnold D (2009) An open framework for the specification and execution of a testable requirements model. Ph.D. thesis, Ottawa-Carleton Institute for Computer Science, Carleton University Arnold D (2009) An open framework for the specification and execution of a testable requirements model. Ph.D. thesis, Ottawa-Carleton Institute for Computer Science, Carleton University
2.
Zurück zum Zitat Arnold D, Corriveau JP, Shi W (2010) Modeling and validating requirements using executable contracts and scenarios. In: Proceedings of the 8th international conference on software engineering research, management and applications (SERA) Arnold D, Corriveau JP, Shi W (2010) Modeling and validating requirements using executable contracts and scenarios. In: Proceedings of the 8th international conference on software engineering research, management and applications (SERA)
3.
Zurück zum Zitat Biere A, Cimatti A, Clarke EM, Zhu Y (1999) Symbolic model checking without BDDs. In: Proceedings of the 5th international conference on tools and algorithms for construction and analysis of systems (TACAS). Springer, London, pp 193–207 Biere A, Cimatti A, Clarke EM, Zhu Y (1999) Symbolic model checking without BDDs. In: Proceedings of the 5th international conference on tools and algorithms for construction and analysis of systems (TACAS). Springer, London, pp 193–207
4.
Zurück zum Zitat Bringmann E, Krämer A (2008) Model-based testing of automotive systems. In: ICST, pp 485–493 Bringmann E, Krämer A (2008) Model-based testing of automotive systems. In: ICST, pp 485–493
5.
Zurück zum Zitat Burnstein I (2003) Practical software testing: a process-oriented approach. Springer, Berlin Burnstein I (2003) Practical software testing: a process-oriented approach. Springer, Berlin
6.
Zurück zum Zitat Chaki S, Clarke E, Groce A, Jha S, Veith H (2003) Modular verification of software components in C. In: Proceedings of the 25th international conference on software engineering (ICSE). IEEE Computer Society, pp 385–395 Chaki S, Clarke E, Groce A, Jha S, Veith H (2003) Modular verification of software components in C. In: Proceedings of the 25th international conference on software engineering (ICSE). IEEE Computer Society, pp 385–395
7.
Zurück zum Zitat Chechik M, Gannon JD (2001) Automatic analysis of consistency between requirements and designs. IEEE Trans Softw Eng 27(7):651–672CrossRef Chechik M, Gannon JD (2001) Automatic analysis of consistency between requirements and designs. IEEE Trans Softw Eng 27(7):651–672CrossRef
8.
Zurück zum Zitat Clarke E, Kroening D, Lerda F (2004) A tool for checking ANSI-C programs. In: Jensen K, Podelski A (eds) Tools and algorithms for the construction and analysis of systems (TACAS). LNCS, vol 2988. Springer, Berlin, pp 168–176 Clarke E, Kroening D, Lerda F (2004) A tool for checking ANSI-C programs. In: Jensen K, Podelski A (eds) Tools and algorithms for the construction and analysis of systems (TACAS). LNCS, vol 2988. Springer, Berlin, pp 168–176
9.
Zurück zum Zitat Cousot P (2007) Proving the absence of run-time errors in safety-critical avionics code. In: Proceedings of the 7th ACM and IEEE international conference on embedded software (EMSOFT). ACM, New York, pp 7–9 Cousot P (2007) Proving the absence of run-time errors in safety-critical avionics code. In: Proceedings of the 7th ACM and IEEE international conference on embedded software (EMSOFT). ACM, New York, pp 7–9
10.
Zurück zum Zitat Cousot P, Cousot R (1977) Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: Proceedings of the fourth ACM symposium on principles of programming languages (POPL), Los Angeles, California, January 1977, pp 238–252 Cousot P, Cousot R (1977) Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: Proceedings of the fourth ACM symposium on principles of programming languages (POPL), Los Angeles, California, January 1977, pp 238–252
11.
Zurück zum Zitat Cousot P, Cousot R, Feret J, Mauborgne L, Miné A, Monniaux D, Rival X (2005) The ASTREÉ analyzer. In: Proceedings of the programming languages and systems, 14th European symposium on programming (ESOP), Edinburgh, UK, April 4–8, pp 21–30 Cousot P, Cousot R, Feret J, Mauborgne L, Miné A, Monniaux D, Rival X (2005) The ASTREÉ analyzer. In: Proceedings of the programming languages and systems, 14th European symposium on programming (ESOP), Edinburgh, UK, April 4–8, pp 21–30
12.
Zurück zum Zitat Crow J, Di Vito B (1998) Formalizing space shuttle software requirements: four case studies. ACM Trans Softw Eng Methodol 7(3):296–332CrossRef Crow J, Di Vito B (1998) Formalizing space shuttle software requirements: four case studies. ACM Trans Softw Eng Methodol 7(3):296–332CrossRef
14.
Zurück zum Zitat Dutertre B, Stavridou V (1997) Formal requirements analysis of an avionics control system. IEEE Trans Softw Eng 23:267–278CrossRef Dutertre B, Stavridou V (1997) Formal requirements analysis of an avionics control system. IEEE Trans Softw Eng 23:267–278CrossRef
15.
Zurück zum Zitat Fraser G, Wotawa F (2006) Property relevant software testing with model-checkers. SIGSOFT Softw Eng Notes 31(6):1–10CrossRef Fraser G, Wotawa F (2006) Property relevant software testing with model-checkers. SIGSOFT Softw Eng Notes 31(6):1–10CrossRef
16.
Zurück zum Zitat Graham D (2002) Requirements and testing: seven missing-link myths. Softw IEEE 19(5):15–17CrossRef Graham D (2002) Requirements and testing: seven missing-link myths. Softw IEEE 19(5):15–17CrossRef
17.
Zurück zum Zitat Heitmeyer CL, Archer M, Leonard EI, McLean J (2008) Applying formal methods to a certifiably secure software system. IEEE Trans Softw Eng 34(1):82–98CrossRef Heitmeyer CL, Archer M, Leonard EI, McLean J (2008) Applying formal methods to a certifiably secure software system. IEEE Trans Softw Eng 34(1):82–98CrossRef
18.
Zurück zum Zitat Heitmeyer CL, Jeffords RD, Labaw BG (1996) Automated consistency checking of requirements specifications. ACM Trans Softw Eng Methodol 5(3):231–261CrossRef Heitmeyer CL, Jeffords RD, Labaw BG (1996) Automated consistency checking of requirements specifications. ACM Trans Softw Eng Methodol 5(3):231–261CrossRef
19.
Zurück zum Zitat Henzinger T, Jhala R, Majumdar R, Sutre G (2003) Software verification with BLAST. In: Proceedings of the 10th international SPIN workshop. LNCS, vol 2648. Springer, Berlin, pp 235–239 Henzinger T, Jhala R, Majumdar R, Sutre G (2003) Software verification with BLAST. In: Proceedings of the 10th international SPIN workshop. LNCS, vol 2648. Springer, Berlin, pp 235–239
20.
Zurück zum Zitat Henzinger TA, Jhala R, Majumdar R, Sutre G (2003) Software verification with blast. In: Proceedings of the 10th international workshop on model checking of software (SPIN). LNCS, vol 2648. Springer, Berlin, pp 235–239 Henzinger TA, Jhala R, Majumdar R, Sutre G (2003) Software verification with blast. In: Proceedings of the 10th international workshop on model checking of software (SPIN). LNCS, vol 2648. Springer, Berlin, pp 235–239
21.
Zurück zum Zitat Hunt JJ, Jenn E, Leriche S, Schmitt P, Tonin I, Wonnemann C (2006) A case study of specification and verification using JML in an avionics application. In: Proceedings of the 4th workshop on java technologies for real-time and embedded systems (JTRES). ACM Press, New York, pp 107–116 Hunt JJ, Jenn E, Leriche S, Schmitt P, Tonin I, Wonnemann C (2006) A case study of specification and verification using JML in an avionics application. In: Proceedings of the 4th workshop on java technologies for real-time and embedded systems (JTRES). ACM Press, New York, pp 107–116
22.
Zurück zum Zitat Jones C, Bonsignour O (2011) The economics of software quality. Prentice Hall, New Jersey (2011) Jones C, Bonsignour O (2011) The economics of software quality. Prentice Hall, New Jersey (2011)
23.
Zurück zum Zitat Kim M, Kim Y, Kim H (2008) Unit testing of flash memory device driver through a SAT-based model checker. In: Proceedings of the international conference on automated software engineering (ASE). IEEE Computer Society Press, Washington, DC, pp 198–207 Kim M, Kim Y, Kim H (2008) Unit testing of flash memory device driver through a SAT-based model checker. In: Proceedings of the international conference on automated software engineering (ASE). IEEE Computer Society Press, Washington, DC, pp 198–207
24.
Zurück zum Zitat Merz F, Falke S, Sinz C (2012) LLBMC: bounded model checking of C and C++ programs using a compiler IR. In: Proceedings of the 4th international conference on verified software: theories, tools, experiments (VSTTE’12). Springer, Berlin, pp 146–161 Merz F, Falke S, Sinz C (2012) LLBMC: bounded model checking of C and C++ programs using a compiler IR. In: Proceedings of the 4th international conference on verified software: theories, tools, experiments (VSTTE’12). Springer, Berlin, pp 146–161
25.
Zurück zum Zitat Merz F, Sinz C, Post H, Gorges T, Kropf T (2010) Abstract testing: connecting source code verification with requirements. In: Proceedings of the 7th international conference on the quality of information and communications technology, pp 89–96 Merz F, Sinz C, Post H, Gorges T, Kropf T (2010) Abstract testing: connecting source code verification with requirements. In: Proceedings of the 7th international conference on the quality of information and communications technology, pp 89–96
26.
Zurück zum Zitat Meyer B (2005) Eiffel as a framework for verification. In: First IFIP TC 2/WG 2.3 conference on verified software: theories, tools, experiments (VSSTE’05), pp 301–307 Meyer B (2005) Eiffel as a framework for verification. In: First IFIP TC 2/WG 2.3 conference on verified software: theories, tools, experiments (VSSTE’05), pp 301–307
27.
Zurück zum Zitat Miller SP, Tribble AC, Whalen MW, Heimdahl MPE (2006) Proving the shalls: early validation of requirements through formal methods. J Softw Tools Technol Transf (STTT) 8(4–5):303–319CrossRef Miller SP, Tribble AC, Whalen MW, Heimdahl MPE (2006) Proving the shalls: early validation of requirements through formal methods. J Softw Tools Technol Transf (STTT) 8(4–5):303–319CrossRef
28.
Zurück zum Zitat MISRA C Working Group (2004) MISRA-C:2004—guidelines for the use of the C language in critical systems. The Motor Industry Software Reliability Association MISRA C Working Group (2004) MISRA-C:2004—guidelines for the use of the C language in critical systems. The Motor Industry Software Reliability Association
30.
Zurück zum Zitat Post H, Küchlin W (2007) Integration of static analysis for linux device driver verification. In: Davies J, Gibbons J (eds) Proceedings of the 6th international conference integrated formal methods (IFM). LNCS, vol 4591. Springer, Berlin, pp 518–537 Post H, Küchlin W (2007) Integration of static analysis for linux device driver verification. In: Davies J, Gibbons J (eds) Proceedings of the 6th international conference integrated formal methods (IFM). LNCS, vol 4591. Springer, Berlin, pp 518–537
31.
Zurück zum Zitat RTI-Health, Social, and Economics Research (2002) The economic impacts of inadequate infrastructure for software testing. Planning Report 02-3. National Institute of Standards and Technology RTI-Health, Social, and Economics Research (2002) The economic impacts of inadequate infrastructure for software testing. Planning Report 02-3. National Institute of Standards and Technology
33.
Zurück zum Zitat Staats M, Heimdahl MPE (2008) Partial translation verification for untrusted code-generators. In: 10th international conference on formal engineering methodsformal (ICFEM’08), Kitakyushu-City, Japan, pp 226–237 Staats M, Heimdahl MPE (2008) Partial translation verification for untrusted code-generators. In: 10th international conference on formal engineering methodsformal (ICFEM’08), Kitakyushu-City, Japan, pp 226–237
34.
Zurück zum Zitat Tillmann N, Schulte W (2005) Parameterized unit tests. SIGSOFT Softw Eng Notes 30:253–262CrossRef Tillmann N, Schulte W (2005) Parameterized unit tests. SIGSOFT Softw Eng Notes 30:253–262CrossRef
35.
Zurück zum Zitat Uusitalo EJ, Komssi M, Kauppinen M, Davis AM (2008) Linking requirements and testing in practice. In: Proceedings of the 2008 16th IEEE international requirements engineering conference RE’08. IEEE Computer Society, Washington, DC, pp 265–270 Uusitalo EJ, Komssi M, Kauppinen M, Davis AM (2008) Linking requirements and testing in practice. In: Proceedings of the 2008 16th IEEE international requirements engineering conference RE’08. IEEE Computer Society, Washington, DC, pp 265–270
36.
Zurück zum Zitat Visser W, Havelund K, Brat GP, Park S, Lerda F (2003) Model checking programs. Autom Softw Eng 10(2):203–232CrossRef Visser W, Havelund K, Brat GP, Park S, Lerda F (2003) Model checking programs. Autom Softw Eng 10(2):203–232CrossRef
37.
Zurück zum Zitat Wassyng A, Lawford M (2006) Software tools for safety-critical software development. STTT 8(4–5):337–354CrossRef Wassyng A, Lawford M (2006) Software tools for safety-critical software development. STTT 8(4–5):337–354CrossRef
Metadaten
Titel
Bridging the gap between test cases and requirements by abstract testing
verfasst von
Florian Merz
Carsten Sinz
Hendrik Post
Thomas Gorges
Thomas Kropf
Publikationsdatum
01.12.2015
Verlag
Springer London
Erschienen in
Innovations in Systems and Software Engineering / Ausgabe 4/2015
Print ISSN: 1614-5046
Elektronische ISSN: 1614-5054
DOI
https://doi.org/10.1007/s11334-015-0245-7

Weitere Artikel der Ausgabe 4/2015

Innovations in Systems and Software Engineering 4/2015 Zur Ausgabe

Premium Partner