Skip to main content
Erschienen in: International Journal on Software Tools for Technology Transfer 3/2016

01.06.2016 | ICTSS 2013

Complete model-based equivalence class testing

verfasst von: Wen-ling Huang, Jan Peleska

Erschienen in: International Journal on Software Tools for Technology Transfer | Ausgabe 3/2016

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this article, we present a model-based black-box equivalence partition testing strategy, together with a formal proof of its completeness properties. The results apply to reactive systems with large, possibly infinite input data types and finite internal and output data ranges that may be enumerated with acceptable effort. The investigation is performed on a semantic level and applies to all concrete test models whose behavioural semantics can be encoded as a variant of state transition systems. Test suite construction is performed in relation to a given fault model \(\mathcal{F}\) for which a finite black-box test suite can be constructed which is complete with respect to \(\mathcal{F}\). It is shown how the test suite generation can be effectively implemented by model-based testing tools, using propositional representations of behavioural model semantics and constraint solvers. A SysML model of the ceiling speed monitoring function of the European Train Control System is presented as a case study, to explain theory application to a concrete modelling formalism.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
Note that other authors, for example [13], reserve the term ‘exhaustiveness’ for test suites containing all possible tests and use validity instead, if (1) is fulfilled, and unbias in situation (2).
 
2
In [20, Chapter 11], for example, it is shown how to construct \(\mathcal{R}_1\) for a class of SysML models that also contains the model used in the case study in Sect. 4.
 
3
The full CSM specification distinguishes between service and emergency brakes, while the configuration presented here corresponds to the situation where the train is equipped with emergency brakes only. A test model comprising the full CSM functionality has been made available by the authors under http://​www.​mbt-benchmarks.​org.
 
4
This would be an exceptional behaviour situation, caused, for example, by temporary unavailability of odometry data, so that a “sudden jump” of \(v\) would be observed by the CSM.
 
Literatur
1.
Zurück zum Zitat Anand, Saswat, Burke, Edmund K., Chen, Tsong Yueh, Clark, John A., Cohen, Myra B., Grieskamp, W., Harman, M., Harrold, M.J., McMinn, P.: An orchestrated survey of methodologies for automated software test case generation. J. Syst. Softw. 86(8), 1978–2001 (2013)CrossRef Anand, Saswat, Burke, Edmund K., Chen, Tsong Yueh, Clark, John A., Cohen, Myra B., Grieskamp, W., Harman, M., Harrold, M.J., McMinn, P.: An orchestrated survey of methodologies for automated software test case generation. J. Syst. Softw. 86(8), 1978–2001 (2013)CrossRef
2.
Zurück zum Zitat Binder, R.V.: Testing object-oriented systems: models, patterns, and tools. Addison-Wesley, Reading (2000) Binder, R.V.: Testing object-oriented systems: models, patterns, and tools. Addison-Wesley, Reading (2000)
4.
Zurück zum Zitat Chen, T.Y., Tse, T.H., Yu, Y.T.: Proportional sampling strategy: a compendium and some insights. J. Syst. Softw. 58(1), 65–81 (2001) Chen, T.Y., Tse, T.H., Yu, Y.T.: Proportional sampling strategy: a compendium and some insights. J. Syst. Softw. 58(1), 65–81 (2001)
5.
Zurück zum Zitat Chow, T.S.: Testing software design modeled by finite-state machines. IEEE Trans. Softw. Eng. 4(3), 178–187 (1978) Chow, T.S.: Testing software design modeled by finite-state machines. IEEE Trans. Softw. Eng. 4(3), 178–187 (1978)
6.
Zurück zum Zitat Clarke, E.M., Grumberg, O., Peled, D.A.: Model Checking. The MIT Press, Cambridge (1999) Clarke, E.M., Grumberg, O., Peled, D.A.: Model Checking. The MIT Press, Cambridge (1999)
7.
Zurück zum Zitat Dick, Jeremy, Faivre, Alain: Automating the generation and sequencing of test cases from model-based specifications. In: Woodcock, J.C.P., Larsen, P.G. (eds.) FME ’93: Industrial-strength formal methods. Lecture notes in computer science, pp. 268–284. Springer, Berlin (1993)CrossRef Dick, Jeremy, Faivre, Alain: Automating the generation and sequencing of test cases from model-based specifications. In: Woodcock, J.C.P., Larsen, P.G. (eds.) FME ’93: Industrial-strength formal methods. Lecture notes in computer science, pp. 268–284. Springer, Berlin (1993)CrossRef
8.
Zurück zum Zitat Doucet, F., Menarini, M., Krüger, I.H., Gupta, R.K., Talpin, J.-P.: A verification approach for gals integration of synchronous components. Electr. Notes Theor. Comput. Sci. 146(2), 105–131 (2006)CrossRef Doucet, F., Menarini, M., Krüger, I.H., Gupta, R.K., Talpin, J.-P.: A verification approach for gals integration of synchronous components. Electr. Notes Theor. Comput. Sci. 146(2), 105–131 (2006)CrossRef
9.
Zurück zum Zitat European Committee for Electrotechnical Standardization: EN 50128—Railway applications—Communications, signalling and processing systems—Software for railway control and protection systems. CENELEC, Brussels (2001) European Committee for Electrotechnical Standardization: EN 50128—Railway applications—Communications, signalling and processing systems—Software for railway control and protection systems. CENELEC, Brussels (2001)
11.
Zurück zum Zitat Frantzen, L., Tretmans, J., Willemse, Tim A.C.: Test generation based on symbolic specifications. In: Grabowski, J., Nielsen, B. (eds.) Formal approaches to software testing. Lecture notes in computer science, pp. 1–15. Springer, Berlin (2005)CrossRef Frantzen, L., Tretmans, J., Willemse, Tim A.C.: Test generation based on symbolic specifications. In: Grabowski, J., Nielsen, B. (eds.) Formal approaches to software testing. Lecture notes in computer science, pp. 1–15. Springer, Berlin (2005)CrossRef
12.
Zurück zum Zitat Fujiwara, S., Bochmann, G., Khendek, F., Amalou, M., Ghedamsi, A.: Test selection based on finite state models. IEEE Trans. Softw. Eng. 17(6), 591–603 (1991)CrossRef Fujiwara, S., Bochmann, G., Khendek, F., Amalou, M., Ghedamsi, A.: Test selection based on finite state models. IEEE Trans. Softw. Eng. 17(6), 591–603 (1991)CrossRef
13.
Zurück zum Zitat Gaudel, M.-C.: Testing can be formal, too. In: Mosses, P.D., Nielsen, M., Schwartzbach, M.I. (eds.) TAPSOFT. Lecture Notes in Computer Science, pp. 82–96. Springer, Berlin (1995) Gaudel, M.-C.: Testing can be formal, too. In: Mosses, P.D., Nielsen, M., Schwartzbach, M.I. (eds.) TAPSOFT. Lecture Notes in Computer Science, pp. 82–96. Springer, Berlin (1995)
14.
Zurück zum Zitat Gnesi, S., Latella, D., Massink, M.: Formal test-case generation for uml statecharts. In: Ninth IEEE International Conference on Engineering Complex Computer Systems (ICECCS’04), iceccs, pp. 75–84 (2004) Gnesi, S., Latella, D., Massink, M.: Formal test-case generation for uml statecharts. In: Ninth IEEE International Conference on Engineering Complex Computer Systems (ICECCS’04), iceccs, pp. 75–84 (2004)
15.
Zurück zum Zitat Gill, A.: Introduction to the Theory of Finite-State Machines. McGraw-Hill, New York (1962) Gill, A.: Introduction to the Theory of Finite-State Machines. McGraw-Hill, New York (1962)
16.
Zurück zum Zitat Grieskamp, W., Gurevich, Y., Schulte, W., Veanes, M.: Generating finite state machines from abstract state machines. ACM SIGSOFT Softw. Eng. Notes 27(4), 112–122 (2002)CrossRef Grieskamp, W., Gurevich, Y., Schulte, W., Veanes, M.: Generating finite state machines from abstract state machines. ACM SIGSOFT Softw. Eng. Notes 27(4), 112–122 (2002)CrossRef
17.
Zurück zum Zitat Helke, S., Neustupny, T., Santen, T.: Automating test case generation from z specifications with isabelle. In: Bowen, J.P., Hinchey, M.G., Till, D. (eds.) ZUM ’97: The Z Formal Specification Notation, vol. 1212. Lecture Notes in Computer Science, pp. 52–71. Springer, Berlin (1997) Helke, S., Neustupny, T., Santen, T.: Automating test case generation from z specifications with isabelle. In: Bowen, J.P., Hinchey, M.G., Till, D. (eds.) ZUM ’97: The Z Formal Specification Notation, vol. 1212. Lecture Notes in Computer Science, pp. 52–71. Springer, Berlin (1997)
18.
Zurück zum Zitat Hong, H.S., Lee, I., Sokolsky, O., Ural, H.: A temporal logic based theory of test coverage and generation. In: Katoen, J.-P., Stevens, P. (eds.) TACAS. Lecture Notes in Computer Science, pp. 327–341. Springer, Berlin (2002) Hong, H.S., Lee, I., Sokolsky, O., Ural, H.: A temporal logic based theory of test coverage and generation. In: Katoen, J.-P., Stevens, P. (eds.) TACAS. Lecture Notes in Computer Science, pp. 327–341. Springer, Berlin (2002)
19.
Zurück zum Zitat Huang, Wen-ling, Peleska, Jan: Exhaustive model-based equivalence class testing. In: Yenigün, Hüsnü, Yilmaz, Cemal, Ulrich, Andreas (eds.) Testing software and systems. Lecture notes in computer science, pp. 49–64. Springer, Berlin (2013) Huang, Wen-ling, Peleska, Jan: Exhaustive model-based equivalence class testing. In: Yenigün, Hüsnü, Yilmaz, Cemal, Ulrich, Andreas (eds.) Testing software and systems. Lecture notes in computer science, pp. 49–64. Springer, Berlin (2013)
21.
Zurück zum Zitat ISO/DIS 26262-4 Road vehicles—Functional safety—Part 4: Product development: system level. Technical report, International Organization for Standardization (2009) ISO/DIS 26262-4 Road vehicles—Functional safety—Part 4: Product development: system level. Technical report, International Organization for Standardization (2009)
22.
Zurück zum Zitat ISO/IEC/IEEE 29119-1:2013(e): Software and systems engineering—software testing—part 1: Concepts and definitions, Sept (2013) ISO/IEC/IEEE 29119-1:2013(e): Software and systems engineering—software testing—part 1: Concepts and definitions, Sept (2013)
23.
Zurück zum Zitat ISO/IEC/IEEE 29119-2:2013(e): Software and systems engineering—software testing—part 2: Test processes. Sept (2013) ISO/IEC/IEEE 29119-2:2013(e): Software and systems engineering—software testing—part 2: Test processes. Sept (2013)
24.
Zurück zum Zitat Kalaji, A.S., Hierons, R.M., Swift, S.: Generating feasible transition paths for testing from an extended finite state machine (efsm). In: ICST, IEEE Computer Society, Silver Spring, pp. 230–239 (2009) Kalaji, A.S., Hierons, R.M., Swift, S.: Generating feasible transition paths for testing from an extended finite state machine (efsm). In: ICST, IEEE Computer Society, Silver Spring, pp. 230–239 (2009)
26.
Zurück zum Zitat Object Management Group: OMG Systems Modeling Language (\(\text{ OMG } \text{ SysML }^{{TM}}\)). Technical report, Object Management Group, 2010. OMG Document Number: formal/2010-06-02 (2010) Object Management Group: OMG Systems Modeling Language (\(\text{ OMG } \text{ SysML }^{{TM}}\)). Technical report, Object Management Group, 2010. OMG Document Number: formal/2010-06-02 (2010)
27.
Zurück zum Zitat Object Management Group: OMG Unified Modeling Language (OMG UML), superstructure, version 2.4.1. Technical report, OMG (2011) Object Management Group: OMG Unified Modeling Language (OMG UML), superstructure, version 2.4.1. Technical report, OMG (2011)
28.
Zurück zum Zitat Peleska, J., Siegel, M.: Test automation of safety–critical reactive systems. S. Afr. Comput. J. 19, 53–77 (1997) Peleska, J., Siegel, M.: Test automation of safety–critical reactive systems. S. Afr. Comput. J. 19, 53–77 (1997)
29.
Zurück zum Zitat Peleska, J.: Industrial-strength model-based testing—state of the art and current challenges. In: Petrenko, A.K., Schlingloff, H. (eds.) Proceedings Eighth Workshop on Model-Based Testing, Rome, Italy, 17th March 2013. Electronic Proceedings in Theoretical Computer Science, vol. 111, pp. 3–28. Open Publishing Association (2013) Peleska, J.: Industrial-strength model-based testing—state of the art and current challenges. In: Petrenko, A.K., Schlingloff, H. (eds.) Proceedings Eighth Workshop on Model-Based Testing, Rome, Italy, 17th March 2013. Electronic Proceedings in Theoretical Computer Science, vol. 111, pp. 3–28. Open Publishing Association (2013)
30.
Zurück zum Zitat Peleska, J., Honisch, A., Lapschies, F., Löding, H., Schmid, H., Smuda, P., Vorobev, E., Zahlten, C.: A real-world benchmark model for testing concurrent real-time systems in the automotive domain. In: Wolff, B., Zaidi, F. (eds.) Testing Software and Systems. Proceedings of the 23rd IFIP WG 6.1 International Conference, ICTSS 2011. LNCS, vol. 7019, pp. 146–161, Nov 2011. IFIP WG 6.1, Springer, Berlin (2011) Peleska, J., Honisch, A., Lapschies, F., Löding, H., Schmid, H., Smuda, P., Vorobev, E., Zahlten, C.: A real-world benchmark model for testing concurrent real-time systems in the automotive domain. In: Wolff, B., Zaidi, F. (eds.) Testing Software and Systems. Proceedings of the 23rd IFIP WG 6.1 International Conference, ICTSS 2011. LNCS, vol. 7019, pp. 146–161, Nov 2011. IFIP WG 6.1, Springer, Berlin (2011)
31.
Zurück zum Zitat Peleska, J., Vorobev, E., Lapschies, F.: Automated test case generation with SMT-solving and abstract interpretation. In: Bobaru, M., Havelund, K., Holzmann, G.J., Joshi, R. (eds.) Nasa Formal Methods. Third International Symposium, NFM 2011. LNCS, vol. 6617, pp. 298–312. Springer, Berlin (2011) Peleska, J., Vorobev, E., Lapschies, F.: Automated test case generation with SMT-solving and abstract interpretation. In: Bobaru, M., Havelund, K., Holzmann, G.J., Joshi, R. (eds.) Nasa Formal Methods. Third International Symposium, NFM 2011. LNCS, vol. 6617, pp. 298–312. Springer, Berlin (2011)
32.
Zurück zum Zitat Petrenko, A., Yevtushenko, N., Bochmann, G.v: Fault models for testing in context. In: Gotzhein, Reinhard, Bredereke, Jan (eds.) Formal description techniques IX—theory, application and tools, pp. 163–177. Chapman & Hall, London (1996)CrossRef Petrenko, A., Yevtushenko, N., Bochmann, G.v: Fault models for testing in context. In: Gotzhein, Reinhard, Bredereke, Jan (eds.) Formal description techniques IX—theory, application and tools, pp. 163–177. Chapman & Hall, London (1996)CrossRef
33.
Zurück zum Zitat Petrenko, A., Simao, A., Maldonado, J.C.: Model-based testing of software and systems: recent advances and challenges. Int. J. Softw. Tools Technol. Transf. 14(4), 383–386 (2012)CrossRef Petrenko, A., Simao, A., Maldonado, J.C.: Model-based testing of software and systems: recent advances and challenges. Int. J. Softw. Tools Technol. Transf. 14(4), 383–386 (2012)CrossRef
34.
Zurück zum Zitat Ranise, S., Tinelli, C.: Satisfiability modulo theories. IEEE Mag. Intell. Syst. Trends Controv. 21(6), 71–81 (2006) Ranise, S., Tinelli, C.: Satisfiability modulo theories. IEEE Mag. Intell. Syst. Trends Controv. 21(6), 71–81 (2006)
35.
Zurück zum Zitat RTCA, SC-167 Software Considerations in Airborne Systems and Equipment Certification, RTCA/DO-178B RTCA (1992) RTCA, SC-167 Software Considerations in Airborne Systems and Equipment Certification, RTCA/DO-178B RTCA (1992)
36.
Zurück zum Zitat Spillner, A., Linz, T., Schaefer, H.: Software testing foundations. Dpunkt.verlag, Heidelberg (2006) Spillner, A., Linz, T., Schaefer, H.: Software testing foundations. Dpunkt.verlag, Heidelberg (2006)
37.
Zurück zum Zitat Springintveld, J.G., Vaandrager, F.W., D’Argenio, P.R.: Testing timed automata. Theor. Comput. Sci. 254(1–2), 225–257 (2001)MathSciNetCrossRefMATH Springintveld, J.G., Vaandrager, F.W., D’Argenio, P.R.: Testing timed automata. Theor. Comput. Sci. 254(1–2), 225–257 (2001)MathSciNetCrossRefMATH
38.
Zurück zum Zitat Tretmans, J.: Conformance testing with labelled transition systems: implementation relations and test generation. Comput. Netw. ISDN Syst. 29(1), 49–79 (1996)CrossRef Tretmans, J.: Conformance testing with labelled transition systems: implementation relations and test generation. Comput. Netw. ISDN Syst. 29(1), 49–79 (1996)CrossRef
39.
Zurück zum Zitat Tretmans, J.: Model based testing with labelled transition systems. In: Hierons, R.M., Bowen, J.P., Harman, M. (eds.) Formal Methods and Testing. Lecture Notes in Computer Science, vol. 4949, pp. 1–38. Springer, Berlin (2008) Tretmans, J.: Model based testing with labelled transition systems. In: Hierons, R.M., Bowen, J.P., Harman, M. (eds.) Formal Methods and Testing. Lecture Notes in Computer Science, vol. 4949, pp. 1–38. Springer, Berlin (2008)
40.
Zurück zum Zitat UNISIG: ERTMS/ETCS System Requirements Specification, chap. 3. Principles, vol. Subset-026-3, chapt. 3. Issue 3.3.0 (2012) UNISIG: ERTMS/ETCS System Requirements Specification, chap. 3. Principles, vol. Subset-026-3, chapt. 3. Issue 3.3.0 (2012)
41.
Zurück zum Zitat Vasilevskii, M.P.: Failure diagnosis of automata. Kibernetika (Transl.) 4, 98–108 (1973)MathSciNet Vasilevskii, M.P.: Failure diagnosis of automata. Kibernetika (Transl.) 4, 98–108 (1973)MathSciNet
Metadaten
Titel
Complete model-based equivalence class testing
verfasst von
Wen-ling Huang
Jan Peleska
Publikationsdatum
01.06.2016
Verlag
Springer Berlin Heidelberg
Erschienen in
International Journal on Software Tools for Technology Transfer / Ausgabe 3/2016
Print ISSN: 1433-2779
Elektronische ISSN: 1433-2787
DOI
https://doi.org/10.1007/s10009-014-0356-8

Weitere Artikel der Ausgabe 3/2016

International Journal on Software Tools for Technology Transfer 3/2016 Zur Ausgabe