nach oben

Software Quality Journal

Erschienen in:

01.03.2012

Generating model-based test cases from natural language requirements for space application software

verfasst von: Valdivino Alexandre de Santiago Júnior, Nandamudi Lankalapalli Vijaykumar

Erschienen in: Software Quality Journal | Ausgabe 1/2012

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Natural Language (NL) deliverables suffer from ambiguity, poor understandability, incompleteness, and inconsistency. Howewer, NL is straightforward and stakeholders are familiar with it to produce their software requirements documents. This paper presents a methodology, SOLIMVA, which aims at model-based test case generation considering NL requirements deliverables. The methodology is supported by a tool that makes it possible to automatically translate NL requirements into Statechart models. Once the Statecharts are derived, another tool, GTSC, is used to generate the test cases. SOLIMVA uses combinatorial designs to identify scenarios for system and acceptance testing, and it requires that a test designer defines the application domain by means of a dictionary. Within the dictionary there is a Semantic Translation Model in which, among other features, a word sense disambiguation method helps in the translation process. Using as a case study a space application software product, we compared SOLIMVA with a previous manual approach developed by an expert under two aspects: test objectives coverage and characteristics of the Executable Test Cases. In the first aspect, the SOLIMVA methodology not only covered the test objectives associated to the expert’s scenarios but also proposed a better strategy with test objectives clearly separated according to the directives of combinatorial designs. The Executable Test Cases derived in accordance with the SOLIMVA methodology not only possessed similar characteristics with the expert’s Executable Test Cases but also predicted behaviors that did not exist in the expert’s strategy. The key benefits from applying the SOLIMVA methodology/tool within a Verification and Validation process are the ease of use and, at the same time, the support of a formal method consequently leading to a potential acceptance of the methodology in complex software projects.

Vorheriger Artikel A test case refactoring approach for pattern-based software development

Nächster Artikel Who tested my software? Testing as an organizationally cross-cutting activity

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

At present, SOLIMVA uses an open source tool, TConfig (University of Ottawa 2008), which has implemented the IPO procedure. The IPO procedure was originally conceived for 2-way (pairwise) testing. The TConfig tool can generate from 2-way to 6-way Mixed-Level Covering Arrays. However, it is not clear if the tool has implemented the In-Parameter-Order-General (IPOG) (Lei et al. 2007) algorithm which deals with general t-way testing or else if another approach was adopted.

Lexical category or part of speech is a linguistic category of words (or more precisely lexical items), which is generally defined by the syntactic or morphological behavior of the lexical item in question. Common linguistic categories include noun and verb, among others.

The Stanford POS Tagger adopts the Penn Treebank POS tagset (Marcus et al. 1993).

In linguistics, one definition of lemma is: the canonical form, dictionary form, or citation form of a set of words (headword). For instance, in English, “run”, “runs”, and “ran” are forms of the same lexeme, with “run” as the lemma.

A path is a finite sequence of adjacent transitions.

POS tagging is also known as word category disambiguation.

Traditionally, an FSM is seen as a computational model and not as a language (Hopcroft and Ullman, 1979).

In hidden inconsistencies, the inconsistency occurs due to the consequences of some requirements rather than the requirements themselves.

Actually, CARL does not really apply model checking according to its most common definition (Clarke and Lerda 2007; Baier and Katoen 2008).

Abrial, J. R. (2006). Formal methods in industry: Achievements, problems, future. In Proceedings of the 28th international conference on software engineering (ICSE) (pp. 761–768). New York, NY: ACM.

Ambriola V., & Gervasi V. (1997). Processing natural language requirements. In Proceedings of the 12th international conference on automated software engineering (ASE) (pp. 36–45). Washington, DC: IEEE Computer Society.

Ambriola, V., & Gervasi, V. (2006). On the systematic analysis of natural language requirements with CIRCE. Automated Software Engineering, 13(1), 107–167.CrossRef

Ambrósio, A. M., Mattiello-Francisco, F., Santiago, V. A., Silva, W. P., & Martins, E. (2007). Designing fault injection experiments using state-based model to test a space software. In Dependable computing (LNCS, Vol. 4746, pp. 170–178). Berlin/Heidelberg, Germany: Springer.

Antoniol, G., Briand, L. C., Di Penta, M., & Labiche, Y. (2002). A case study using the round-trip strategy for state-based class testing. In Proceedings of the 13th IEEE international symposium on software reliability engineering (ISSRE) (pp. 269–279). Washington, DC: IEEE Computer Society.

Baier, C., & Katoen, J. P. (2008). Principles of model checking. Cambridge, MA: The MIT Press.MATH

Balcer, M., Hasling, W., & Ostrand, T. (1989). Automatic generation of test scripts from formal test specifications. ACM SIGSOFT Software Engeneering Notes, 14(8), 210–218.CrossRef

Basili, V. R., Green, S., Laitenberger, O., Lanubile, F., Shull, F., Sørumgård, S., et al. (1996). The empirical investigation of perspective-based reading. Empirical Software Engineering Journal, 1(2), 133–164.CrossRef

Bertolino, A., & Gnesi, S. (2003). Use case-based testing of product lines. ACM SIGSOFT Software Engineering Notes, 28(5), 355–358.CrossRef

Binder, R. V. (1999). Testing object-oriented systems. USA: Addison-Wesley.

Bresciani, P., Perini, A., Giorgini, P., Giunchiglia, F., & Mylopoulos, J. (2004). Tropos: An agent-oriented software development methodology. Autonomous Agents and Multi-Agent Systems, 8(3), 203–236.CrossRef

Briand, L. C., & Labiche, Y. (2002). A UML-based approach to system testing. Journal of Software and Systems Modeling, 1(1), 10–42.CrossRef

Briand, L. C., Labiche. Y., & Wang, Y. (2004). Using simulation to empirically investigate test coverage criteria based on Statechart. In Proceedings of the 26th international conference on software engineering (ICSE) (pp. 86–95). Washington, DC: IEEE Computer Society.

Chow, T. S. (1978). Testing software design modeled by finite-state machines. IEEE Transactions on Software Engineering, SE-4(3), 178–187.CrossRef

Clarke, E. M., & Lerda, F. (2007). Model checking: Software and beyond. Journal of Universal Computer Science, 13(5), 639–649.MathSciNet

Cristiá, M., & Monetti, P. (2009). Implementing and applying the Stocks–Carrington framework for model-based testing. In K. Breitman, & A. Cavalcanti (Eds.), Formal methods and software engineering (LNCS, Vol. 5885, pp. 167–185). Berlin/Heidelberg, Germany: Springer.CrossRef

Cristiá, M., Santiago, V., & Vijaykumar, N. L. (2010). On comparing and complementing two MBT approaches. In Proceedings of the 11th Latin–American test workshop (LATW) (pp. 1–6). Washington, DC: IEEE Computer Society.

ECSS. (2008). ECSS-S-ST-00C: ECSS system—description, implementation and general requirements. Noordwijk, The Netherlands: European Cooperation for Space Standardization (ECSS).

ECSS. (2009). ECSS-E-ST-10-06C: ECSS space engineering—technical requirements specification. Noordwijk, The Netherlands: European Cooperation for Space Standardization (ECSS).

El-Far, I. K., & Whittaker, J. A. (2001). Model-based software testing. In J. J. Marciniak (Ed.), Encyclopedia of software engineering. USA: Wiley.

Fantechi, A., Gnesi, S., Lami, G., & Maccari, A. (2003). Applications of linguistic techniques for use case analysis. Requirements Engineering, 8(3), 161–170.CrossRef

Fantechi, A., & Spinicci, E. (2005). A content analysis technique for inconsistency detection in software requirements documents. In Proceedings of the VIII workshop on requirements engineering (WER) (pp. 245–256).

Fraser, M. D., Kumar, K., & Vaishnavi, V. K. (1991). Informal and formal requirements specification languages: Bridging the gap. IEEE Transactions on Software Engineering, 17(5), 454–466.CrossRef

Frohlich, P., & Link, J. (2000). Automated test case generation from dynamic models. In ECOOP 2000: Object-oriented programming (LNCS, Vol. 1850, pp. 472–491). Berlin/Heidelberg, Germany: Springer.

Fuchs, N. E., Schwertel, U., & Schwitter, R. (1999). Attempto Controlled English—not just another logic specification language. In Logic-based program synthesis and transformation (LNCS, Vol. 1559, pp. 1–20). Berlin/Heidelberg, Germany: Springer.

Fuchs, N. E., Schwertel, U., & Torge, S. (2000). A natural language front-end to model generation. Journal of Language and Computation, 1(2), 199–214.

Gervasi, V., & Zowghi, D. (2005). Reasoning about inconsistencies in natural language requirements. ACM Transactions on Software Engineering and Methodology, 14(3), 277–330.CrossRef

Harel, D. (1987). Statecharts: A visual formalism for complex systems. Science of Computer Programming, 8, 231–274.CrossRefMATHMathSciNet

Harel, D., Pnueli, A., Schmidt, J. P., & Sherman, R. (1987). On the formal semantics of Statecharts (extended abstract). In Proceedings of the 2nd IEEE symposium on logic in computer science (LICS) (pp. 54–64). Washington, DC: IEEE Computer Society.

Hierons, R. M. (1997). Testing from a Z specification. The Journal of Software Testing, Verification and Reliability, 7(1), 19–33.CrossRef

Hierons, R. M., Bogdanov, K., Bowen, J. P., Cleaveland, R., Derrick, J., Dick, J., et al. (2009). Using formal specifications to support testing. ACM Computing Surveys, 41(2), 1–76.CrossRef

Hoare, C. A. R. (1985). Communicating sequential processes. Englewood Cliffs, NJ: Prentice-Hall.MATH

Hopcroft, J. E., & Ullman, J. D. (1979). Introduction to automata theory, languages, and computation. Reading, MA: Addison Wesley.MATH

Howden, W. E. (1976). Reliability of the path analysis testing strategy. IEEE Transactions on Software Engineering, SE-2(3), 208–215.CrossRefMathSciNet

IEEE. (1990). IEEE Std 610.12-1990: IEEE standard glossary of software engineering terminology. New York, NY: The Institute of Electrical and Electronics Engineers (IEEE).

Jiang, J. J., & Conrath, D. W. (1997). Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of the 10th international conference research on computational linguistics (ROCLING) (pp. 19–33).

Kim, H. Y., & Sheldon, F. T. (2004). Testing software requirements with Z and Statecharts applied to an embedded control system. Software Quality Journal, 12(3), 231–264.CrossRef

Leacock, C., & Chodorow, M. (1998). Combining local context and WordNet similarity for word sense identification. In C. Fellbaum (Ed.), WordNet: An electronic lexical database, chap 11 (pp. 265–283). Cambridge, MA: The MIT Press.

Lee, D., & Yannakakis, M. (1996). Principles and methods of testing finite state machines: A survey. Proceedings of the IEEE, 84(8), 1090–1123.

Lei, Y., Kacker, R., Kuhn, D. R., Okun, V., & Lawrence, J. (2007). IPOG: A general strategy for t-way software testing. In Proceedings of the 14th annual IEEE international conference and workshops on the engineering of computer-based systems (ECBS) (pp. 549–556). Washington, DC: IEEE Computer Society.

Lei, Y., & Tai, K. C. (1998). In-parameter-order: A test generation strategy for pairwise testing. In Proceedings of the 3rd IEEE international symposium on high-assurance systems engineering (HASE) (pp. 254–261). Washington, DC: IEEE Computer Society.

Lesk, M. (1986). Automatic sense disambiguation using machine readable dictionaries: How to tell a pine cone from an ice cream cone. In Proceedings of the 5th international conference on systems documentation (SIGDOC) (pp. 24–26). New York, NY: ACM.

Liang, J., & Palmer, J. D. (1994). A pattern matching and clustering based approach for supporting requirements transformation. In Proceedings of the 1st IEEE international conference on requirements engineering (ICRE) (pp. 180–183). Washington, DC: IEEE Computer Society.

Lu, C. W., Chang, C. H., Chu, W. C., Cheng, Y. W., & Chang, H.C. (2008). A requirement tool to support model-based requirement engineering. In Proceedings of the 32nd annual IEEE international computer software and applications conference (COMPSAC) (pp. 712–717). Washington, DC: IEEE Computer Society.

Marcus, M. P., Marcinkiewicz, M. A., & Santorini, B. (1993). Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2), 313–330.

Masiero, P. C., Maldonado, J. C., & Boaventura, I. G. (1994). A reachability tree for statecharts and analysis of some properties. Information and Software Technology, 36(10), 615–624.CrossRef

Mathur, A. P. (2008). Foundations of software testing. Delhi, India: Dorling Kindersley (India) Pearson Education in South Asia.

Mich, L. (1996). NL-OOPS: From natural language to object oriented requirements using the natural language processing system LOLITA. Natural Language Engineering, 2(2), 161–187.CrossRef

Mich, L., Franch, M., & Inverardi, P. (2004). Market research for requirements analysis using linguistic tools. Requirements Engineering Journal, 9(1), 40–56.CrossRef

Miller, G. A. (1998). Nouns in WordNet. In C. Fellbaum (Ed.), WordNet: An electronic lexical database, chap 1 (pp. 23–46). Cambridge, MA: The MIT Press.

Miller, G. A., Leacock, C., Tengi, R., & Bunker, R. T. (1993). A semantic concordance. In Proceedings of the workshop on human language technology (HLT) (pp. 303–308). Morristown, NJ: Association for Computational Linguistics.

Morgan, R., Garigliano, R., Callaghan, P., Poria, S., Smith, M., Urbanowicz, A., et al. (1995). University of Durham: Description of the LOLITA system as used in MUC-6. In Proceedings of the 6th message understanding conference (MUC-6) (pp. 71–85).

Navigli, R. (2009). Word sense disambiguation: A survey. ACM Computing Surveys, 41(2), 1–69.CrossRef

Offutt, J., & Abdurazik, A. (1999). Generating tests from UML specifications. In UML’99: The unified modeling language (LNCS, Vol. 1723, pp 416–429). Berlin/Heidelberg, Germany: Springer.

OMG. (2007). OMG unified modeling language (OMG UML), Superstructure, V2.1.2. Needham, MA: The Object Management Group (OMG).

Ostrand, T. J., & Balcer, M. J. (1988). The category-partition method for specifying and generating functional tests. Communications of the ACM, 31(6), 676–686.CrossRef

Paradkar, A. (2003). Towards model-based generation of self-priming and self-checking conformance tests for interactive systems. In Proceedings of the 18th ACM symposium on applied computing (SAC) (pp. 1110–1117). New York, NY: ACM.

Pedersen, T., Patwardhan, S., & Michelizzi, J. (2004). WordNet::Similarity: Measuring the relatedness of concepts. In Proceedings of the 5th annual conference of the North American chapter of the association for computational linguistics (NAACL) (pp. 38–41). Morristown, NJ: Association for Computational Linguistics.

Petrenko, A., & Yevtushenko, N. (2005). Testing from partial deterministic FSM specifications. IEEE Transactions on Computers, 54(9), 1154–1165.CrossRef

Pimont, S., & Rault, J. C. (1976). A software reliability assessment based on a structural and behavioral analysis of programs. In Proceedings of the 2nd international conference on software engineering (ICSE) (pp. 486–491). New York, NY: ACM.

Pressman, R. S. (2001). Software Engineering: A practitioner’s approach (5th ed.). New York, NY: McGraw-Hill.

Santiago, V., Amaral, A. S. M., Vijaykumar, N. L., Mattiello-Francisco, M. F., Martins, E., & Lopes, O. C. (2006). A practical approach for automated test case generation using statecharts. In Proceedings of the 30th annual international computer software and applications conference (COMPSAC)—international workshop on testing and quality assurance for component-based systems (TQACBS) (pp. 183–188). Los Alamitos, CA: IEEE Computer Society.

Santiago, V., Mattiello-Francisco, F., Costa, R., Silva, W. P., & Ambrosio, A. M. (2007). QSEE project: An experience in outsourcing software development for space applications. In Proceedings of the 19th international conference on software engineering and knowledge engineering (SEKE) (pp. 51–56). Skokie, IL: Knowledge Systems Institute Graduate School.

Santiago, V., Silva, W. P., & Vijaykumar, N. L. (2008a). Shortening test case execution time for embedded software. In Proceedings of the 2nd IEEE international conference on secure system integration and reliability improvement (SSIRI) (pp.81–88). Washington, DC: IEEE Computer Society.

Santiago, V., Vijaykumar, N. L., Guimaraes, D., Amaral, A. S., & Ferreira, E. (2008b). An environment for automated test case generation from statechart-based and finite state machine-based behavioral models. In Proceedings of the 1st international conference on software testing, verification and validation (ICST)—workshop on advances in model based testing (A-MOST) (pp. 63–72). Washington, DC: IEEE Computer Society.

Santiago Júnior, V. A., Cristiá, M., & Vijaykumar, N. L. (2010). Model-based test case generation using Statecharts and Z: A comparison and a combined approach. INPE, São José dos Campos, http://www.urlib.net/sid.inpe.br/mtc-m19@80/2010/02.26.14.05, (INPE-16677-RPQ/850)

Sarma, M., & Mall, R. (2009). Automatic generation of test specifications for coverage of system state transitions. Information and Software Technology, 51(2), 418–432.CrossRef

Sidhu, D. P., & Leung, T. K. (1989). Formal methods for protocol testing: A detailed study. IEEE Transactions on Software Engineering, 15(4), 413–426.CrossRef

Sinha, R., & Mihalcea, R. (2007). Unsupervised graph-based word sense disambiguation using measures of word semantic similarity. In Proceedings of the international conference on semantic computing (ICSC) (pp. 363–369). Washington, DC: IEEE Computer Society.

Sleator, D. D., & Temperley, D. (1993). Parsing English with a link grammar. In Proceedings of the 3rd international workshop on parsing technologies (pp. 277–292).

Sneed, H. M. (2007). Testing against natural language requirements. In Proceedings of the 7th international conference on quality software (QSIC) (pp. 380–387). Washington, DC: IEEE Computer Society.

Souza, É. F. (2010). Geração de casos de teste para sistemas da área espacial usando critérios de teste para máquinas de estados finitos. Master Dissertation (133 p.). Instituto Nacional de Pesquisas Espaciais, São José dos Campos, SP.

Souza, S. R. S. (2000). Validação de especificações de sistemas reativos: Definição e análise de critérios de teste. PhD Thesis (264 p.). Universidade de São Paulo, São Carlos, SP.

Spivey, J. M. (1989). The Z notation: a reference manual. Upper Saddle River, NJ: Prentice-HallMATH

Toutanova, K., Klein, D., Manning, C. D., & Singer, Y. (2003). Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the conference of the North American chapter of the association for computational linguistics on human language technology (pp. 173–180).

University of Ottawa. (2008). Alan Williams’ page. Available from: http://www.site.uottawa.ca/~awilliam/. Accessed in July 22, 2010.

University of Sussex. (2010). David Hope’s page. Available from: http://www.cogs.susx.ac.uk/users/drh21/. Accessed in July 22, 2010.

Vijaykumar, N. L., Carvalho, S. V., Francês, C. R. L., Abdurahiman, V., & Amaral, A. S. M. (2006). Performance evaluation from statecharts representation of complex systems: Markov approach. In Anais do XXVI Congresso da Sociedade Brasileira de Computação (CSBC)—Workshop em Desempenho de Sistemas Computacionais e de Comunicação (pp. 183–202). Porto Alegre, RS: Sociedade Brasileira de Computação.

Titel: Generating model-based test cases from natural language requirements for space application software
verfasst von: Valdivino Alexandre de Santiago Júnior
Nandamudi Lankalapalli Vijaykumar
Publikationsdatum: 01.03.2012
Verlag: Springer US
Erschienen in: Software Quality Journal / Ausgabe 1/2012
Print ISSN: 0963-9314
Elektronische ISSN: 1573-1367
DOI: https://doi.org/10.1007/s11219-011-9155-6

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Weitere Artikel der Ausgabe 1/2012

A conceptual modeling quality framework

Who tested my software? Testing as an organizationally cross-cutting activity

A test case refactoring approach for pattern-based software development

Requirements and constructors for tailoring software processes: a systematic literature review

In this issue

Predicting high-risk program modules by selecting the right software measurements

Premium Partner