Skip to main content
Erschienen in: Automated Software Engineering 4/2017

16.02.2017

Incremental test data generation for database queries

verfasst von: María José Suárez-Cabal, Claudio de la Riva, Javier Tuya, Raquel Blanco

Erschienen in: Automated Software Engineering | Ausgabe 4/2017

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Testing database applications is a complex task since it involves designing test databases with meaningful test data in order to reveal faults and, at the same time, with a small size in order to carry out the testing process in an efficient way. This paper presents an automated approach to generating test data (test relational databases and test inputs for query parameters) for a set of SQL queries, with the aim of covering test requirements as obtained from said queries. The test data generation follows an incremental approach where, in each increment, test data are generated to cover a test requirement by re-using test data previously generated for other test requirements. The test data generation for each test requirement is formulated as a constraint satisfaction problem, where constraints are derived from the test requirement, initial database states and previously generated test data. The generation process is fully automated and supports the execution on complex queries and databases. Evaluation is carried out on a real life application, and the results show that small-size generated test relational databases achieve high coverage scores for the queries under test in a short generating time.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Binnig, C., Kossmann, D., Lo, E.: Reverse query processing. In: IEEE 23rd International Conference on Data Engineering, 2007. ICDE 2007, pp. 506–515 (2007a) Binnig, C., Kossmann, D., Lo, E.: Reverse query processing. In: IEEE 23rd International Conference on Data Engineering, 2007. ICDE 2007, pp. 506–515 (2007a)
Zurück zum Zitat Binnig, C., Kossmann, D., Lo, E., Özsu, M.T.: Qagen: generating query-aware test databases. In: Chan, C.Y., Ooi, B.C., Zhou, A. (eds) Proceedings of the ACM SIGMOD International Conference on Management of Data, Beijing, China, June 12–14, 2007, pp. 341–352. ACM, New York, NY (2007b) Binnig, C., Kossmann, D., Lo, E., Özsu, M.T.: Qagen: generating query-aware test databases. In: Chan, C.Y., Ooi, B.C., Zhou, A. (eds) Proceedings of the ACM SIGMOD International Conference on Management of Data, Beijing, China, June 12–14, 2007, pp. 341–352. ACM, New York, NY (2007b)
Zurück zum Zitat Binnig, C., Kossmann, D., Lo, E.: Multi-rqp: generating test databases for the functional testing of oltp applications. In: Proceedings of the 1st International Workshop on Testing Database Systems, DBTest ’08, pp 5:1–5:6. ACM, New York, NY (2008) Binnig, C., Kossmann, D., Lo, E.: Multi-rqp: generating test databases for the functional testing of oltp applications. In: Proceedings of the 1st International Workshop on Testing Database Systems, DBTest ’08, pp 5:1–5:6. ACM, New York, NY (2008)
Zurück zum Zitat Blanco, R., Tuya, J., Seco, R.: Test adequacy evaluation for the user-database interaction: A specification-based approach. In: 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation (ICST), pp. 71–80 (2012) Blanco, R., Tuya, J., Seco, R.: Test adequacy evaluation for the user-database interaction: A specification-based approach. In: 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation (ICST), pp. 71–80 (2012)
Zurück zum Zitat Bruno, N., Chaudhuri, S.: Flexible database generators. In: Proceedings of the 31st International Conference on Very Large Data Bases, VLDB Endowment, VLDB ’05, pp. 1097–1107 (2005) Bruno, N., Chaudhuri, S.: Flexible database generators. In: Proceedings of the 31st International Conference on Very Large Data Bases, VLDB Endowment, VLDB ’05, pp. 1097–1107 (2005)
Zurück zum Zitat Bruno, N., Chaudhuri, S., Thomas, D.: Generating queries with cardinality constraints for dbms testing. IEEE Trans. Knowl. Data Eng. 18(12), 1721–1725 (2006)CrossRef Bruno, N., Chaudhuri, S., Thomas, D.: Generating queries with cardinality constraints for dbms testing. IEEE Trans. Knowl. Data Eng. 18(12), 1721–1725 (2006)CrossRef
Zurück zum Zitat Caballero, R., García-Ruiz, Y., Sáenz-Pérez, F.: Applying constraint logic programming to sql test case generation. In: Proceedings of the 10th International Conference on Functional and Logic Programming, FLOPS’10, pp. 191–206. Springer, Berlin (2010) Caballero, R., García-Ruiz, Y., Sáenz-Pérez, F.: Applying constraint logic programming to sql test case generation. In: Proceedings of the 10th International Conference on Functional and Logic Programming, FLOPS’10, pp. 191–206. Springer, Berlin (2010)
Zurück zum Zitat Chandra, B., Chawda, B., Kar, B., Reddy, K., Shah, S., Sudarshan, S.: Data generation for testing and grading sql queries. VLDB J. 24(6), 731–755 (2015)CrossRef Chandra, B., Chawda, B., Kar, B., Reddy, K., Shah, S., Sudarshan, S.: Data generation for testing and grading sql queries. VLDB J. 24(6), 731–755 (2015)CrossRef
Zurück zum Zitat Chays, D., Deng, Y., Frankl, P.G., Dan, S., Vokolos, F.I., Weyuker, E.J.: An agenda for testing relational database applications: research articles. Softw. Test. Verif. Reliab. 14(1), 17–44 (2004)CrossRef Chays, D., Deng, Y., Frankl, P.G., Dan, S., Vokolos, F.I., Weyuker, E.J.: An agenda for testing relational database applications: research articles. Softw. Test. Verif. Reliab. 14(1), 17–44 (2004)CrossRef
Zurück zum Zitat Chays, D., Shahid, J., Frankl, P.G.: Query-based test generation for database applications. In: Proceedings of the 1st International Workshop on Testing Database Systems, DBTest ’08, pp. 6:1–6:6. ACM, New York, NY (2008) Chays, D., Shahid, J., Frankl, P.G.: Query-based test generation for database applications. In: Proceedings of the 1st International Workshop on Testing Database Systems, DBTest ’08, pp. 6:1–6:6. ACM, New York, NY (2008)
Zurück zum Zitat Chilenski, J.J.: An investigation of three forms of the modifiedcondition decision coverage (mcdc) criterion. Tech. rep., Office of Aviation Research (2001) Chilenski, J.J.: An investigation of three forms of the modifiedcondition decision coverage (mcdc) criterion. Tech. rep., Office of Aviation Research (2001)
Zurück zum Zitat Codd, E.F.: The Relational Model for Database Management: Version 2. Addison-Wesley Longman Publishing Co. Inc, Boston, MA (1990)MATH Codd, E.F.: The Relational Model for Database Management: Version 2. Addison-Wesley Longman Publishing Co. Inc, Boston, MA (1990)MATH
Zurück zum Zitat de la Riva, C., Suárez-Cabal, M.J., Tuya, J.: Constraint-based test database generation for sql queries. In: Proceedings of the 5th Workshop on Automation of Software Test, AST ’10, pp. 67–74. ACM, New York, NY (2010) de la Riva, C., Suárez-Cabal, M.J., Tuya, J.: Constraint-based test database generation for sql queries. In: Proceedings of the 5th Workshop on Automation of Software Test, AST ’10, pp. 67–74. ACM, New York, NY (2010)
Zurück zum Zitat Emmi, M., Majumdar, R., Sen, K.: Dynamic test input generation for database applications. In: Proceedings of the 2007 International Symposium on Software Testing and Analysis, ISSTA ’07, pp. 151–162. ACM, New York, NY (2007) Emmi, M., Majumdar, R., Sen, K.: Dynamic test input generation for database applications. In: Proceedings of the 2007 International Symposium on Software Testing and Analysis, ISSTA ’07, pp. 151–162. ACM, New York, NY (2007)
Zurück zum Zitat Halfond, W., Orso, A.: Command-form coverage for testing database applications. In: 21st IEEE/ACM International Conference on Automated Software Engineering, 2006. ASE ’06. pp. 69–80 (2006) Halfond, W., Orso, A.: Command-form coverage for testing database applications. In: 21st IEEE/ACM International Conference on Automated Software Engineering, 2006. ASE ’06. pp. 69–80 (2006)
Zurück zum Zitat Houkjær, K., Torp, K., Wind, R.: Simple and realistic data generation. In: Proceedings of the 32Nd International Conference on Very Large Data Bases, VLDB Endowment, VLDB ’06, pp. 1243–1246 (2006) Houkjær, K., Torp, K., Wind, R.: Simple and realistic data generation. In: Proceedings of the 32Nd International Conference on Very Large Data Bases, VLDB Endowment, VLDB ’06, pp. 1243–1246 (2006)
Zurück zum Zitat Kapfhammer, G., McMinn, P., Wright, C.: Search-based testing of relational schema integrity constraints across multiple database management systems. In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation (ICST), pp. 31–40 (2013) Kapfhammer, G., McMinn, P., Wright, C.: Search-based testing of relational schema integrity constraints across multiple database management systems. In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation (ICST), pp. 31–40 (2013)
Zurück zum Zitat Kapfhammer, G.M., Soffa, M.L.: A family of test adequacy criteria for database-driven applications. In: Proceedings of the 9th European Software Engineering Conference Held Jointly with 11th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC/FSE-11, pp. 98–107. ACM, New York, NY (2003) Kapfhammer, G.M., Soffa, M.L.: A family of test adequacy criteria for database-driven applications. In: Proceedings of the 9th European Software Engineering Conference Held Jointly with 11th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC/FSE-11, pp. 98–107. ACM, New York, NY (2003)
Zurück zum Zitat Khalek, S., Khurshid, S.: Systematic testing of database engines using a relational constraint solver. In: 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation (ICST), pp. 50–59 (2011) Khalek, S., Khurshid, S.: Systematic testing of database engines using a relational constraint solver. In: 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation (ICST), pp. 50–59 (2011)
Zurück zum Zitat Khalek, S.A., Elkarablieh, B., Laleye, Y.O., Khurshid, S.: Query-aware test generation using a relational constraint solver. In: 23rd IEEE/ACM International Conference on Automated Software Engineering (ASE 2008), 15–19 September 2008, pp. 238–247. IEEE Computer Society, L’Aquila (2008) Khalek, S.A., Elkarablieh, B., Laleye, Y.O., Khurshid, S.: Query-aware test generation using a relational constraint solver. In: 23rd IEEE/ACM International Conference on Automated Software Engineering (ASE 2008), 15–19 September 2008, pp. 238–247. IEEE Computer Society, L’Aquila (2008)
Zurück zum Zitat Li, C., Csallner, C.: Dynamic symbolic database application testing. In: Proceedings of the Third International Workshop on Testing Database Systems, DBTest ’10, pp. 7:1–7:6. ACM, New York, NY (2010) Li, C., Csallner, C.: Dynamic symbolic database application testing. In: Proceedings of the Third International Workshop on Testing Database Systems, DBTest ’10, pp. 7:1–7:6. ACM, New York, NY (2010)
Zurück zum Zitat Lo, E., Cheng, N., Lin, W.W., Hon, W.K., Choi, B.: Mybenchmark: generating databases for query workloads. VLDB J. 23(6), 895–913 (2014)CrossRef Lo, E., Cheng, N., Lin, W.W., Hon, W.K., Choi, B.: Mybenchmark: generating databases for query workloads. VLDB J. 23(6), 895–913 (2014)CrossRef
Zurück zum Zitat Marcozzi, M., Vanhoof, W., Hainaut, J.L.: Relational symbolic execution of SQL code for unit testing of database programs. Sci. Comput. Program. 105, 44–72 (2015)CrossRef Marcozzi, M., Vanhoof, W., Hainaut, J.L.: Relational symbolic execution of SQL code for unit testing of database programs. Sci. Comput. Program. 105, 44–72 (2015)CrossRef
Zurück zum Zitat Pan, K., Wu, X., Xie, T.: Generating program inputs for database application testing. In: Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering, ASE ’11, pp. 73–82. IEEE Computer Society, Washington, DC (2011) Pan, K., Wu, X., Xie, T.: Generating program inputs for database application testing. In: Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering, ASE ’11, pp. 73–82. IEEE Computer Society, Washington, DC (2011)
Zurück zum Zitat Pan, K., Wu, X., Xie, T.: Automatic test generation for mutation testing on database applications. In: 2013 8th International Workshop on Automation of Software Test (AST), pp. 111–117 (2013) Pan, K., Wu, X., Xie, T.: Automatic test generation for mutation testing on database applications. In: 2013 8th International Workshop on Automation of Software Test (AST), pp. 111–117 (2013)
Zurück zum Zitat Pan, K., Wu, X., Xie, T.: Guided test generation for database applications via synthesized database interactions. ACM Trans. Softw. Eng. Methodol. 23(2), 12:1–12:27 (2014)CrossRef Pan, K., Wu, X., Xie, T.: Guided test generation for database applications via synthesized database interactions. ACM Trans. Softw. Eng. Methodol. 23(2), 12:1–12:27 (2014)CrossRef
Zurück zum Zitat Pan, K., Wu, X., Xie, T.: Program-input generation for testing database applications using existing database states. Autom. Softw. Eng. 22(4), 439–473 (2015)CrossRef Pan, K., Wu, X., Xie, T.: Program-input generation for testing database applications using existing database states. Autom. Softw. Eng. 22(4), 439–473 (2015)CrossRef
Zurück zum Zitat Sen, K., Marinov, D., Agha, G.: Cute: A concolic unit testing engine for c. In: Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC/FSE-13, pp. 263–272. ACM, New York, NY (2005) Sen, K., Marinov, D., Agha, G.: Cute: A concolic unit testing engine for c. In: Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC/FSE-13, pp. 263–272. ACM, New York, NY (2005)
Zurück zum Zitat Shah, S., Sudarshan, S., Kajbaje, S., Patidar, S., Gupta, B., Vira, D.: Generating test data for killing sql mutants: a constraint-based approach. In: 2011 IEEE 27th International Conference on Data Engineering (ICDE), pp. 1175–1186 (2011) Shah, S., Sudarshan, S., Kajbaje, S., Patidar, S., Gupta, B., Vira, D.: Generating test data for killing sql mutants: a constraint-based approach. In: 2011 IEEE 27th International Conference on Data Engineering (ICDE), pp. 1175–1186 (2011)
Zurück zum Zitat Suárez-Cabal, M.J., Tuya, J.: Structural coverage criteria for testing SQL queries. J. UCS 15(3), 584–619 (2009) Suárez-Cabal, M.J., Tuya, J.: Structural coverage criteria for testing SQL queries. J. UCS 15(3), 584–619 (2009)
Zurück zum Zitat Tsang, E.P.K.: Foundations of Constraint Satisfaction. Computation in Cognitive Science. Academic Press, Cambridge (1993) Tsang, E.P.K.: Foundations of Constraint Satisfaction. Computation in Cognitive Science. Academic Press, Cambridge (1993)
Zurück zum Zitat Tuya, J., Suárez-Cabal, M.J., de la Riva, C.: Mutating database queries. Inf. Softw. Technol. 49(4), 398–417 (2007)CrossRef Tuya, J., Suárez-Cabal, M.J., de la Riva, C.: Mutating database queries. Inf. Softw. Technol. 49(4), 398–417 (2007)CrossRef
Zurück zum Zitat Tuya, J., Suárez-Cabal, M.J., de la Riva, C.: Full predicate coverage for testing sql database queries. Softw. Test Verif. Reliab. 20(3), 237–288 (2010)CrossRef Tuya, J., Suárez-Cabal, M.J., de la Riva, C.: Full predicate coverage for testing sql database queries. Softw. Test Verif. Reliab. 20(3), 237–288 (2010)CrossRef
Zurück zum Zitat Veanes, M., Grigorenko, P., Halleux, P., Tillmann, N.: Symbolic query exploration. In: Proceedings of the 11th International Conference on Formal Engineering Methods: Formal Methods and Software Engineering, ICFEM ’09, pp. 49–68. Springer, Berlin (2009) Veanes, M., Grigorenko, P., Halleux, P., Tillmann, N.: Symbolic query exploration. In: Proceedings of the 11th International Conference on Formal Engineering Methods: Formal Methods and Software Engineering, ICFEM ’09, pp. 49–68. Springer, Berlin (2009)
Zurück zum Zitat Vemasani, P., Brodsky, A., Ammann, P.: Generating test data to distinguish conjunctive queries with equalities. In: Proceedings of the 2014 IEEE International Conference on Software Testing, Verification, and Validation Workshops, ICSTW ’14, pp. 216–221. IEEE Computer Society, Washington, DC (2014) Vemasani, P., Brodsky, A., Ammann, P.: Generating test data to distinguish conjunctive queries with equalities. In: Proceedings of the 2014 IEEE International Conference on Software Testing, Verification, and Validation Workshops, ICSTW ’14, pp. 216–221. IEEE Computer Society, Washington, DC (2014)
Zurück zum Zitat Willmor, D., Embury, S.: An intensional approach to the specification of test cases for database applications. In: Proceedings of the 28th International Conference on Software Engineering, ICSE ’06, pp. 102–111. ACM, New York, NY (2006a) Willmor, D., Embury, S.: An intensional approach to the specification of test cases for database applications. In: Proceedings of the 28th International Conference on Software Engineering, ICSE ’06, pp. 102–111. ACM, New York, NY (2006a)
Zurück zum Zitat Willmor, D., Embury, S.: Testing the implementation of business rules using intensional database tests. In: Testing: Academic and Industrial Conference–Practice and Research Techniques, 2006. TAIC PART 2006. Proceedings, pp. 115–126 (2006b) Willmor, D., Embury, S.: Testing the implementation of business rules using intensional database tests. In: Testing: Academic and Industrial Conference–Practice and Research Techniques, 2006. TAIC PART 2006. Proceedings, pp. 115–126 (2006b)
Zurück zum Zitat Zhang, J., Xu, C., Cheung, S.C.: Automatic generation of database instances for white-box testing. In: Proceedings of the 25th International Computer Software and Applications Conference on Invigorating Software Development, COMPSAC ’01, pp. 161–165. IEEE Computer Society, Washington, DC (2001) Zhang, J., Xu, C., Cheung, S.C.: Automatic generation of database instances for white-box testing. In: Proceedings of the 25th International Computer Software and Applications Conference on Invigorating Software Development, COMPSAC ’01, pp. 161–165. IEEE Computer Society, Washington, DC (2001)
Zurück zum Zitat Zhou, C., Frankl, P.: Inferential checking for mutants modifying database states. In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation 0:259–268, (2011) doi:10.1109/ICST.2011.63 Zhou, C., Frankl, P.: Inferential checking for mutants modifying database states. In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation 0:259–268, (2011) doi:10.​1109/​ICST.​2011.​63
Metadaten
Titel
Incremental test data generation for database queries
verfasst von
María José Suárez-Cabal
Claudio de la Riva
Javier Tuya
Raquel Blanco
Publikationsdatum
16.02.2017
Verlag
Springer US
Erschienen in
Automated Software Engineering / Ausgabe 4/2017
Print ISSN: 0928-8910
Elektronische ISSN: 1573-7535
DOI
https://doi.org/10.1007/s10515-017-0212-7

Weitere Artikel der Ausgabe 4/2017

Automated Software Engineering 4/2017 Zur Ausgabe