Skip to main content
Erschienen in: Software Quality Journal 3/2008

01.09.2008

Enhancing adaptive random testing for programs with high dimensional input domains or failure-unrelated parameters

verfasst von: Fei-Ching Kuo, Tsong Yueh Chen, Huai Liu, Wing Kwong Chan

Erschienen in: Software Quality Journal | Ausgabe 3/2008

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Adaptive random testing (ART), an enhancement of random testing (RT), aims to both randomly select and evenly spread test cases. Recently, it has been observed that the effectiveness of some ART algorithms may deteriorate as the number of program input parameters (dimensionality) increases. In this article, we analyse various problems of one ART algorithm, namely fixed-sized-candidate-set ART (FSCS-ART), in the high dimensional input domain setting, and study how FSCS-ART can be further enhanced to address these problems. We propose to add a filtering process of inputs into FSCS-ART to achieve a more even-spread of test cases and better failure detection effectiveness in high dimensional space. Our study shows that this solution, termed as FSCS-ART-FE, can improve FSCS-ART not only in the case of high dimensional space, but also in the case of having failure-unrelated parameters. Both cases are common in real life programs. Therefore, we recommend using FSCS-ART-FE instead of FSCS-ART whenever possible. Other ART algorithms may face similar problems as FSCS-ART; hence our study also brings insight into the improvement of other ART algorithms in high dimensional space.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Ammann, P. E., & Knight, J. C. (1988). Data diversity: An approach to software fault tolerance. IEEE Transactions on Computers, 37(4), 418–425.CrossRef Ammann, P. E., & Knight, J. C. (1988). Data diversity: An approach to software fault tolerance. IEEE Transactions on Computers, 37(4), 418–425.CrossRef
Zurück zum Zitat Bellman, R. (1957). Dynamic programming. New Jersey: Princeton University Press. Bellman, R. (1957). Dynamic programming. New Jersey: Princeton University Press.
Zurück zum Zitat Bird, D. L., & Munoz, C. U. (1983). Automatic generation of random self-checking test cases. IBM Systems Journal, 22(3), 229–245.CrossRef Bird, D. L., & Munoz, C. U. (1983). Automatic generation of random self-checking test cases. IBM Systems Journal, 22(3), 229–245.CrossRef
Zurück zum Zitat Bishop, P. G. (1993). The variation of software survival times for different operational input profiles. In Proceedings of the 23rd International Symposium on Fault-Tolerant Computing (FTCS-23) (pp. 98–107). IEEE Computer Society Press. Bishop, P. G. (1993). The variation of software survival times for different operational input profiles. In Proceedings of the 23rd International Symposium on Fault-Tolerant Computing (FTCS-23) (pp. 98–107). IEEE Computer Society Press.
Zurück zum Zitat Branicky, M. S., LaValle, S. M., Olson, K., & Yang, L. (2001). Quasi-randomized path planning. In Proceedings of the 2001 IEEE International Conference on Robotics and Automation (pp. 1481–1487). Branicky, M. S., LaValle, S. M., Olson, K., & Yang, L. (2001). Quasi-randomized path planning. In Proceedings of the 2001 IEEE International Conference on Robotics and Automation (pp. 1481–1487).
Zurück zum Zitat Chan, K. P., Chen, T. Y., & Towey, D. (2006). Restricted random testing: Adaptive random testing by exclusion. International Journal of Software Engineering and Knowledge Engineering, 16(4), 553–584.CrossRef Chan, K. P., Chen, T. Y., & Towey, D. (2006). Restricted random testing: Adaptive random testing by exclusion. International Journal of Software Engineering and Knowledge Engineering, 16(4), 553–584.CrossRef
Zurück zum Zitat Chen, T. Y., Kuo, F.-C., & Liu, H. (2007). On test case distributions of adaptive random testing. In Proceedings of the 19th International Conference on Software Engineering and Knowledge Engineering (SEKE’07) (pp. 141–144). Boston. Chen, T. Y., Kuo, F.-C., & Liu, H. (2007). On test case distributions of adaptive random testing. In Proceedings of the 19th International Conference on Software Engineering and Knowledge Engineering (SEKE’07) (pp. 141–144). Boston.
Zurück zum Zitat Chen, T. Y., Kuo, F.-C., & Merkel, R. (2006). On the statistical properties of testing effectiveness measures. Journal of Systems and Software, 79(5), 591–601.CrossRef Chen, T. Y., Kuo, F.-C., & Merkel, R. (2006). On the statistical properties of testing effectiveness measures. Journal of Systems and Software, 79(5), 591–601.CrossRef
Zurück zum Zitat Chen, T. Y., Kuo, F.-C., Merkel, R. G., & Ng, S. P. (2004). Mirror adaptive random testing. Information and Software Technology, 46(15), 1001–1010.CrossRef Chen, T. Y., Kuo, F.-C., Merkel, R. G., & Ng, S. P. (2004). Mirror adaptive random testing. Information and Software Technology, 46(15), 1001–1010.CrossRef
Zurück zum Zitat Chen, T. Y., Kuo, F.-C., & Zhou, Z. Q. (2005). On the Relationships between the distribution of failure-causing inputs and effectiveness of adaptive random testing. In Proceedings of the 17th International Conference on Software Engineering and Knowledge Engineering (SEKE’05) (pp. 306–311). Taipei, Taiwan. Chen, T. Y., Kuo, F.-C., & Zhou, Z. Q. (2005). On the Relationships between the distribution of failure-causing inputs and effectiveness of adaptive random testing. In Proceedings of the 17th International Conference on Software Engineering and Knowledge Engineering (SEKE’05) (pp. 306–311). Taipei, Taiwan.
Zurück zum Zitat Chen, T. Y., & Merkel, R. (2008). An upper bound on software testing effectiveness. ACM Transaction on Software Engineering Methodologies. Chen, T. Y., & Merkel, R. (2008). An upper bound on software testing effectiveness. ACM Transaction on Software Engineering Methodologies.
Zurück zum Zitat Chen, T. Y., Tse, T. H., & Yu, Y. T. (2001). Proportional sampling strategy: A compendium and some insights. Journal of Systems and Software, 58(1), 65–81.CrossRef Chen, T. Y., Tse, T. H., & Yu, Y. T. (2001). Proportional sampling strategy: A compendium and some insights. Journal of Systems and Software, 58(1), 65–81.CrossRef
Zurück zum Zitat Cobb, R., & Mills, H. D. (1990). Engineering software under statistical quality control. IEEE Software, 7(6), 45–54.CrossRef Cobb, R., & Mills, H. D. (1990). Engineering software under statistical quality control. IEEE Software, 7(6), 45–54.CrossRef
Zurück zum Zitat Dabóczi, T., Kollár, I., Simon, G., & Megyeri, T. (2003). Automatic testing of graphical user interfaces. In Proceedings of the 20th IEEE Instrumentation and Measurement Technology Conference 2003 (IMTC’03) (pp. 441–445). Vail, CO. Dabóczi, T., Kollár, I., Simon, G., & Megyeri, T. (2003). Automatic testing of graphical user interfaces. In Proceedings of the 20th IEEE Instrumentation and Measurement Technology Conference 2003 (IMTC’03) (pp. 441–445). Vail, CO.
Zurück zum Zitat Finelli, G. B. (1991). NASA software failure characterization experiments. Reliability Engineering and System Safety, 32(1–2), 155–169.CrossRef Finelli, G. B. (1991). NASA software failure characterization experiments. Reliability Engineering and System Safety, 32(1–2), 155–169.CrossRef
Zurück zum Zitat Forrester, J. E., & Miller, B. P. (2000). An empirical study of the robustness of Windows NT applications using random testing. In Proceedings of the 4th USENIX Windows Systems Symposium (pp. 59–68). Seattle. Forrester, J. E., & Miller, B. P. (2000). An empirical study of the robustness of Windows NT applications using random testing. In Proceedings of the 4th USENIX Windows Systems Symposium (pp. 59–68). Seattle.
Zurück zum Zitat Godefroid, P., Klarlund, N., & Sen, K. (2005). Dart: Directed automated random testing. In Proceedings of ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’05) (pp. 213–223). Godefroid, P., Klarlund, N., & Sen, K. (2005). Dart: Directed automated random testing. In Proceedings of ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’05) (pp. 213–223).
Zurück zum Zitat Hamlet, R. (2002). Random testing. In J. Marciniak (Ed.), Encyclopedia of software engineering (2nd edn.). Wiley. Hamlet, R. (2002). Random testing. In J. Marciniak (Ed.), Encyclopedia of software engineering (2nd edn.). Wiley.
Zurück zum Zitat Kuo, F.-C., Chen, T. Y., Liu, H., & Chan, W. K. (2007). Enhancing adaptive random testing in high dimensional input domain. In Proceedings of the 22nd Annual ACM Symposium on Applied Computing (SAC’07) (pp. 1467–1472). ACM Press. Kuo, F.-C., Chen, T. Y., Liu, H., & Chan, W. K. (2007). Enhancing adaptive random testing in high dimensional input domain. In Proceedings of the 22nd Annual ACM Symposium on Applied Computing (SAC’07) (pp. 1467–1472). ACM Press.
Zurück zum Zitat Mak, I. K. (1997). On the effectiveness of random testing. Master’s thesis, Department of Computer Science, University of Melbourne. Mak, I. K. (1997). On the effectiveness of random testing. Master’s thesis, Department of Computer Science, University of Melbourne.
Zurück zum Zitat Matsumoto, M., & Nishimura, T. (1998). Mersenne twister: A 623-dimensionally equidistributed uniform pseudo-random number generator. ACM Transactions on Modeling and Computer Simulation, 8(1), 3–30.MATHCrossRef Matsumoto, M., & Nishimura, T. (1998). Mersenne twister: A 623-dimensionally equidistributed uniform pseudo-random number generator. ACM Transactions on Modeling and Computer Simulation, 8(1), 3–30.MATHCrossRef
Zurück zum Zitat Mayer, J. (2005). Lattice-based adaptive random testing. In Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering (ASE’05) (pp. 333–336). New York: ACM Press. Mayer, J. (2005). Lattice-based adaptive random testing. In Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering (ASE’05) (pp. 333–336). New York: ACM Press.
Zurück zum Zitat Miller, B. P., Fredriksen, L., & So, B. (1990). An empirical study of the reliability of UNIX utilities. Communications of the ACM, 33(12), 32–44.CrossRef Miller, B. P., Fredriksen, L., & So, B. (1990). An empirical study of the reliability of UNIX utilities. Communications of the ACM, 33(12), 32–44.CrossRef
Zurück zum Zitat Miller, B. P., Koski, D., Lee, C. P., Maganty, V., Murthy, R., Natarajan, A., & Steidl, J. (1995). Fuzz revisited: A re-examination of the reliability of UNIX utilities and services. Tech. Rep. CS-TR-1995-1268, University of Wisconsin. Miller, B. P., Koski, D., Lee, C. P., Maganty, V., Murthy, R., Natarajan, A., & Steidl, J. (1995). Fuzz revisited: A re-examination of the reliability of UNIX utilities and services. Tech. Rep. CS-TR-1995-1268, University of Wisconsin.
Zurück zum Zitat Myers, G. J., Sandler, C., Badgett, T., & Thomas, T. M. (2004). The art of software testing (2nd edn.). New Jersey: Wiley. Myers, G. J., Sandler, C., Badgett, T., & Thomas, T. M. (2004). The art of software testing (2nd edn.). New Jersey: Wiley.
Zurück zum Zitat Regehr, J. (2005). Random testing of interrupt-driven software. In Proceedings of the 5th ACM International Conference on Embedded software (EMSOFT’05) (pp. 290–298). New York, NY: ACM Press. Regehr, J. (2005). Random testing of interrupt-driven software. In Proceedings of the 5th ACM International Conference on Embedded software (EMSOFT’05) (pp. 290–298). New York, NY: ACM Press.
Zurück zum Zitat Sen, K., Marinov, D., & Agha, G. (2005). CUTE: A concolic unit testing engine for C. In Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE-13) (pp. 263–272). New York, NY: ACM Press. Sen, K., Marinov, D., & Agha, G. (2005). CUTE: A concolic unit testing engine for C. In Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE-13) (pp. 263–272). New York, NY: ACM Press.
Zurück zum Zitat Slutz, D. (1998). Massive stochastic testing of SQL. In Proceedings of the 24th International Conference on Very Large Databases (VLDB’98) (pp. 618–622). Slutz, D. (1998). Massive stochastic testing of SQL. In Proceedings of the 24th International Conference on Very Large Databases (VLDB’98) (pp. 618–622).
Zurück zum Zitat Yoshikawa, T., Shimura, K., & Ozawa, T. (2003). Random program generator for Java JIT compiler test system. In Proceedings of the 3rd International Conference on Quality Software (QSIC’03) (pp. 20–24). IEEE Computer Society Press. Yoshikawa, T., Shimura, K., & Ozawa, T. (2003). Random program generator for Java JIT compiler test system. In Proceedings of the 3rd International Conference on Quality Software (QSIC’03) (pp. 20–24). IEEE Computer Society Press.
Metadaten
Titel
Enhancing adaptive random testing for programs with high dimensional input domains or failure-unrelated parameters
verfasst von
Fei-Ching Kuo
Tsong Yueh Chen
Huai Liu
Wing Kwong Chan
Publikationsdatum
01.09.2008
Verlag
Springer US
Erschienen in
Software Quality Journal / Ausgabe 3/2008
Print ISSN: 0963-9314
Elektronische ISSN: 1573-1367
DOI
https://doi.org/10.1007/s11219-008-9047-6

Weitere Artikel der Ausgabe 3/2008

Software Quality Journal 3/2008 Zur Ausgabe

EditorialNotes

In this issue

Premium Partner