skip to main content
survey

A Survey on Data-Flow Testing

Authors Info & Claims
Published:10 March 2017Publication History
Skip Abstract Section

Abstract

Data-flow testing (DFT) is a family of testing strategies designed to verify the interactions between each program variable’s definition and its uses. Such a test objective of interest is referred to as a def-use pair. DFT selects test data with respect to various test adequacy criteria (i.e., data-flow coverage criteria) to exercise each pair. The original conception of DFT was introduced by Herman in 1976. Since then, a number of studies have been conducted, both theoretically and empirically, to analyze DFT’s complexity and effectiveness. In the past four decades, DFT has been continuously concerned, and various approaches from different aspects are proposed to pursue automatic and efficient data-flow testing. This survey presents a detailed overview of data-flow testing, including challenges and approaches in enforcing and automating it: (1) it introduces the data-flow analysis techniques that are used to identify def-use pairs; (2) it classifies and discusses techniques for data-flow-based test data generation, such as search-based testing, random testing, collateral-coverage-based testing, symbolic-execution-based testing, and model-checking-based testing; (3) it discusses techniques for tracking data-flow coverage; (4) it presents several DFT applications, including software fault localization, web security testing, and specification consistency checking; and (5) it summarizes recent advances and discusses future research directions toward more practical data-flow testing.

References

  1. Hiralal Agrawal, Joseph R. Horgan, Saul London, and W. Eric Wong. 1995. Fault localization using execution slices and dataflow tests. In Proceedings of the Sixth International Symposium on Software Reliability Engineering (ISSRE’95). 143--151. Google ScholarGoogle ScholarCross RefCross Ref
  2. Alfred V. Aho, Ravi Sethi, and Jeffrey D. Ullman. 1986. Compilers: Principles, Techniques, and Tools. Addison-Wesley Longman Publishing Co., Boston, MA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Roger T. Alexander, Jeff Offutt, and Andreas Stefik. 2010. Testing coupling relationships in object-oriented programs. Softw. Test., Verif. Reliab. 20, 4 (2010), 291--327. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Frances E. Allen and John Cocke. 1976. A program data flow analysis procedure. Commun. ACM 19, 3 (1976), 137--147. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Nadia Alshahwan and Mark Harman. 2012. State aware test case regeneration for improving web application test suite coverage and fault detection. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA’12). 45--55. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Paul Ammann, A. Jefferson Offutt, and Hong Huang. 2003. Coverage criteria for logical expressions. In Proceedings of the International Symposium on Software Reliability Engineering. 99--107. Google ScholarGoogle ScholarCross RefCross Ref
  7. Paul Ammann and Jeff Offutt. 2008. Introduction to Software Testing. Cambridge University Press, New York, NY. Google ScholarGoogle ScholarCross RefCross Ref
  8. Saswat Anand, Edmund K. Burke, Tsong Yueh Chen, John A. Clark, Myra B. Cohen, Wolfgang Grieskamp, Mark Harman, Mary Jean Harrold, and Phil McMinn. 2013. An orchestrated survey of methodologies for automated software test case generation. Journal of Systems and Software 86, 8 (2013), 1978--2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Andrea Arcuri and Lionel C. Briand. 2011. Adaptive random testing: An illusion of effectiveness? In Proceedings of the 20th International Symposium on Software Testing and Analysis (ISSTA’11). 265--275. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Zeina Awedikian, Kamel Ayari, and Giuliano Antoniol. 2009. MC/DC automatic test input data generation. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’09). 1657--1664. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Arthur I. Baars, Mark Harman, Youssef Hassoun, Kiran Lakhotia, Phil McMinn, Paolo Tonella, and Tanja E. J. Vos. 2011. Symbolic search-based testing. In Proceedings of the 26th IEEE/ACM International Conference on Automated Software Engineering (ASE’11). 53--62. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Janvi Badlaney, Rohit Ghatol, and Romit Jadhwani. 2006. An Introduction to Data-Flow Testing. Technical Report NCSU CSC TR-2006-22. Department of Computer Science, North Carolina State University, Raleigh, NC 27695. Retrieved from http://people.eecs.ku.edu/∼aiedian/Teaching/Fa09/814/Lectures/intro-df-testing-1.pdf.Google ScholarGoogle Scholar
  13. Roberto Bagnara, Matthieu Carlier, Roberta Gori, and Arnaud Gotlieb. 2013. Symbolic path-oriented test data generation for floating-point programs. In 2013 IEEE 6th International Conference on Software Testing, Verification and Validation. 1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Thomas Ball and Sriram K. Rajamani. 2002. The SLAM project: Debugging system software via static analysis. In Proceedings of the 29th SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’02). 1--3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Mauro Baluda. 2011. Automatic structural testing with abstraction refinement and coarsening. In SIGSOFT/FSE’11 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-19) and ESEC’11: 13rd European Software Engineering Conference (ESEC-13). 400--403. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Mauro Baluda, Pietro Braione, Giovanni Denaro, and Mauro Pezzè. 2010. Structural coverage of feasible code. In Proceedings of the 5th Workshop on Automation of Software Test (AST’10). ACM, New York, NY, 59--66. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Mauro Baluda, Pietro Braione, Giovanni Denaro, and Mauro Pezzè. 2011. Enhancing structural software coverage by incrementally computing branch executability. Softw. Qual. J. 19, 4 (2011), 725--751. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Luciano Baresi, Pier Luca Lanzi, and Matteo Miraz. 2010. TestFul: An evolutionary test approach for java. In Proceedings of the 3rd International Conference on Software Testing, Verification and Validation (ICST’10). 185--194. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Luciano Baresi and Matteo Miraz. 2010. TestFul: Automatic unit-test generation for Java classes. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2 (ICSE’10). 281--284. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Nels E. Beckman, Aditya V. Nori, Sriram K. Rajamani, and Robert J. Simmons. 2008. Proofs from tests. In Proceedings of the ACM/SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’08). 3--14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. B. Beizer. 1990. Software Testing Techniques. International Thomson Computer Press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Dirk Beyer, Adam J. Chlipala, Thomas A. Henzinger, Ranjit Jhala, and Rupak Majumdar. 2004. Generating tests from counterexamples. In Proceedings of the 26th International Conference on Software Engineering (ICSE’04). IEEE Computer Society, 326--335. Google ScholarGoogle ScholarCross RefCross Ref
  23. Dirk Beyer, Thomas A. Henzinger, Ranjit Jhala, and Rupak Majumdar. 2007. The software model checker Blast: Applications to software engineering. Int. J. Softw. Tools Technol. Transf. 9, 5 (Oct. 2007), 505--525. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Dirk Beyer and M. Erkan Keremoglu. 2011. CPAchecker: A tool for configurable software verification. In CAV. 184--190. Google ScholarGoogle ScholarCross RefCross Ref
  25. David L. Bird and Carlos Urias Munoz. 1983. Automatic generation of random self-checking test cases. IBM Syst. J. 22, 3 (1983), 229--245. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Ilona Bluemke and Artur Rembiszewski. 2009. Dataflow approach to testing java programs. In Proceedings of the 4th International Conference on Dependability of Computer Systems, 2009 (DepCos-RELCOMEX’09). IEEE, 69--76. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Ilona Bluemke and Artur Rembiszewski. 2012. Dataflow testing of java programs with DFC. Adv. Softw. Eng. Techniques, LNCS 7054, 3 (2012), 215--228.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Rastislav Bodík, Rajiv Gupta, and Mary Lou Soffa. 1997. Refining data flow information using infeasible paths. SIGSOFT Softw. Eng. Notes 22, 6 (Nov. 1997), 361--377. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Jacob Burnim and Koushik Sen. 2008. Heuristics for scalable dynamic test generation. In ASE. 443--446. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Ugo A. Buy, Alessandro Orso, and Mauro Pezzè. 2000. Automated testing of classes. In ISSTA. 39--48. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Cristian Cadar, Daniel Dunbar, and Dawson R. Engler. 2008. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation. 209--224.Google ScholarGoogle Scholar
  32. Cristian Cadar, Vijay Ganesh, Peter M. Pawlowski, David L. Dill, and Dawson R. Engler. 2006. EXE: Automatically generating inputs of death. In Proceedings of the 13th ACM Conference on Computer and Communications Security (CCS’06). 322--335. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Cristian Cadar and Koushik Sen. 2013. Symbolic execution for software testing: Three decades later. Commun. ACM 56, 2 (2013), 82--90. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. M. L. Chaim. 1991. POKE-TOOL-A tool to support structural program testing based on data flow analysis. School of Electrical and Computer Engineering, University of Campinas, Campinas, SP, Brazil.Google ScholarGoogle Scholar
  35. Marcos L. Chaim, Anthony Accioly, Delano Medeiros Beder, and Marcelo Morandini. 2011. Evaluating instrumentation strategies by program simulation. IADIS Applied Computing.Google ScholarGoogle Scholar
  36. Marcos Lordello Chaim and Roberto Paulo Andrioli de Araujo. 2013a. An efficient bitwise algorithm for intra-procedural data-flow testing coverage. Inf. Process. Lett. 113, 8 (2013), 293--300. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Marcos Lordello Chaim and Roberto Paulo Andrioli de Araujo. 2013b. Proof of Correctness of the Bitwise Algorithm for Intra-procedural Data-Flow Testing Coverage. Technical Report PPgSI-001/2013. School of Arts, Sciences and Humanities, University of Sao Paulo. http://ppgsi.each.usp.br/arquivos/RelTec/PPgSI-001_2013.pdf.Google ScholarGoogle Scholar
  38. Sagar Chaki, Edmund M. Clarke, Alex Groce, Somesh Jha, and Helmut Veith. 2003. Modular verification of software components in C. In ICSE. 385--395.Google ScholarGoogle Scholar
  39. BRamkrishna Chatterjee and Barbara G. Ryder. 1999. Data-Flow-Based Testing of Object-Oriented Libraries. Technical Report DCS-TR-382. Rutgers University.Google ScholarGoogle Scholar
  40. T. Y. Chen. 2008. Adaptive random testing. In Proceedings of the 8th International Conference on Quality Software (QSIC’08). 443. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Ilinca Ciupa, Andreas Leitner, Manuel Oriol, and Bertrand Meyer. 2008. ARTOO: Adaptive random testing for object-oriented software. In Proceedings of the 30th International Conference on Software Engineering (ICSE’08). 71--80. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Edmund M. Clarke and E. Allen Emerson. 1981. Design and synthesis of synchronization skeletons using branching-time temporal logic. In Logics of Programs, Workshop. 52--71.Google ScholarGoogle Scholar
  43. Edmund M. Clarke Jr., Orna Grumberg, and Doron A. Peled. 1999. Model Checking. MIT Press, Cambridge, MA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Lori A. Clarke, Andy Podgurski, Debra J. Richardson, and Steven J. Zeil. 1989. A formal evaluation of data flow path selection criteria. IEEE Trans. Software Eng. 15, 11 (1989), 1318--1332. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Lee Copeland. 2003. A Practitioner’s Guide to Software Test Design. Artech House, Norwood, MA.Google ScholarGoogle Scholar
  46. Roberto Paulo Andrioli de Araujo and Marcos Lordello Chaim. 2014. Data-flow testing in the large. In Proceedings of the IEEE 7th International Conference on Software Testing, Verification and Validation (ICST’14). 81--90.Google ScholarGoogle Scholar
  47. Giovanni Denaro, Alessandra Gorla, and Mauro Pezzè. 2008. Contextual integration testing of classes. In Proceedings of the Fundamental Approaches to Software Engineering (FASE’08), Held as Part of the Joint European Conferences on Theory and Practice of Software (ETAPS’08). 246--260. Google ScholarGoogle ScholarCross RefCross Ref
  48. Giovanni Denaro, Alessandra Gorla, and Mauro Pezzè. 2009. DaTeC: Contextual data flow testing of java classes. In 31st International Conference on Software Engineering (ICSE’09). Companion Volume. 421--422.Google ScholarGoogle ScholarCross RefCross Ref
  49. Giovanni Denaro, Alessandro Margara, Mauro Pezzè, and Mattia Vivanti. 2015. Dynamic data flow testing of object oriented systems. In Proceedings of the 37th IEEE/ACM International Conference on Software Engineering (ICSE’15), Volume 1. 947--958. Google ScholarGoogle ScholarCross RefCross Ref
  50. Giovanni Denaro, Mauro Pezzè, and Mattia Vivanti. 2013. Quantifying the complexity of dataflow testing. In AST. 132--138. Google ScholarGoogle ScholarCross RefCross Ref
  51. Giovanni Denaro, Mauro Pezzè, and Mattia Vivanti. 2014. On the right objectives of data flow testing. In ICST. 71--80. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Mingjie Deng, Rong Chen, and Zhenjun Du. 2009. Automatic test data generation model by combining dataflow analysis with genetic algorithm. In Proceedings of the 2009 Joint Conferences on Pervasive Computing (JCPC’09). 429--434. Google ScholarGoogle ScholarCross RefCross Ref
  53. Evelyn Duesterwald, Rajiv Gupta, and Mary Lou Soffa. 1996. A demand-driven analyzer for data flow testing at the integration level. In Proceedings of the 18th International Conference on Software Engineering (ICSE’96). IEEE Computer Society, 575--584. Google ScholarGoogle ScholarCross RefCross Ref
  54. Evelyn Duesterwald, Rajiv Gupta, and Mary Lou Soffa. 1997. A practical framework for demand-driven interprocedural data flow analysis. ACM Trans. Program. Lang. Syst. 19, 6 (Nov. 1997), 992--1030. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Jon Edvardsson. 1999. A Survey on Automatic Test Data Generation. In Proceedings of the 2nd Conference on Computer Science and Engineering. 21--28.Google ScholarGoogle Scholar
  56. Lynn M. Foreman and Stuart H. Zweben. 1993. A study of the effectiveness of control and data flow testing strategies. J. Syst. Softw. 21, 3 (1993), 215--228. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. P. G. Frankl. 1987. The Use of Data Flow Information for the Selection and Evaluation of Software Test Data. Ph.D. Dissertation. University of New York, NY.Google ScholarGoogle Scholar
  58. Phyllis G. Frankl and Oleg Iakounenko. 1998. Further empirical studies of test effectiveness. In Proceedings of the ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT’98). 153--162. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. P. G. Frankl and S. N. Weiss. 1993. An experimental comparison of the effectiveness of branch testing and data flow testing. IEEE Trans. Softw. Eng. 19, 8 (Aug. 1993), 774--787. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Phyllis G. Frankl, Stewart N. Weiss, and Elaine J. Weyuker. 1985. Asset: A System to Select and Evaluate Tests. Courant Institute of Mathematical Sciences, New York University.Google ScholarGoogle Scholar
  61. Phyllis G. Frankl and Elaine J. Weyuker. 1985. A data flow testing tool. In Proceedings of the 2nd Conference on Software Development Tools, Techniques, and Alternatives. IEEE Computer Society Press, Los Alamitos, CA, 46--53.Google ScholarGoogle Scholar
  62. P. G. Frankl and E. J. Weyuker. 1988. An applicable family of data flow testing criteria. IEEE Trans. Softw. Eng. 14, 10 (Oct. 1988), 1483--1498. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Gordon Fraser and Andrea Arcuri. 2012. Sound empirical evidence in software testing. In Proceedings of the 34th International Conference on Software Engineering (ICSE’12). 178--188. Google ScholarGoogle ScholarCross RefCross Ref
  64. Gordon Fraser and Andrea Arcuri. 2013. Whole test suite generation. IEEE Trans. Software Eng. 39, 2 (2013), 276--291. Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Gordon Fraser, Franz Wotawa, and Paul Ammann. 2009. Testing with model checkers: A survey. Softw. Test., Verif. Reliab. 19, 3 (2009), 215--261. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Kamran Ghani and John A. Clark. 2009. Automatic test data generation for multiple condition and MCDC coverage. In The Proceedings of the 4th International Conference on Software Engineering Advances (ICSEA’09). 152--157. Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Ahmed S. Ghiduk. 2010. A new software data-flow testing approach via ant colony algorithms. Universal J. Comput. Sci. Eng. Technol. 1, 1 (Oct. 2010), 64--72.Google ScholarGoogle Scholar
  68. Ahmed S. Ghiduk, Mary Jean Harrold, and Moheb R. Girgis. 2007. Using genetic algorithms to aid test-data generation for data-flow coverage. In APSEC. 41--48. Google ScholarGoogle ScholarCross RefCross Ref
  69. Moheb R. Girgis. 1993. Using symbolic execution and data flow criteria to aid test data selection. Softw. Test., Verif. Reliab. 3, 2 (1993), 101--112. Google ScholarGoogle ScholarCross RefCross Ref
  70. Moheb R. Girgis. 2005. Automatic test data generation for data flow testing using a genetic algorithm. J. UCS 11, 6 (2005), 898--915.Google ScholarGoogle Scholar
  71. Moheb R. Girgis, Ahmed S. Ghiduk, and Eman H. Abd-elkawy. 2014. Automatic generation of data flow test paths using a genetic algorithm. Int. J. Comput. Appl. 89, 12 (March 2014), 29--36.Google ScholarGoogle Scholar
  72. Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: Directed automated random testing. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, New York, NY, 213--223. Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. Bhargav S. Gulavani, Thomas A. Henzinger, Yamini Kannan, Aditya V. Nori, and Sriram K. Rajamani. 2006. SYNERGY: A new algorithm for property checking. In SIGSOFT FSE. 117--127. Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Mark Harman, Yue Jia, and Yuanyuan Zhang. 2015. Achievements, open problems and challenges for search based software testing. In Proceedings of the 8th IEEE International Conference on Software Testing, Verification and Validation (ICST’15). 1--12. Google ScholarGoogle ScholarCross RefCross Ref
  75. Mark Harman, Sung Gon Kim, Kiran Lakhotia, Phil McMinn, and Shin Yoo. 2010. Optimizing for the number of tests generated in search based test data generation with an application to the oracle cost problem. In Proceedings of the 3rd International Conference on Software Testing, Verification and Validation (ICST’10), Workshop Proceedings. 182--191. Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. Mary Jean Harrold. 1994. Performing data flow testing in parallel. In Proceedings of the 8th International Symposium on Parallel Processing. 200--207. Google ScholarGoogle ScholarCross RefCross Ref
  77. M. Jean Harrold, Rajiv Gupta, and Mary Lou Soffa. 1993. A methodology for controlling the size of a test suite. ACM Trans. Softw. Eng. Methodol. 2, 3 (July 1993), 270--285. Google ScholarGoogle ScholarDigital LibraryDigital Library
  78. Mary Jean Harrold and Brian A Malloy. 1992. Data flow testing of parallelized code. In Proceedings of the Conference on Software Maintenance, 1992. IEEE, 272--281. Google ScholarGoogle ScholarCross RefCross Ref
  79. Mary Jean Harrold and Gregg Rothermel. 1994. Performing data flow testing on classes. In SIGSOFT FSE. 154--163. Google ScholarGoogle ScholarDigital LibraryDigital Library
  80. Mary Jean Harrold and Mary Lou Soffa. 1994. Efficient computation of interprocedural definition-use chains. ACM Trans. Program. Lang. Syst. 16, 2 (March 1994), 175--204. Google ScholarGoogle ScholarDigital LibraryDigital Library
  81. Mohammad Mahdi Hassan and James H. Andrews. 2013. Comparing multi-point stride coverage and dataflow coverage. In Proceedings of the 35th International Conference on Software Engineering (ICSE’13). 172--181. Google ScholarGoogle ScholarCross RefCross Ref
  82. Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar, and Grégoire Sutre. 2002. Lazy abstraction. In POPL. 58--70. Google ScholarGoogle ScholarDigital LibraryDigital Library
  83. P. M. Herman. 1976. A data flow analysis approach to program testing. Australian Comput. J. 8, 3 (1976), 92--96.Google ScholarGoogle Scholar
  84. John H. Holland. 1992. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence. MIT Press, Cambridge, MA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  85. Hyoung Seok Hong, Sung Deok Cha, Insup Lee, Oleg Sokolsky, and Hasan Ural. 2003. Data flow testing as model checking. In ICSE. 232--243.Google ScholarGoogle Scholar
  86. Hyoung Seok Hong, Young Gon Kim, Sung Deok Cha, Doo-Hwan Bae, and Hasan Ural. 2000. A test sequence selection method for statecharts. Softw. Test., Verif. Reliab. 10, 4 (2000), 203--227. Google ScholarGoogle ScholarCross RefCross Ref
  87. Hyoung Seok Hong and Hasan Ural. 2005. Dependence testing: Extending data flow testing with control dependence. In Proceedings of the 17th IFIP TC6/WG 6.1 International Conference on Testing of Communicating Systems (TestCom’05). 23--39.Google ScholarGoogle ScholarDigital LibraryDigital Library
  88. J. R. Horgan and London. 1992. ATAC: A data flow coverage testing tool for C. In Proceedings of the Symposium on Assessment of Quality Software Development Tools. 2--10. Google ScholarGoogle ScholarCross RefCross Ref
  89. J. C. Huang. 1979. Detection of data flow anomaly through program instrumentation. IEEE Trans. Softw. Eng. 5, 3 (1979), 226--236. Google ScholarGoogle ScholarDigital LibraryDigital Library
  90. Monica Hutchins, Herbert Foster, Tarak Goradia, and Thomas J. Ostrand. 1994. Experiments of the effectiveness of dataflow- and controlflow-based test adequacy criteria. In ICSE. 191--200.Google ScholarGoogle Scholar
  91. RTCA. 1992. DO-178b: Software considerations in airborne systems and equipment certification. Requirements and Technical Concepts for Aviation (December 1992).Google ScholarGoogle Scholar
  92. Kobi Inkumsah and Tao Xie. 2008. Improving structural testing of object-oriented programs via integrating evolutionary testing and symbolic execution. In Proceedings of the 23rd IEEE/ACM International Conference on Automated Software Engineering (ASE’08). 297--306. Google ScholarGoogle ScholarDigital LibraryDigital Library
  93. Laura Inozemtseva and Reid Holmes. 2014. Coverage is not strongly correlated with test suite effectiveness. In Proceedings of the 36th International Conference on Software Engineering (ICSE’14). 435--445. Google ScholarGoogle ScholarDigital LibraryDigital Library
  94. Mariam Kamkar, Peter Fritzson, and Nahid Shahmehri. 1993. Interprocedural dynamic slicing applied to interprocedural data flow testing. In ICSM. 386--395.Google ScholarGoogle Scholar
  95. Ken Kennedy. 1979. A Survey of Data Flow Analysis Techniques. IBM Thomas J. Watson Research Division.Google ScholarGoogle Scholar
  96. Abdelaziz Khamis, Reem Bahgat, and Rana Abdelaziz. 2011. Automatic test data generation using data flow information. Dogus Univ. J. 2 (2011), 140--153.Google ScholarGoogle ScholarCross RefCross Ref
  97. Arunkumar Khannur. 2011. Software Testing - Techniques and Applications. Pearson Publications.Google ScholarGoogle Scholar
  98. James C. King. 1976. Symbolic execution and program testing. Commun. ACM 19, 7 (July 1976), 385--394. Google ScholarGoogle ScholarDigital LibraryDigital Library
  99. Kiran Lakhotia, Mark Harman, and Phil McMinn. 2007. A multi-objective approach to search-based test data generation. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’07). 1098--1105. Google ScholarGoogle ScholarDigital LibraryDigital Library
  100. Kiran Lakhotia, Phil McMinn, and Mark Harman. 2009. Automated test data generation for coverage: Haven’t we solved this problem yet? In Proceedings of the 2009 Testing: Academic and Industrial Conference - Practice and Research Techniques. IEEE Computer Society, 95--104.Google ScholarGoogle ScholarDigital LibraryDigital Library
  101. Kiran Lakhotia, Nikolai Tillmann, Mark Harman, and Jonathan de Halleux. 2010. FloPSy - Search-based floating point constraint solving for symbolic execution. In Proceedings of the 22nd IFIP WG 6.1 International Conference on Testing Software and Systems (ICTSS’10). 142--157.Google ScholarGoogle ScholarCross RefCross Ref
  102. Janusz W. Laski and Bogdan Korel. 1983. A data flow oriented program testing strategy. IEEE Trans. Software Eng. 9, 3 (1983), 347--354. Google ScholarGoogle ScholarDigital LibraryDigital Library
  103. Thomas Lengauer and Robert Endre Tarjan. 1979. A fast algorithm for finding dominators in a flowgraph. ACM Trans. Program. Lang. Syst. 1, 1 (1979), 121--141. Google ScholarGoogle ScholarDigital LibraryDigital Library
  104. You Li, Zhendong Su, Linzhang Wang, and Xuandong Li. 2013. Steering symbolic execution to less traveled paths. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages 8 Applications (OOPSLA’13), part of SPLASH 2013. 19--32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  105. Konstantinos Liaskos and Marc Roper. 2007. Automatic test-data generation: An immunological approach. In Proceedings of Testing: Accademic and Industrial Conference - Practice and Research Techniques (TAIC PART’07). 77--81.Google ScholarGoogle ScholarCross RefCross Ref
  106. Konstantinos Liaskos and Marc Roper. 2008. Hybridizing evolutionary testing with artificial immune systems and local search. In Proceedings of the 1st International Conference on Software Testing Verification and Validation (ICST’08), Workshops Proceedings. 211--220. Google ScholarGoogle ScholarDigital LibraryDigital Library
  107. Konstantinos Liaskos, Marc Roper, and Murray Wood. 2007. Investigating data-flow coverage of classes using evolutionary algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’07). 1140. Google ScholarGoogle ScholarDigital LibraryDigital Library
  108. Yu Lin, Xucheng Tang, Yuting Chen, and Jianjun Zhao. 2009. A divergence-oriented approach to adaptive random testing of java programs. In Proceedings of the 24th IEEE/ACM International Conference on Automated Software Engineering (ASE’09). 221--232. Google ScholarGoogle ScholarDigital LibraryDigital Library
  109. Chien-Hung Liu, David Chenho Kung, and Pei Hsia. 2000. Object-based data flow testing of web applications. In Proceedings of the 1st Asia-Pacific Conference on Quality Software, 2000. IEEE, 7--16.Google ScholarGoogle Scholar
  110. Kin-Keung Ma, Yit Phang Khoo, Jeffrey S. Foster, and Michael Hicks. 2011. Directed symbolic execution. In SAS. 95--111. Google ScholarGoogle ScholarCross RefCross Ref
  111. Nicos Malevris and Derek F. Yates. 2006. The collateral coverage of data flow criteria when branch testing. Inf. Softw. Technol. 48, 8 (2006), 676--686. Google ScholarGoogle ScholarCross RefCross Ref
  112. Martina Marré and Antonia Bertolino. 1996. Unconstrained duas and their use in achieving all-uses coverage. In Proceedings of the 1996 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’96). ACM, New York, NY, 147--157.Google ScholarGoogle ScholarDigital LibraryDigital Library
  113. Martina Marré and Antonia Bertolino. 2003. Using spanning sets for coverage testing. IEEE Trans. Softw. Eng. 29, 11 (Nov. 2003), 974--984. Google ScholarGoogle ScholarDigital LibraryDigital Library
  114. Vincenzo Martena, Alessandro Orso, and Mauro Pezzè. 2002. Interclass testing of object oriented software. In Proceedings of the 8th International Conference on Engineering of Complex Computer Systems (ICECCS’02). 135--144. Google ScholarGoogle ScholarCross RefCross Ref
  115. Phil McMinn. 2004. Search-based software test data generation: A survey. Softw. Test. Verif. Reliab. 14, 2 (2004), 105--156. Google ScholarGoogle ScholarDigital LibraryDigital Library
  116. Lijun Mei, W. K. Chan, and T. H. Tse. 2008. Data flow testing of service-oriented workflow applications. In Proceedings of the ACM/IEEE 30th International Conference on Software Engineering, 2008 (ICSE’08). IEEE, 371--380. Google ScholarGoogle ScholarDigital LibraryDigital Library
  117. Lijun Mei, W. K. Chan, and T. H. Tse. 2009. Data flow testing of service choreography. In Proceedings of the the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering. ACM, 151--160. Google ScholarGoogle ScholarDigital LibraryDigital Library
  118. Ettore Merlo and Giuliano Antoniol. 1999. A static measure of a subset of intra-procedural data flow testing coverage based on node coverage. In CASCON. 7.Google ScholarGoogle Scholar
  119. Zbigniew Michalewicz. 1994. Genetic Algorithms + Data Structures = Evolution Programs (2nd extended ed.). Springer-Verlag New York, New York, NY.Google ScholarGoogle ScholarDigital LibraryDigital Library
  120. Matteo Miraz. 2010. Evolutionary Testing of Stateful Systems: A Holistic Approach. Ph.D. Dissertation. Politecnico di Milano.Google ScholarGoogle Scholar
  121. Jonathan Misurda, Jim Clause, Juliya Reed, Bruce R. Childers, and Mary Lou Soffa. 2005a. Jazz: A tool for demand-driven structural testing. In Compiler Construction. Springer, 242--245. Google ScholarGoogle ScholarDigital LibraryDigital Library
  122. Jonathan Misurda, James A. Clause, Juliya L. Reed, Bruce R. Childers, and Mary Lou Soffa. 2005b. Demand-driven structural testing with dynamic instrumentation. In Proceedings of the 27th International Conference on Software Engineering (ICSE’05). 156--165.Google ScholarGoogle ScholarDigital LibraryDigital Library
  123. Akbar Siami Namin and James H. Andrews. 2009. The influence of size and coverage on test suite effectiveness. In Proceedings of the 18th International Symposium on Software Testing and Analysis (ISSTA’09). 57--68. Google ScholarGoogle ScholarDigital LibraryDigital Library
  124. Narmada Nayak and Durga Prasad Mohapatra. 2010. Automatic test data generation for data flow testing using particle swarm optimization. In IC3 (2). 1--12. Google ScholarGoogle ScholarCross RefCross Ref
  125. A. Jefferson Offutt, Zhenyi Jin, and Jie Pan. 1999. The dynamic domain reduction procedure for test data generation. Softw. Pract. Exper. 29, 2 (1999), 167--193. Google ScholarGoogle ScholarDigital LibraryDigital Library
  126. Norbert Oster. 2005. Automated generation and evaluation of dataflow-based test data for object-oriented software. In QoSA/SOQUA. 212--226. Google ScholarGoogle ScholarDigital LibraryDigital Library
  127. Thomas J. Ostrand and Elaine J. Weyuker. 1991a. Data flow-based test adequacy analysis for languages with pointers. In Proceedings of the Symposium on Testing, Analysis, and Verification. 74--86.Google ScholarGoogle Scholar
  128. Thomas J. Ostrand and Elaine J. Weyuker. 1991b. Data flow-based test adequacy analysis for languages with pointers. In Proceedings of the Symposium on Testing, Analysis, and Verification. ACM, 74--86. Google ScholarGoogle ScholarDigital LibraryDigital Library
  129. Carlos Pacheco, Shuvendu K. Lahiri, Michael D. Ernst, and Thomas Ball. 2007. Feedback-directed random test generation. In Proceedings of the 29th International Conference on Software Engineering (ICSE’07). 75--84. Google ScholarGoogle ScholarDigital LibraryDigital Library
  130. H. D. Pande, W. A. Landi, and B. G. Ryder. 1994. Interprocedural def-use associations for C systems with single level pointers. IEEE Trans. Softw. Eng. 20, 5 (May 1994), 385--403. Google ScholarGoogle ScholarDigital LibraryDigital Library
  131. Mauro Pezzè and Michal Young. 2007. Software Testing and Analysis: Process, Principles and Techniques. Wiley.Google ScholarGoogle Scholar
  132. Yu Qi, David Kung, and Eric Wong. 2006. An agent-based data-flow testing approach for Web applications. Inf. Softw. Technol. 48, 12 (2006), 1159--1171. Google ScholarGoogle ScholarDigital LibraryDigital Library
  133. Sandra Rapps and Elaine J. Weyuker. 1982. Data flow analysis techniques for test data selection. In Proceedings of the 6th International Conference on Software Engineering (ICSE’82). IEEE Computer Society Press, Los Alamitos, CA, 272--278.Google ScholarGoogle Scholar
  134. Sandra Rapps and Elaine J. Weyuker. 1985. Selecting software test data using data flow information. IEEE Trans. Software Eng. 11, 4 (1985), 367--375. Google ScholarGoogle ScholarDigital LibraryDigital Library
  135. Torsten Robschink and Gregor Snelting. 2002. Efficient path conditions in dependence graphs. In Proceedings of the 24th International Conference on Software Engineering (ICSE’02). ACM, New York, NY, 478--488. Google ScholarGoogle ScholarCross RefCross Ref
  136. Raul Santelices and Mary Jean Harrold. 2007. Efficiently monitoring data-flow test coverage. In Proceedings of the 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE’07). ACM, New York, NY, 343--352. Google ScholarGoogle ScholarDigital LibraryDigital Library
  137. Raúl A. Santelices, James A. Jones, Yanbing Yu, and Mary Jean Harrold. 2009. Lightweight fault-localization using multiple coverage types. In Proceedings of the 31st International Conference on Software Engineering, (ICSE’09). 56--66. Google ScholarGoogle ScholarDigital LibraryDigital Library
  138. Raúl A. Santelices, Saurabh Sinha, and Mary Jean Harrold. 2006. Subsumption of program entities for efficient coverage and monitoring. In Proceedings of the 3rd International Workshop on Software Quality Assurance (SOQUA’06). 2--5. Google ScholarGoogle ScholarDigital LibraryDigital Library
  139. Koushik Sen, Darko Marinov, and Gul Agha. 2005. CUTE: A concolic unit testing engine for C. In Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, New York, NY, 263--272. Google ScholarGoogle ScholarDigital LibraryDigital Library
  140. Sanjay Singla, Dharminder Kumar, H. M. Rai, and Priti Singla. 2011a. A hybrid PSO approach to automate test data generation for data flow coverage with dominance concepts. J. Adv. Sci. Technol. 37 (2011), 15--26.Google ScholarGoogle Scholar
  141. Sanjay Singla, Priti Singla, and H. M. Rai. 2011b. An automatic test data generation for data flow coverage using soft computing approach. IJRRCS 2, 2 (2011), 265--270.Google ScholarGoogle Scholar
  142. Amie L. Souter and Lori L. Pollock. 2003. The construction of contextual def-use associations for object-oriented systems. IEEE Trans. Software Eng. 29, 11 (2003), 1005--1018. Google ScholarGoogle ScholarDigital LibraryDigital Library
  143. Ting Su, Zhoulai Fu, Geguang Pu, Jifeng He, and Zhendong Su. 2015. Combining symbolic execution and model checking for data flow testing. In Proceedings of the 37th IEEE/ACM International Conference on Software Engineering (ICSE’15), Volume 1. 654--665. Google ScholarGoogle ScholarCross RefCross Ref
  144. Ting Su, Geguang Pu, Bin Fang, Jifeng He, Jun Yan, Siyuan Jiang, and Jianjun Zhao. 2014. Automated coverage-driven test data generation using dynamic symbolic execution. In Proceedings of the 8th International Conference on Software Security and Reliability (SERE’14). 98--107. Google ScholarGoogle ScholarDigital LibraryDigital Library
  145. Tao Sun, Zheng Wang, Geguang Pu, Xiao Yu, Zongyan Qiu, and Bin Gu. 2009. Towards scalable compositional test generation. In QSIC. 353--358. Google ScholarGoogle ScholarDigital LibraryDigital Library
  146. Paolo Tonella. 2004. Evolutionary testing of classes. In Proceedings of the ACM/SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’04). 119--128. Google ScholarGoogle ScholarDigital LibraryDigital Library
  147. Hasan Ural, Kassem Saleh, and Alan W. Williams. 2000. Test generation based on control and data dependencies within system specifications in SDL. Comput. Commun. 23, 7 (2000), 609--627. Google ScholarGoogle ScholarDigital LibraryDigital Library
  148. Auri Marcelo Rizzo Vincenzi, José Carlos Maldonado, W. Eric Wong, and Márcio Eduardo Delamaro. 2005. Coverage testing of java programs and components. Sci. Comput. Program. 56, 1 (2005), 211--230.Google ScholarGoogle ScholarDigital LibraryDigital Library
  149. Mattia Vivanti. 2014. Dynamic data-flow testing. In Proceedings of the 36th International Conference on Software Engineering (ICSE’14), Companion Proceedings. 682--685. Google ScholarGoogle ScholarDigital LibraryDigital Library
  150. Mattia Vivanti, Andre Mis, Alessandra Gorla, and Gordon Fraser. 2013. Search-based data-flow test generation. In ISSRE. 370--379. Google ScholarGoogle ScholarCross RefCross Ref
  151. Chen-Wei Wang and Alessandra Cavarra. 2009. Checking model consistency using data-flow testing. In Proceedings of the Asia-Pacific Software Engineering Conference, 2009 (APSEC’09). IEEE, 414--421. Google ScholarGoogle ScholarDigital LibraryDigital Library
  152. Zheng Wang, Xiao Yu, Tao Sun, Geguang Pu, Zuohua Ding, and Jueliang Hu. 2009. Test data generation for derived types in C program. In TASE. 155--162. Google ScholarGoogle ScholarDigital LibraryDigital Library
  153. Joachim Wegener, André Baresel, and Harmen Sthamer. 2001. Evolutionary test environment for automatic structural testing. Inf. Softw. Technol. 43, 14 (2001), 841--854. Google ScholarGoogle ScholarCross RefCross Ref
  154. Elaine J. Weyuker. 1990. The cost of data flow testing: An empirical study. IEEE Trans. Software Eng. 16, 2 (1990), 121--128. Google ScholarGoogle ScholarDigital LibraryDigital Library
  155. Elaine J. Weyuker. 1993. More experience with data flow testing. IEEE Trans. Software Eng. 19, 9 (1993), 912--919. Google ScholarGoogle ScholarDigital LibraryDigital Library
  156. Qian Yang, J. Jenny Li, and David M. Weiss. 2009. A survey of coverage-based testing tools. Comput. J. 52, 5 (2009), 589--597. Google ScholarGoogle ScholarDigital LibraryDigital Library
  157. Xiao Yu, Shuai Sun, Geguang Pu, Siyuan Jiang, and Zheng Wang. 2011. A parallel approach to concolic testing with low-cost synchronization. Electr. Notes Theor. Comput. Sci. 274 (2011), 83--96. Google ScholarGoogle ScholarDigital LibraryDigital Library
  158. Cristian Zamfir and George Candea. 2010. Execution synthesis: A technique for automated software debugging. In EuroSys. 321--334. Google ScholarGoogle ScholarDigital LibraryDigital Library
  159. Jianjun Zhao. 2003. Data-flow-based unit testing of aspect-oriented programs. In Proceedings of the 27th Annual International Computer Software and Applications Conference, 2003 (COMPSAC’03). IEEE, 188--197.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A Survey on Data-Flow Testing

              Recommendations

              Comments

              Login options

              Check if you have access through your login credentials or your institution to get full access on this article.

              Sign in

              Full Access

              • Published in

                cover image ACM Computing Surveys
                ACM Computing Surveys  Volume 50, Issue 1
                January 2018
                588 pages
                ISSN:0360-0300
                EISSN:1557-7341
                DOI:10.1145/3058791
                • Editor:
                • Sartaj Sahni
                Issue’s Table of Contents

                Copyright © 2017 ACM

                Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

                Publisher

                Association for Computing Machinery

                New York, NY, United States

                Publication History

                • Published: 10 March 2017
                • Accepted: 1 November 2016
                • Revised: 1 September 2016
                • Received: 1 April 2015
                Published in csur Volume 50, Issue 1

                Permissions

                Request permissions about this article.

                Request Permissions

                Check for updates

                Qualifiers

                • survey
                • Research
                • Refereed

              PDF Format

              View or Download as a PDF file.

              PDF

              eReader

              View online with eReader.

              eReader