Abstract
Data-flow testing (DFT) is a family of testing strategies designed to verify the interactions between each program variable’s definition and its uses. Such a test objective of interest is referred to as a def-use pair. DFT selects test data with respect to various test adequacy criteria (i.e., data-flow coverage criteria) to exercise each pair. The original conception of DFT was introduced by Herman in 1976. Since then, a number of studies have been conducted, both theoretically and empirically, to analyze DFT’s complexity and effectiveness. In the past four decades, DFT has been continuously concerned, and various approaches from different aspects are proposed to pursue automatic and efficient data-flow testing. This survey presents a detailed overview of data-flow testing, including challenges and approaches in enforcing and automating it: (1) it introduces the data-flow analysis techniques that are used to identify def-use pairs; (2) it classifies and discusses techniques for data-flow-based test data generation, such as search-based testing, random testing, collateral-coverage-based testing, symbolic-execution-based testing, and model-checking-based testing; (3) it discusses techniques for tracking data-flow coverage; (4) it presents several DFT applications, including software fault localization, web security testing, and specification consistency checking; and (5) it summarizes recent advances and discusses future research directions toward more practical data-flow testing.
- Hiralal Agrawal, Joseph R. Horgan, Saul London, and W. Eric Wong. 1995. Fault localization using execution slices and dataflow tests. In Proceedings of the Sixth International Symposium on Software Reliability Engineering (ISSRE’95). 143--151. Google ScholarCross Ref
- Alfred V. Aho, Ravi Sethi, and Jeffrey D. Ullman. 1986. Compilers: Principles, Techniques, and Tools. Addison-Wesley Longman Publishing Co., Boston, MA.Google ScholarDigital Library
- Roger T. Alexander, Jeff Offutt, and Andreas Stefik. 2010. Testing coupling relationships in object-oriented programs. Softw. Test., Verif. Reliab. 20, 4 (2010), 291--327. Google ScholarDigital Library
- Frances E. Allen and John Cocke. 1976. A program data flow analysis procedure. Commun. ACM 19, 3 (1976), 137--147. Google ScholarDigital Library
- Nadia Alshahwan and Mark Harman. 2012. State aware test case regeneration for improving web application test suite coverage and fault detection. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA’12). 45--55. Google ScholarDigital Library
- Paul Ammann, A. Jefferson Offutt, and Hong Huang. 2003. Coverage criteria for logical expressions. In Proceedings of the International Symposium on Software Reliability Engineering. 99--107. Google ScholarCross Ref
- Paul Ammann and Jeff Offutt. 2008. Introduction to Software Testing. Cambridge University Press, New York, NY. Google ScholarCross Ref
- Saswat Anand, Edmund K. Burke, Tsong Yueh Chen, John A. Clark, Myra B. Cohen, Wolfgang Grieskamp, Mark Harman, Mary Jean Harrold, and Phil McMinn. 2013. An orchestrated survey of methodologies for automated software test case generation. Journal of Systems and Software 86, 8 (2013), 1978--2001. Google ScholarDigital Library
- Andrea Arcuri and Lionel C. Briand. 2011. Adaptive random testing: An illusion of effectiveness? In Proceedings of the 20th International Symposium on Software Testing and Analysis (ISSTA’11). 265--275. Google ScholarDigital Library
- Zeina Awedikian, Kamel Ayari, and Giuliano Antoniol. 2009. MC/DC automatic test input data generation. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’09). 1657--1664. Google ScholarDigital Library
- Arthur I. Baars, Mark Harman, Youssef Hassoun, Kiran Lakhotia, Phil McMinn, Paolo Tonella, and Tanja E. J. Vos. 2011. Symbolic search-based testing. In Proceedings of the 26th IEEE/ACM International Conference on Automated Software Engineering (ASE’11). 53--62. Google ScholarDigital Library
- Janvi Badlaney, Rohit Ghatol, and Romit Jadhwani. 2006. An Introduction to Data-Flow Testing. Technical Report NCSU CSC TR-2006-22. Department of Computer Science, North Carolina State University, Raleigh, NC 27695. Retrieved from http://people.eecs.ku.edu/∼aiedian/Teaching/Fa09/814/Lectures/intro-df-testing-1.pdf.Google Scholar
- Roberto Bagnara, Matthieu Carlier, Roberta Gori, and Arnaud Gotlieb. 2013. Symbolic path-oriented test data generation for floating-point programs. In 2013 IEEE 6th International Conference on Software Testing, Verification and Validation. 1--10. Google ScholarDigital Library
- Thomas Ball and Sriram K. Rajamani. 2002. The SLAM project: Debugging system software via static analysis. In Proceedings of the 29th SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’02). 1--3. Google ScholarDigital Library
- Mauro Baluda. 2011. Automatic structural testing with abstraction refinement and coarsening. In SIGSOFT/FSE’11 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-19) and ESEC’11: 13rd European Software Engineering Conference (ESEC-13). 400--403. Google ScholarDigital Library
- Mauro Baluda, Pietro Braione, Giovanni Denaro, and Mauro Pezzè. 2010. Structural coverage of feasible code. In Proceedings of the 5th Workshop on Automation of Software Test (AST’10). ACM, New York, NY, 59--66. Google ScholarDigital Library
- Mauro Baluda, Pietro Braione, Giovanni Denaro, and Mauro Pezzè. 2011. Enhancing structural software coverage by incrementally computing branch executability. Softw. Qual. J. 19, 4 (2011), 725--751. Google ScholarDigital Library
- Luciano Baresi, Pier Luca Lanzi, and Matteo Miraz. 2010. TestFul: An evolutionary test approach for java. In Proceedings of the 3rd International Conference on Software Testing, Verification and Validation (ICST’10). 185--194. Google ScholarDigital Library
- Luciano Baresi and Matteo Miraz. 2010. TestFul: Automatic unit-test generation for Java classes. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2 (ICSE’10). 281--284. Google ScholarDigital Library
- Nels E. Beckman, Aditya V. Nori, Sriram K. Rajamani, and Robert J. Simmons. 2008. Proofs from tests. In Proceedings of the ACM/SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’08). 3--14. Google ScholarDigital Library
- B. Beizer. 1990. Software Testing Techniques. International Thomson Computer Press.Google ScholarDigital Library
- Dirk Beyer, Adam J. Chlipala, Thomas A. Henzinger, Ranjit Jhala, and Rupak Majumdar. 2004. Generating tests from counterexamples. In Proceedings of the 26th International Conference on Software Engineering (ICSE’04). IEEE Computer Society, 326--335. Google ScholarCross Ref
- Dirk Beyer, Thomas A. Henzinger, Ranjit Jhala, and Rupak Majumdar. 2007. The software model checker Blast: Applications to software engineering. Int. J. Softw. Tools Technol. Transf. 9, 5 (Oct. 2007), 505--525. Google ScholarDigital Library
- Dirk Beyer and M. Erkan Keremoglu. 2011. CPAchecker: A tool for configurable software verification. In CAV. 184--190. Google ScholarCross Ref
- David L. Bird and Carlos Urias Munoz. 1983. Automatic generation of random self-checking test cases. IBM Syst. J. 22, 3 (1983), 229--245. Google ScholarDigital Library
- Ilona Bluemke and Artur Rembiszewski. 2009. Dataflow approach to testing java programs. In Proceedings of the 4th International Conference on Dependability of Computer Systems, 2009 (DepCos-RELCOMEX’09). IEEE, 69--76. Google ScholarDigital Library
- Ilona Bluemke and Artur Rembiszewski. 2012. Dataflow testing of java programs with DFC. Adv. Softw. Eng. Techniques, LNCS 7054, 3 (2012), 215--228.Google ScholarDigital Library
- Rastislav Bodík, Rajiv Gupta, and Mary Lou Soffa. 1997. Refining data flow information using infeasible paths. SIGSOFT Softw. Eng. Notes 22, 6 (Nov. 1997), 361--377. Google ScholarDigital Library
- Jacob Burnim and Koushik Sen. 2008. Heuristics for scalable dynamic test generation. In ASE. 443--446. Google ScholarDigital Library
- Ugo A. Buy, Alessandro Orso, and Mauro Pezzè. 2000. Automated testing of classes. In ISSTA. 39--48. Google ScholarDigital Library
- Cristian Cadar, Daniel Dunbar, and Dawson R. Engler. 2008. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation. 209--224.Google Scholar
- Cristian Cadar, Vijay Ganesh, Peter M. Pawlowski, David L. Dill, and Dawson R. Engler. 2006. EXE: Automatically generating inputs of death. In Proceedings of the 13th ACM Conference on Computer and Communications Security (CCS’06). 322--335. Google ScholarDigital Library
- Cristian Cadar and Koushik Sen. 2013. Symbolic execution for software testing: Three decades later. Commun. ACM 56, 2 (2013), 82--90. Google ScholarDigital Library
- M. L. Chaim. 1991. POKE-TOOL-A tool to support structural program testing based on data flow analysis. School of Electrical and Computer Engineering, University of Campinas, Campinas, SP, Brazil.Google Scholar
- Marcos L. Chaim, Anthony Accioly, Delano Medeiros Beder, and Marcelo Morandini. 2011. Evaluating instrumentation strategies by program simulation. IADIS Applied Computing.Google Scholar
- Marcos Lordello Chaim and Roberto Paulo Andrioli de Araujo. 2013a. An efficient bitwise algorithm for intra-procedural data-flow testing coverage. Inf. Process. Lett. 113, 8 (2013), 293--300. Google ScholarDigital Library
- Marcos Lordello Chaim and Roberto Paulo Andrioli de Araujo. 2013b. Proof of Correctness of the Bitwise Algorithm for Intra-procedural Data-Flow Testing Coverage. Technical Report PPgSI-001/2013. School of Arts, Sciences and Humanities, University of Sao Paulo. http://ppgsi.each.usp.br/arquivos/RelTec/PPgSI-001_2013.pdf.Google Scholar
- Sagar Chaki, Edmund M. Clarke, Alex Groce, Somesh Jha, and Helmut Veith. 2003. Modular verification of software components in C. In ICSE. 385--395.Google Scholar
- BRamkrishna Chatterjee and Barbara G. Ryder. 1999. Data-Flow-Based Testing of Object-Oriented Libraries. Technical Report DCS-TR-382. Rutgers University.Google Scholar
- T. Y. Chen. 2008. Adaptive random testing. In Proceedings of the 8th International Conference on Quality Software (QSIC’08). 443. Google ScholarDigital Library
- Ilinca Ciupa, Andreas Leitner, Manuel Oriol, and Bertrand Meyer. 2008. ARTOO: Adaptive random testing for object-oriented software. In Proceedings of the 30th International Conference on Software Engineering (ICSE’08). 71--80. Google ScholarDigital Library
- Edmund M. Clarke and E. Allen Emerson. 1981. Design and synthesis of synchronization skeletons using branching-time temporal logic. In Logics of Programs, Workshop. 52--71.Google Scholar
- Edmund M. Clarke Jr., Orna Grumberg, and Doron A. Peled. 1999. Model Checking. MIT Press, Cambridge, MA.Google ScholarDigital Library
- Lori A. Clarke, Andy Podgurski, Debra J. Richardson, and Steven J. Zeil. 1989. A formal evaluation of data flow path selection criteria. IEEE Trans. Software Eng. 15, 11 (1989), 1318--1332. Google ScholarDigital Library
- Lee Copeland. 2003. A Practitioner’s Guide to Software Test Design. Artech House, Norwood, MA.Google Scholar
- Roberto Paulo Andrioli de Araujo and Marcos Lordello Chaim. 2014. Data-flow testing in the large. In Proceedings of the IEEE 7th International Conference on Software Testing, Verification and Validation (ICST’14). 81--90.Google Scholar
- Giovanni Denaro, Alessandra Gorla, and Mauro Pezzè. 2008. Contextual integration testing of classes. In Proceedings of the Fundamental Approaches to Software Engineering (FASE’08), Held as Part of the Joint European Conferences on Theory and Practice of Software (ETAPS’08). 246--260. Google ScholarCross Ref
- Giovanni Denaro, Alessandra Gorla, and Mauro Pezzè. 2009. DaTeC: Contextual data flow testing of java classes. In 31st International Conference on Software Engineering (ICSE’09). Companion Volume. 421--422.Google ScholarCross Ref
- Giovanni Denaro, Alessandro Margara, Mauro Pezzè, and Mattia Vivanti. 2015. Dynamic data flow testing of object oriented systems. In Proceedings of the 37th IEEE/ACM International Conference on Software Engineering (ICSE’15), Volume 1. 947--958. Google ScholarCross Ref
- Giovanni Denaro, Mauro Pezzè, and Mattia Vivanti. 2013. Quantifying the complexity of dataflow testing. In AST. 132--138. Google ScholarCross Ref
- Giovanni Denaro, Mauro Pezzè, and Mattia Vivanti. 2014. On the right objectives of data flow testing. In ICST. 71--80. Google ScholarDigital Library
- Mingjie Deng, Rong Chen, and Zhenjun Du. 2009. Automatic test data generation model by combining dataflow analysis with genetic algorithm. In Proceedings of the 2009 Joint Conferences on Pervasive Computing (JCPC’09). 429--434. Google ScholarCross Ref
- Evelyn Duesterwald, Rajiv Gupta, and Mary Lou Soffa. 1996. A demand-driven analyzer for data flow testing at the integration level. In Proceedings of the 18th International Conference on Software Engineering (ICSE’96). IEEE Computer Society, 575--584. Google ScholarCross Ref
- Evelyn Duesterwald, Rajiv Gupta, and Mary Lou Soffa. 1997. A practical framework for demand-driven interprocedural data flow analysis. ACM Trans. Program. Lang. Syst. 19, 6 (Nov. 1997), 992--1030. Google ScholarDigital Library
- Jon Edvardsson. 1999. A Survey on Automatic Test Data Generation. In Proceedings of the 2nd Conference on Computer Science and Engineering. 21--28.Google Scholar
- Lynn M. Foreman and Stuart H. Zweben. 1993. A study of the effectiveness of control and data flow testing strategies. J. Syst. Softw. 21, 3 (1993), 215--228. Google ScholarDigital Library
- P. G. Frankl. 1987. The Use of Data Flow Information for the Selection and Evaluation of Software Test Data. Ph.D. Dissertation. University of New York, NY.Google Scholar
- Phyllis G. Frankl and Oleg Iakounenko. 1998. Further empirical studies of test effectiveness. In Proceedings of the ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT’98). 153--162. Google ScholarDigital Library
- P. G. Frankl and S. N. Weiss. 1993. An experimental comparison of the effectiveness of branch testing and data flow testing. IEEE Trans. Softw. Eng. 19, 8 (Aug. 1993), 774--787. Google ScholarDigital Library
- Phyllis G. Frankl, Stewart N. Weiss, and Elaine J. Weyuker. 1985. Asset: A System to Select and Evaluate Tests. Courant Institute of Mathematical Sciences, New York University.Google Scholar
- Phyllis G. Frankl and Elaine J. Weyuker. 1985. A data flow testing tool. In Proceedings of the 2nd Conference on Software Development Tools, Techniques, and Alternatives. IEEE Computer Society Press, Los Alamitos, CA, 46--53.Google Scholar
- P. G. Frankl and E. J. Weyuker. 1988. An applicable family of data flow testing criteria. IEEE Trans. Softw. Eng. 14, 10 (Oct. 1988), 1483--1498. Google ScholarDigital Library
- Gordon Fraser and Andrea Arcuri. 2012. Sound empirical evidence in software testing. In Proceedings of the 34th International Conference on Software Engineering (ICSE’12). 178--188. Google ScholarCross Ref
- Gordon Fraser and Andrea Arcuri. 2013. Whole test suite generation. IEEE Trans. Software Eng. 39, 2 (2013), 276--291. Google ScholarDigital Library
- Gordon Fraser, Franz Wotawa, and Paul Ammann. 2009. Testing with model checkers: A survey. Softw. Test., Verif. Reliab. 19, 3 (2009), 215--261. Google ScholarDigital Library
- Kamran Ghani and John A. Clark. 2009. Automatic test data generation for multiple condition and MCDC coverage. In The Proceedings of the 4th International Conference on Software Engineering Advances (ICSEA’09). 152--157. Google ScholarDigital Library
- Ahmed S. Ghiduk. 2010. A new software data-flow testing approach via ant colony algorithms. Universal J. Comput. Sci. Eng. Technol. 1, 1 (Oct. 2010), 64--72.Google Scholar
- Ahmed S. Ghiduk, Mary Jean Harrold, and Moheb R. Girgis. 2007. Using genetic algorithms to aid test-data generation for data-flow coverage. In APSEC. 41--48. Google ScholarCross Ref
- Moheb R. Girgis. 1993. Using symbolic execution and data flow criteria to aid test data selection. Softw. Test., Verif. Reliab. 3, 2 (1993), 101--112. Google ScholarCross Ref
- Moheb R. Girgis. 2005. Automatic test data generation for data flow testing using a genetic algorithm. J. UCS 11, 6 (2005), 898--915.Google Scholar
- Moheb R. Girgis, Ahmed S. Ghiduk, and Eman H. Abd-elkawy. 2014. Automatic generation of data flow test paths using a genetic algorithm. Int. J. Comput. Appl. 89, 12 (March 2014), 29--36.Google Scholar
- Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: Directed automated random testing. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, New York, NY, 213--223. Google ScholarDigital Library
- Bhargav S. Gulavani, Thomas A. Henzinger, Yamini Kannan, Aditya V. Nori, and Sriram K. Rajamani. 2006. SYNERGY: A new algorithm for property checking. In SIGSOFT FSE. 117--127. Google ScholarDigital Library
- Mark Harman, Yue Jia, and Yuanyuan Zhang. 2015. Achievements, open problems and challenges for search based software testing. In Proceedings of the 8th IEEE International Conference on Software Testing, Verification and Validation (ICST’15). 1--12. Google ScholarCross Ref
- Mark Harman, Sung Gon Kim, Kiran Lakhotia, Phil McMinn, and Shin Yoo. 2010. Optimizing for the number of tests generated in search based test data generation with an application to the oracle cost problem. In Proceedings of the 3rd International Conference on Software Testing, Verification and Validation (ICST’10), Workshop Proceedings. 182--191. Google ScholarDigital Library
- Mary Jean Harrold. 1994. Performing data flow testing in parallel. In Proceedings of the 8th International Symposium on Parallel Processing. 200--207. Google ScholarCross Ref
- M. Jean Harrold, Rajiv Gupta, and Mary Lou Soffa. 1993. A methodology for controlling the size of a test suite. ACM Trans. Softw. Eng. Methodol. 2, 3 (July 1993), 270--285. Google ScholarDigital Library
- Mary Jean Harrold and Brian A Malloy. 1992. Data flow testing of parallelized code. In Proceedings of the Conference on Software Maintenance, 1992. IEEE, 272--281. Google ScholarCross Ref
- Mary Jean Harrold and Gregg Rothermel. 1994. Performing data flow testing on classes. In SIGSOFT FSE. 154--163. Google ScholarDigital Library
- Mary Jean Harrold and Mary Lou Soffa. 1994. Efficient computation of interprocedural definition-use chains. ACM Trans. Program. Lang. Syst. 16, 2 (March 1994), 175--204. Google ScholarDigital Library
- Mohammad Mahdi Hassan and James H. Andrews. 2013. Comparing multi-point stride coverage and dataflow coverage. In Proceedings of the 35th International Conference on Software Engineering (ICSE’13). 172--181. Google ScholarCross Ref
- Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar, and Grégoire Sutre. 2002. Lazy abstraction. In POPL. 58--70. Google ScholarDigital Library
- P. M. Herman. 1976. A data flow analysis approach to program testing. Australian Comput. J. 8, 3 (1976), 92--96.Google Scholar
- John H. Holland. 1992. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence. MIT Press, Cambridge, MA.Google ScholarDigital Library
- Hyoung Seok Hong, Sung Deok Cha, Insup Lee, Oleg Sokolsky, and Hasan Ural. 2003. Data flow testing as model checking. In ICSE. 232--243.Google Scholar
- Hyoung Seok Hong, Young Gon Kim, Sung Deok Cha, Doo-Hwan Bae, and Hasan Ural. 2000. A test sequence selection method for statecharts. Softw. Test., Verif. Reliab. 10, 4 (2000), 203--227. Google ScholarCross Ref
- Hyoung Seok Hong and Hasan Ural. 2005. Dependence testing: Extending data flow testing with control dependence. In Proceedings of the 17th IFIP TC6/WG 6.1 International Conference on Testing of Communicating Systems (TestCom’05). 23--39.Google ScholarDigital Library
- J. R. Horgan and London. 1992. ATAC: A data flow coverage testing tool for C. In Proceedings of the Symposium on Assessment of Quality Software Development Tools. 2--10. Google ScholarCross Ref
- J. C. Huang. 1979. Detection of data flow anomaly through program instrumentation. IEEE Trans. Softw. Eng. 5, 3 (1979), 226--236. Google ScholarDigital Library
- Monica Hutchins, Herbert Foster, Tarak Goradia, and Thomas J. Ostrand. 1994. Experiments of the effectiveness of dataflow- and controlflow-based test adequacy criteria. In ICSE. 191--200.Google Scholar
- RTCA. 1992. DO-178b: Software considerations in airborne systems and equipment certification. Requirements and Technical Concepts for Aviation (December 1992).Google Scholar
- Kobi Inkumsah and Tao Xie. 2008. Improving structural testing of object-oriented programs via integrating evolutionary testing and symbolic execution. In Proceedings of the 23rd IEEE/ACM International Conference on Automated Software Engineering (ASE’08). 297--306. Google ScholarDigital Library
- Laura Inozemtseva and Reid Holmes. 2014. Coverage is not strongly correlated with test suite effectiveness. In Proceedings of the 36th International Conference on Software Engineering (ICSE’14). 435--445. Google ScholarDigital Library
- Mariam Kamkar, Peter Fritzson, and Nahid Shahmehri. 1993. Interprocedural dynamic slicing applied to interprocedural data flow testing. In ICSM. 386--395.Google Scholar
- Ken Kennedy. 1979. A Survey of Data Flow Analysis Techniques. IBM Thomas J. Watson Research Division.Google Scholar
- Abdelaziz Khamis, Reem Bahgat, and Rana Abdelaziz. 2011. Automatic test data generation using data flow information. Dogus Univ. J. 2 (2011), 140--153.Google ScholarCross Ref
- Arunkumar Khannur. 2011. Software Testing - Techniques and Applications. Pearson Publications.Google Scholar
- James C. King. 1976. Symbolic execution and program testing. Commun. ACM 19, 7 (July 1976), 385--394. Google ScholarDigital Library
- Kiran Lakhotia, Mark Harman, and Phil McMinn. 2007. A multi-objective approach to search-based test data generation. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’07). 1098--1105. Google ScholarDigital Library
- Kiran Lakhotia, Phil McMinn, and Mark Harman. 2009. Automated test data generation for coverage: Haven’t we solved this problem yet? In Proceedings of the 2009 Testing: Academic and Industrial Conference - Practice and Research Techniques. IEEE Computer Society, 95--104.Google ScholarDigital Library
- Kiran Lakhotia, Nikolai Tillmann, Mark Harman, and Jonathan de Halleux. 2010. FloPSy - Search-based floating point constraint solving for symbolic execution. In Proceedings of the 22nd IFIP WG 6.1 International Conference on Testing Software and Systems (ICTSS’10). 142--157.Google ScholarCross Ref
- Janusz W. Laski and Bogdan Korel. 1983. A data flow oriented program testing strategy. IEEE Trans. Software Eng. 9, 3 (1983), 347--354. Google ScholarDigital Library
- Thomas Lengauer and Robert Endre Tarjan. 1979. A fast algorithm for finding dominators in a flowgraph. ACM Trans. Program. Lang. Syst. 1, 1 (1979), 121--141. Google ScholarDigital Library
- You Li, Zhendong Su, Linzhang Wang, and Xuandong Li. 2013. Steering symbolic execution to less traveled paths. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages 8 Applications (OOPSLA’13), part of SPLASH 2013. 19--32. Google ScholarDigital Library
- Konstantinos Liaskos and Marc Roper. 2007. Automatic test-data generation: An immunological approach. In Proceedings of Testing: Accademic and Industrial Conference - Practice and Research Techniques (TAIC PART’07). 77--81.Google ScholarCross Ref
- Konstantinos Liaskos and Marc Roper. 2008. Hybridizing evolutionary testing with artificial immune systems and local search. In Proceedings of the 1st International Conference on Software Testing Verification and Validation (ICST’08), Workshops Proceedings. 211--220. Google ScholarDigital Library
- Konstantinos Liaskos, Marc Roper, and Murray Wood. 2007. Investigating data-flow coverage of classes using evolutionary algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’07). 1140. Google ScholarDigital Library
- Yu Lin, Xucheng Tang, Yuting Chen, and Jianjun Zhao. 2009. A divergence-oriented approach to adaptive random testing of java programs. In Proceedings of the 24th IEEE/ACM International Conference on Automated Software Engineering (ASE’09). 221--232. Google ScholarDigital Library
- Chien-Hung Liu, David Chenho Kung, and Pei Hsia. 2000. Object-based data flow testing of web applications. In Proceedings of the 1st Asia-Pacific Conference on Quality Software, 2000. IEEE, 7--16.Google Scholar
- Kin-Keung Ma, Yit Phang Khoo, Jeffrey S. Foster, and Michael Hicks. 2011. Directed symbolic execution. In SAS. 95--111. Google ScholarCross Ref
- Nicos Malevris and Derek F. Yates. 2006. The collateral coverage of data flow criteria when branch testing. Inf. Softw. Technol. 48, 8 (2006), 676--686. Google ScholarCross Ref
- Martina Marré and Antonia Bertolino. 1996. Unconstrained duas and their use in achieving all-uses coverage. In Proceedings of the 1996 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’96). ACM, New York, NY, 147--157.Google ScholarDigital Library
- Martina Marré and Antonia Bertolino. 2003. Using spanning sets for coverage testing. IEEE Trans. Softw. Eng. 29, 11 (Nov. 2003), 974--984. Google ScholarDigital Library
- Vincenzo Martena, Alessandro Orso, and Mauro Pezzè. 2002. Interclass testing of object oriented software. In Proceedings of the 8th International Conference on Engineering of Complex Computer Systems (ICECCS’02). 135--144. Google ScholarCross Ref
- Phil McMinn. 2004. Search-based software test data generation: A survey. Softw. Test. Verif. Reliab. 14, 2 (2004), 105--156. Google ScholarDigital Library
- Lijun Mei, W. K. Chan, and T. H. Tse. 2008. Data flow testing of service-oriented workflow applications. In Proceedings of the ACM/IEEE 30th International Conference on Software Engineering, 2008 (ICSE’08). IEEE, 371--380. Google ScholarDigital Library
- Lijun Mei, W. K. Chan, and T. H. Tse. 2009. Data flow testing of service choreography. In Proceedings of the the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering. ACM, 151--160. Google ScholarDigital Library
- Ettore Merlo and Giuliano Antoniol. 1999. A static measure of a subset of intra-procedural data flow testing coverage based on node coverage. In CASCON. 7.Google Scholar
- Zbigniew Michalewicz. 1994. Genetic Algorithms + Data Structures = Evolution Programs (2nd extended ed.). Springer-Verlag New York, New York, NY.Google ScholarDigital Library
- Matteo Miraz. 2010. Evolutionary Testing of Stateful Systems: A Holistic Approach. Ph.D. Dissertation. Politecnico di Milano.Google Scholar
- Jonathan Misurda, Jim Clause, Juliya Reed, Bruce R. Childers, and Mary Lou Soffa. 2005a. Jazz: A tool for demand-driven structural testing. In Compiler Construction. Springer, 242--245. Google ScholarDigital Library
- Jonathan Misurda, James A. Clause, Juliya L. Reed, Bruce R. Childers, and Mary Lou Soffa. 2005b. Demand-driven structural testing with dynamic instrumentation. In Proceedings of the 27th International Conference on Software Engineering (ICSE’05). 156--165.Google ScholarDigital Library
- Akbar Siami Namin and James H. Andrews. 2009. The influence of size and coverage on test suite effectiveness. In Proceedings of the 18th International Symposium on Software Testing and Analysis (ISSTA’09). 57--68. Google ScholarDigital Library
- Narmada Nayak and Durga Prasad Mohapatra. 2010. Automatic test data generation for data flow testing using particle swarm optimization. In IC3 (2). 1--12. Google ScholarCross Ref
- A. Jefferson Offutt, Zhenyi Jin, and Jie Pan. 1999. The dynamic domain reduction procedure for test data generation. Softw. Pract. Exper. 29, 2 (1999), 167--193. Google ScholarDigital Library
- Norbert Oster. 2005. Automated generation and evaluation of dataflow-based test data for object-oriented software. In QoSA/SOQUA. 212--226. Google ScholarDigital Library
- Thomas J. Ostrand and Elaine J. Weyuker. 1991a. Data flow-based test adequacy analysis for languages with pointers. In Proceedings of the Symposium on Testing, Analysis, and Verification. 74--86.Google Scholar
- Thomas J. Ostrand and Elaine J. Weyuker. 1991b. Data flow-based test adequacy analysis for languages with pointers. In Proceedings of the Symposium on Testing, Analysis, and Verification. ACM, 74--86. Google ScholarDigital Library
- Carlos Pacheco, Shuvendu K. Lahiri, Michael D. Ernst, and Thomas Ball. 2007. Feedback-directed random test generation. In Proceedings of the 29th International Conference on Software Engineering (ICSE’07). 75--84. Google ScholarDigital Library
- H. D. Pande, W. A. Landi, and B. G. Ryder. 1994. Interprocedural def-use associations for C systems with single level pointers. IEEE Trans. Softw. Eng. 20, 5 (May 1994), 385--403. Google ScholarDigital Library
- Mauro Pezzè and Michal Young. 2007. Software Testing and Analysis: Process, Principles and Techniques. Wiley.Google Scholar
- Yu Qi, David Kung, and Eric Wong. 2006. An agent-based data-flow testing approach for Web applications. Inf. Softw. Technol. 48, 12 (2006), 1159--1171. Google ScholarDigital Library
- Sandra Rapps and Elaine J. Weyuker. 1982. Data flow analysis techniques for test data selection. In Proceedings of the 6th International Conference on Software Engineering (ICSE’82). IEEE Computer Society Press, Los Alamitos, CA, 272--278.Google Scholar
- Sandra Rapps and Elaine J. Weyuker. 1985. Selecting software test data using data flow information. IEEE Trans. Software Eng. 11, 4 (1985), 367--375. Google ScholarDigital Library
- Torsten Robschink and Gregor Snelting. 2002. Efficient path conditions in dependence graphs. In Proceedings of the 24th International Conference on Software Engineering (ICSE’02). ACM, New York, NY, 478--488. Google ScholarCross Ref
- Raul Santelices and Mary Jean Harrold. 2007. Efficiently monitoring data-flow test coverage. In Proceedings of the 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE’07). ACM, New York, NY, 343--352. Google ScholarDigital Library
- Raúl A. Santelices, James A. Jones, Yanbing Yu, and Mary Jean Harrold. 2009. Lightweight fault-localization using multiple coverage types. In Proceedings of the 31st International Conference on Software Engineering, (ICSE’09). 56--66. Google ScholarDigital Library
- Raúl A. Santelices, Saurabh Sinha, and Mary Jean Harrold. 2006. Subsumption of program entities for efficient coverage and monitoring. In Proceedings of the 3rd International Workshop on Software Quality Assurance (SOQUA’06). 2--5. Google ScholarDigital Library
- Koushik Sen, Darko Marinov, and Gul Agha. 2005. CUTE: A concolic unit testing engine for C. In Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, New York, NY, 263--272. Google ScholarDigital Library
- Sanjay Singla, Dharminder Kumar, H. M. Rai, and Priti Singla. 2011a. A hybrid PSO approach to automate test data generation for data flow coverage with dominance concepts. J. Adv. Sci. Technol. 37 (2011), 15--26.Google Scholar
- Sanjay Singla, Priti Singla, and H. M. Rai. 2011b. An automatic test data generation for data flow coverage using soft computing approach. IJRRCS 2, 2 (2011), 265--270.Google Scholar
- Amie L. Souter and Lori L. Pollock. 2003. The construction of contextual def-use associations for object-oriented systems. IEEE Trans. Software Eng. 29, 11 (2003), 1005--1018. Google ScholarDigital Library
- Ting Su, Zhoulai Fu, Geguang Pu, Jifeng He, and Zhendong Su. 2015. Combining symbolic execution and model checking for data flow testing. In Proceedings of the 37th IEEE/ACM International Conference on Software Engineering (ICSE’15), Volume 1. 654--665. Google ScholarCross Ref
- Ting Su, Geguang Pu, Bin Fang, Jifeng He, Jun Yan, Siyuan Jiang, and Jianjun Zhao. 2014. Automated coverage-driven test data generation using dynamic symbolic execution. In Proceedings of the 8th International Conference on Software Security and Reliability (SERE’14). 98--107. Google ScholarDigital Library
- Tao Sun, Zheng Wang, Geguang Pu, Xiao Yu, Zongyan Qiu, and Bin Gu. 2009. Towards scalable compositional test generation. In QSIC. 353--358. Google ScholarDigital Library
- Paolo Tonella. 2004. Evolutionary testing of classes. In Proceedings of the ACM/SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’04). 119--128. Google ScholarDigital Library
- Hasan Ural, Kassem Saleh, and Alan W. Williams. 2000. Test generation based on control and data dependencies within system specifications in SDL. Comput. Commun. 23, 7 (2000), 609--627. Google ScholarDigital Library
- Auri Marcelo Rizzo Vincenzi, José Carlos Maldonado, W. Eric Wong, and Márcio Eduardo Delamaro. 2005. Coverage testing of java programs and components. Sci. Comput. Program. 56, 1 (2005), 211--230.Google ScholarDigital Library
- Mattia Vivanti. 2014. Dynamic data-flow testing. In Proceedings of the 36th International Conference on Software Engineering (ICSE’14), Companion Proceedings. 682--685. Google ScholarDigital Library
- Mattia Vivanti, Andre Mis, Alessandra Gorla, and Gordon Fraser. 2013. Search-based data-flow test generation. In ISSRE. 370--379. Google ScholarCross Ref
- Chen-Wei Wang and Alessandra Cavarra. 2009. Checking model consistency using data-flow testing. In Proceedings of the Asia-Pacific Software Engineering Conference, 2009 (APSEC’09). IEEE, 414--421. Google ScholarDigital Library
- Zheng Wang, Xiao Yu, Tao Sun, Geguang Pu, Zuohua Ding, and Jueliang Hu. 2009. Test data generation for derived types in C program. In TASE. 155--162. Google ScholarDigital Library
- Joachim Wegener, André Baresel, and Harmen Sthamer. 2001. Evolutionary test environment for automatic structural testing. Inf. Softw. Technol. 43, 14 (2001), 841--854. Google ScholarCross Ref
- Elaine J. Weyuker. 1990. The cost of data flow testing: An empirical study. IEEE Trans. Software Eng. 16, 2 (1990), 121--128. Google ScholarDigital Library
- Elaine J. Weyuker. 1993. More experience with data flow testing. IEEE Trans. Software Eng. 19, 9 (1993), 912--919. Google ScholarDigital Library
- Qian Yang, J. Jenny Li, and David M. Weiss. 2009. A survey of coverage-based testing tools. Comput. J. 52, 5 (2009), 589--597. Google ScholarDigital Library
- Xiao Yu, Shuai Sun, Geguang Pu, Siyuan Jiang, and Zheng Wang. 2011. A parallel approach to concolic testing with low-cost synchronization. Electr. Notes Theor. Comput. Sci. 274 (2011), 83--96. Google ScholarDigital Library
- Cristian Zamfir and George Candea. 2010. Execution synthesis: A technique for automated software debugging. In EuroSys. 321--334. Google ScholarDigital Library
- Jianjun Zhao. 2003. Data-flow-based unit testing of aspect-oriented programs. In Proceedings of the 27th Annual International Computer Software and Applications Conference, 2003 (COMPSAC’03). IEEE, 188--197.Google ScholarDigital Library
Index Terms
- A Survey on Data-Flow Testing
Recommendations
Dynamic data-flow testing
ICSE Companion 2014: Companion Proceedings of the 36th International Conference on Software EngineeringData-flow testing techniques have long been discussed in the literature, yet to date they are still of little practical relevance. The applicability of data-flow testing is limited by the complexity and the imprecision of the approach: writing a test ...
Efficiently monitoring data-flow test coverage
ASE '07: Proceedings of the 22nd IEEE/ACM International Conference on Automated Software EngineeringStructural testing of software requires monitoring the software's execution to determine which program entities are executed by a test suite. Such monitoring can add considerable overhead to the execution of the program, adversely affecting the cost of ...
Precise and efficient integration of interprocedural alias information into data-flow analysis
Data-flow analysis is a basis for program optimization and parallelizing transformations. The mechanism of passing reference parameters at call sites generates interprocedural aliases which complicate this analysis. Solutions have been developed for ...
Comments