Atif M. Memon
Mary Lou Soffa
ABSTRACT
A widespread recognition of the usefulness of graphical user interfaces (GUIs) has established their importance as critical components of today's software. GUIs have characteristics different from traditional software, and conventional testing techniques do not directly apply to GUIs. This paper's focus is on coverage critieria for GUIs, important rules that provide an objective measure of test quality. We present new coverage criteria to help determine whether a GUI has been adequately tested. These coverage criteria use events and event sequences to specify a measure of test adequacy. Since the total number of permutations of event sequences in any non-trivial GUI is extremely large, the GUI's hierarchical structure is exploited to identify the important event sequences to be tested. A GUI is decomposed into GUI components, each of which is used as a basic unit of testing. A representation of a GUI component, called an event-flow graph, identifies the interaction of events within a component and intra-component criteria are used to evaluate the adequacy of tests on these events. The hierarchical relationship among components is represented by an integration tree, and inter-component coverage criteria are used to evaluate the adequacy of test sequences that cross components. Algorithms are given to construct event-flow graphs and an integration tree for a given GUI, and to evaluate the coverage of a given test suite with respect to the new coverage criteria. A case study illustrates the usefulness of the coverage report to guide further testing and an important correlation between event-based coverage of a GUI and statement coverage of its software's underlying code.
- 1.D. Chays, S. Dan, P. G. Frankl, F. I. Vokolos, and E. J. Weyuker. A framework for testing database applications. In Proceedings of the PO00 International Symposium on Software Testing and Analysis (ISSTA), pages 147-157, 2000. Google Scholar
Digital Library
- 2.J. S. Gourlay. A mathematicM framework for the investigation of testing. IEEE Transactions on Software En9ineering, 9(6):686-709, Nov. 1983.Google ScholarDigital Library
- 3.M. L. Hammontree, J. J. Hendrickson, and B. W. Hensley. Integrated data capture and analysis tools for research and testing an graphical user interfaces. In Proceedings of the Conference on Human Factors in Computing Systems, pages 431-432, New York, NY, USA, May 1992. ACM Press. Google ScholarDigital Library
- 4.M. J. Harrold and M. L. Sofia. Interprocedual data flow testing. In R. A. Kemmerer, editor, Proceedings of the ACM SIGSOFT '89 Third Symposium on Testing, Analysis, and Verification (TA V3), pages 158-167, 1989. Google ScholarDigital Library
- 5.P. C. Jorgensen and C. Erickson. Object-oriented integration testing. Communications of the A CM, 37(9):30-38, Sept. 1994. Google ScholarDigital Library
- 6.D. J. Kasik and H. G. George. Toward automatic generation of novice user test scripts. In Proceedings of the Conference on Human Factors in Computing Systems : Common Ground, pages 244-251, New York, 13-18 Apr. 1996. ACM Press. Google ScholarDigital Library
- 7.L. R. Kepple. The black art of GUI testing. Dr. Dobb's Journal of Software Tools, 19(2):40, Feb. 1994.Google Scholar
- 8.A. M. Memon, M. E. Pollack, and M. L. Sofia. Using a goal-driven approach to generate test cases for GUIs. In Proceedings of the 21st International Conference on Software En9ineerin9 , pages 257-266. ACM Press, May 1999. Google ScholarDigital Library
- 9.A. M. Memon, M. E. Pollack, and M. L. Sofia. Automated test oracles for GUIs. In D. S. Rosenblum, editor, Proceedings of the A CM SIGSOFT 8th International Symposium on the Foundations of Software Engineering (FSE-O0), pages 30-39, NY, Nov. 8-10 2000. ACM Press. Google ScholarDigital Library
- 10.A. M. Memon, M. E. Pollack, and M. L. Sofia. A planning-based approach to GUI testing. In Proceedings of The 13th International Software/Internet Quality Week, May 2000.Google Scholar
- 11.A. M. Memon, M. E. Pollack, and M. L. Sofia. Hierarchical GUI test case generation using automated planning. IEEE Transactions on Software Engineering, 27(2):144-155, Feb. 2001. Google ScholarDigital Library
- 12.T. Ostrand, A. Anodide, H. Foster, and T. Goradia. A visual test development environment for GUI systems. In Proceedings of the A CM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA-98), pages 82-92, New York, Mar.2-5 1998. ACM Press. Google Scholar
- 13.S. Rapps and E. J. Weyuker. Selecting software test data using data flow information. IEEE Transactions on Software Engineering, 11(4):367-375, Apr. 1985. Google ScholarDigital Library
- 14.R. K. Shehady and D. P. Siewiorek. A method to automate user interface testing using variable finite state machines. In Proceedings of The Twenty-Seventh Annual International Symposium on Fault-Tolerant Computing (FTCS'97), pages 80-88, Washington- Brussels - Tokyo, June 1997. IEEE Press. Google ScholarDigital Library
- 15.L. The. Stress Tests For GUI Programs. Datamation, 38(18):37, Sept. 1992.Google Scholar
- 16.E. J. Weyuker. The applicability of program schema results to programs. International Journal of Computer and Information Sciences, 8(5):387-403, Oct. 1979.Google ScholarCross Ref
- 17.E. J. Weyuker. Translatability and decidability questions for restricted classes of program schemas. SIAM Journal on Computing, 8(4):587-598, 1979.Google ScholarCross Ref
- 18.L. White and H. Almezen. Generating test cases for GUI responsibilities using complete interaction sequences. In Proceedings of the International Symposium on Software Reliability Engineering, pages 110-121, Oct. 8-11 2000. Google ScholarDigital Library
- 19.S. Wolfram. Mathematiea: A System for Doing Mathematics by Computer. Addison-Wesley, Reading, Massachusetts, 1988. Google ScholarDigital Library
- 20.H. Zhu and P. Hall. Test data adequacy measurements. Software Engineering Journal, 8(1):21-30, Jan. 1993. Google ScholarDigital Library
Index Terms
- Coverage criteria for GUI testing
Recommendations
Automated model-based Android GUI testing using multi-level GUI comparison criteria
ASE '16: Proceedings of the 31st IEEE/ACM International Conference on Automated Software EngineeringAutomated Graphical User Interface (GUI) testing is one of the most widely used techniques to detect faults in mobile applications (apps) and to test functionality and usability. GUI testing exercises behaviors of an application under test (AUT) by ...
Coverage criteria for GUI testing
A widespread recognition of the usefulness of graphical user interfaces (GUIs) has established their importance as critical components of today's software. GUIs have characteristics different from traditional software, and conventional testing ...
An event-flow model of GUI-based applications for testing: Research Articles
Graphical user interfaces (GUIs) are by far the most popular means used to interact with today's software. The functional correctness of a GUI is required to ensure the safety, robustness and usability of an entire software system. GUI testing ...
Comments