Jing Guan
ABSTRACT
Effective testing of safety-critical real-time embedded software is difficult and expensive. Many companies are hesitant about the cost of formalized criteria-based testing and are not convinced of the benefits. This paper presents the results of an industrial case study that compared the normal testing at a company (manual functional testing) with testing based on the logic-based criterion of correlated active clause coverage (CACC). The evaluation was performed during the testing of embedded, real-time control software that has been deployed in a safety-critical application in the transportation industry. We found in our study that the test cases generated to satisfy the CACC criterion detected major safety-critical faults that were not detected by functional testing. We also found that the cost required for CACC testing was not necessarily higher than the cost of functional testing. There were also several faults that were found by the functional tests that were not found by CACC tests.
- P. Ammann, J. Offutt, and H. Huang. Coverage criteria for logical expressions. In Proceedings of the 14th International Symposium on Software Reliability Engineering, pages 99--107, Denver, CO, November 2003. IEEE Computer Society Press. Google Scholar
Digital Library
- J. Chilenski and L. A. Richey. Definition for a masking form of modified condition decision coverage (MCDC). Technical report, Boeing, Seattle, WA, 1997. http://www.boeing.com/nosearch/mcdc/.Google Scholar
- J. J. Chilenski and S. P. Miller. Applicability of modified condition/decision coverage to software testing. Software Engineering Journal, 9(5):193--200, September 1994.Google ScholarCross Ref
- R. A. DeMillo, R. J. Lipton, and F. G. Sayward. Hints on test data selection: Help for the practicing programmer. IEEE Computer, 11(4):34--41, April 1978.Google ScholarDigital Library
- A. Dupuy and N. Leveson. An empirical evaluation of the MC/DC coverage criterion on the HETE-2 satellite software. In Proceedings of the Digital Aviations Systems Conference (DASC), October 2000.Google ScholarCross Ref
- K. Foster. Error sensitive test case analysis. IEEE Transactions on Software Engineering, 6(3):258--264, May 1980.Google ScholarDigital Library
- M. Grindal, J. Offutt, and J. Mellin. On the testing maturity of software producing organizations. In Testing: Academia & Industry Conference - Practice And Research Techniques (TAIC / PART 2006), Windsor, UK, August 2006. IEEE Computer Society Press. Google ScholarDigital Library
- J. Offutt, S. Liu, A. Abdurazik, and P. Ammann. Generating test data from state-based specifications. Software Testing, Verification, and Reliability, 13(1):25--53, March 2003.Google Scholar
- RTCA-DO-178B. Software considerations in airborne systems and equipment certification, December 1992.Google Scholar
- S. A. Vilkomir and J. P. Bowen. Reinforced condition/decision coverage (RC/DC): A new criterion for software testing. In Proceedings of ZB2002: 2nd International Conference of Z and B Users, pages 295--313, Grenoble, France, January 2002. Springer-Verlag, LNCS 2272. Google ScholarDigital Library
- E. Weyuker, T. Goradia, and A. Singh. Automatically generating test data from a boolean specification. IEEE Transactions on Software Engineering, 20(5):353--363, May 1994. Google ScholarDigital Library
Index Terms
- An industrial case study of structural testing applied to safety-critical embedded software
Recommendations
Using aspects for testing of embedded software: experiences from two industrial case studies
Aspect-oriented software testing is emerging as an important alternative to conventional procedural and object-oriented testing techniques. This paper reports experiences from two case studies where aspects were used for the testing of embedded software ...
A Search-Based Approach for Cost-Effective Software Test Automation Decision Support and an Industrial Case Study
ICSTW '14: Proceedings of the 2014 IEEE International Conference on Software Testing, Verification, and Validation WorkshopsTest automation is a widely-used approach to reduce the cost of manual software testing. However, if it is not planned or conducted properly, automated testing would not necessarily be more cost effective than manual testing. Deciding what parts of a ...
A Novel Testing Approach for Safety-Critical Software
ATS '99: Proceedings of the 8th Asian Test SymposiumIn this paper we propose a new black-box testing approach on how to select test cases to overcome the shortcoming that some conventional black-box testing methods lack precise testing adequacy measures to measure the quality of testing and direct the ...
Comments