Eric J. Rapos
James R. Cordy
ABSTRACT
This paper presents an industrial case study that explores the co-evolution relationship between Matlab Simulink Models and their associated test suites. Through an analysis of differences between releases of both the models and their tests, we are able to determine what the relation between the model evolution and test evolution is, or if one exists at all. Using this comparison methodology, we present empirical results from a production system of 64 Matlab Simulink Models evolving over 9 releases. In our work we show that in this system there is a strong co-evolution relationship (a correlation value of r = 0.9, p < 0.01) between the models and tests, and we examine the cases where the relationship does not exist. We also pose, and answer, three specific research questions about the practices of development and testing over time for the system under study.
- MathWorks Simulink product page. http://www.mathworks.com/products/simulink/. Accessed: 2015-10-27.Google Scholar
- Arcuri, A., and Yao, X. A novel co-evolutionary approach to automatic software bug fixing. In Congress on Evolutionary Computation, 2008 (2008), IEEE, pp. 162--168.Google ScholarCross Ref
- Dias Neto, A., Subramanyan, R., Vieira, M., and Travassos, G. A survey on model-based testing approaches: a systematic review. In ASE '07 (2007), ACM, pp. 31--36. Google ScholarDigital Library
- Ens, B., Rea, D., Shpaner, R., Hemmati, H., Young, J., and Irani, P. Chronotwigger: A visual analytics tool for understanding source and test co-evolution. In VISSOFT '14 (2014), IEEE, pp. 117--126. Google ScholarDigital Library
- Lehman, M. Programs, life cycles, and laws of software evolution. Proceedings of the IEEE 68, 9 (1980), 1060--1076.Google ScholarCross Ref
- Lubsen, Z., Zaidman, A., and Pinzger, M. Using association rules to study the co-evolution of production & test code. In MSR '09 (2009), IEEE, pp. 151--154. Google ScholarDigital Library
- Marinescu, P., Hosek, P., and Cadar, C. Covrig: A framework for the analysis of code, test, and coverage evolution in real software. In International Symposium on Software Testing and Analysis, 2014 (2014), ACM, pp. 93--104. Google ScholarDigital Library
- Marsavina, C., Romano, D., and Zaidman, A. Studying fine-grained co-evolution patterns of production and test code. In SCAM '14 (2014), IEEE, pp. 195--204. Google ScholarDigital Library
- Matinnejad, R., Nejati, S., and Briand, L. Automated test suite generation for time-continuous simulink models. Tech. Rep. TR-SnT-2015-7, 2015.Google Scholar
- Mirzaaghaei, M., Pastore, F., and Pezze, M. Automatically repairing test cases for evolving method declarations. In ICSM '10 (2010), IEEE, pp. 1--5. Google ScholarDigital Library
- Paige, R., Matragkas, N., and Rose, L. Evolving models in model-driven engineering: State-of-the-art and future challenges. J. Syst. Software (2015). Google ScholarDigital Library
- Pinto, L., Sinha, S., and Orso, A. Understanding myths and realities of test-suite evolution. In FSE '12 (2012), ACM, p. 33. Google ScholarDigital Library
- Rapos, E. Co-evolution of model-based tests for industrial automotive software. In ICSME '14 (September 2014), IEEE, pp. 663--663. Google ScholarDigital Library
- Rapos, E. Co-evolution of model-based tests for industrial automotive software. In ICST '15 (April 2015), pp. 1--2.Google ScholarCross Ref
- Zaidman, A., Van Rompaey, B., Demeyer, S., and Van Deursen, A. Mining software repositories to study co-evolution of production & test code. In ICST '08 (2008), IEEE, pp. 220--229. Google ScholarDigital Library
- Zaidman, A., Van Rompaey, B., van Deursen, A., and Demeyer, S. Studying the co-evolution of production and test code in open source and industrial developer test processes through repository mining. Empirical Software Engineering 16, 3 (2011), 325--364. Google ScholarDigital Library
- Zech, P., Felderer, M., Kalb, P., and Breu, R. A generic platform for model-based regression testing. In Leveraging Applications of Formal Methods, Verification and Validation. Technologies for Mastering Change. Springer, 2012, pp. 112--126. Google ScholarDigital Library
Index Terms
- Examining the co-evolution relationship between simulink models and their test cases
Recommendations
Co-evolution of Model-Based Tests for Industrial Automotive Software
ICSME '14: Proceedings of the 2014 IEEE International Conference on Software Maintenance and EvolutionModel-based software is evolving at an increasing rate, and this has an impact on model-based test suites, often causing unnecessary regeneration of tests. Our work proposes that by examining evolution patterns of Simulink automotive models and their ...
On the Relationship between Model Coverage and Code Coverage Using MATLAB's Simulink
QRS '15: Proceedings of the 2015 IEEE International Conference on Software Quality, Reliability and SecuritySoftware Testing is an approach to ensuring the quality of software systems. Testing of safety-critical systems often requires conformance to certain code coverage criteria, including for example, in aviation, Modified Condition/Decision Coverage (MC/DC)...
Regression test suite prioritization using system models
During regression testing, a modified system is often retested using an existing test suite. Since the size of the test suite may be very large, testers are interested in detecting faults in the modified system as early as possible during this retesting ...
Comments