Mark van den Brand
ABSTRACT
Model comparison is an important aspect of model driven software engineering. In particular, exploring the evolution of a model would be impossible without means for comparing different versions of that model. However, the techniques and tools for model comparison are still being perfected for practical application. Moreover, there exist no systematic methods and no controlled benchmarks that could be used for assessing the quality of tools for model comparison.
In this paper, we describe a systematic method for assessing the quality of model-comparison tools, and we present a data set to be used for controlled assessment experiments. Additionally, we use our method, and the specified data, to asses the quality of two model-comparison tools, namely EMFCompare and RCVDiff. The results of the experiments show that, in generic cases, both tools exhibit similar performance, and that both tools are of similar quality, though there are some notable difference in the details.
The defined method, the selected dataset, and the results obtained by assessing the two mentioned tools, constitute a benchmark for model-comparison tools.
- ATL transformation language. http://www.eclipse.org/atl/ (Viewed January 2011).Google Scholar
- ATL transformations zoo. http://www.eclipse.org/m2m/atl/atlTransformations/ (Viewed January 2011).Google Scholar
- ATLAN metamodel zoo. http://www.emn.fr/z--info/atlanmod/index.php/Ecore (Viewed January 2011).Google Scholar
- A benchmark set of experimental data. http://www.win.tue.nl/~zprotic/benchmark.html (Viewed March 2011).Google Scholar
- CIF: The compositional interchange format for hybrid systems. http://se.wtb.tue.nl/sewiki/cif/start (Viewed March 2011).Google Scholar
- Cumulative frequency analysis with probability distributions. http://www.waterlog.info/cumfreq.htm (Viewed January 2011).Google Scholar
- Eclipse. www.eclipse.org (Viewed June 2010).Google Scholar
- Ecore. download.eclipse.org/modeling/emf/emf/javadoc/2.5.0/org/eclipse/emf/ecore/package-summary.html#details (Viewed June 2010).Google Scholar
- EMF compare project. http://wiki.eclipse.org/index.php/EMFCompare (Viewed October 2009).Google Scholar
- Epsilon Transformation Language. http://www.eclipse.org/gmt/epsilon/doc/etl/ (Viewed April 2011).Google Scholar
- Fujaba. http://www.fujaba.de (Viewed March 2011).Google Scholar
- Metaobject facility. www.omg.org/mof (Viewed June 2010).Google Scholar
- Xtext. http://www.eclipse.org/Xtext/ (Viewed March 2011).Google Scholar
- D. A. v. Beek, P. Collins, D. E. Nadales, J. Rooda, and R. R. H. Schiffelers. New concepts in the abstract format of the compositional interchange format. In A. Giua, C. Mahuela, M. Silva, and J. Zaytoon, editors, 3rd IFAC Conference on Analysis and Design of Hybrid Systems, pages 250--255, Zaragoza, Spain, 2009.Google Scholar
- A. Cicchetti, D. D. Ruscio, R. Eramo, and A. Pierantonio. A metamodel independent approach to difference representation. Journal of Object Technology, pages 165--185, 2007.Google Scholar
- K. Garcés, F. Jouault, P. Cointe, and J. Bézivin. Managing model adaptation by precise detection of metamodel changes. In Proceedings of the 5th European Conference on Model Driven Architecture - Foundations and Applications, ECMDA-FA '09, pages 34--49, Berlin, Heidelberg, 2009. Springer-Verlag. Google ScholarDigital Library
- M. Herrmannsdoerfer, S. D. Vermolen, and G. Wachsmuth. An extensive catalog of operators for the coupled evolution of metamodels and models. In M. van den Brand, B. Malloy, and S. Staab, editors, Software Language Engineering, Third International Conference, SLE 2010, Eindhoven, The Netherlands, October 12--13, 2010, Revised Selected Papers, Lecture Notes in Computer Science. Springer, 2010. Google ScholarDigital Library
- ISO - International Organization for Standardization. International Standard ISO/IEC 25000 - Software engineering -- Software product Quality Requirements and Evaluation (SQuaRE). Geneva, Switzerland, 2005.Google Scholar
- U. Kelter, J. Wehren, and J. Niere. A generic difference algorithm for uml models. In P. Liggesmeyer, K. Pohl, and M. Goedicke, editors, Software Engineering 2005, volume 64 of LNI, pages 105--116. GI, 2005.Google Scholar
- D. S. Kolovos. Establishing correspondences between models with the epsilon comparison language. In ECMDA-FA '09: Proceedings of the 5th European Conference on Model Driven Architecture - Foundations and Applications, pages 146--157, Berlin, Heidelberg, 2009. Springer-Verlag. Google ScholarDigital Library
- D. S. Kolovos, D. Di Ruscio, A. Pierantonio, and R. F. Paige. Different models for model matching: An analysis of approaches to support model differencing. ICSE Workshop on Comparison and Versioning of Software Models, pages 1--6, 2009. Google ScholarDigital Library
- P. Konemann. Model-independent differences. In Proceedings of the 2009 ICSE Workshop on Comparison and Versioning of Software Models (CVSM '09), pages 37--42, Washington, DC, USA, 2009. IEEE Computer Society. Google ScholarDigital Library
- A. Mougenot, A. Darrasse, X. Blanc, and M. Soria. Uniform random generation of huge metamodel instances. In Proceedings of the 5th European Conference on Model Driven Architecture - Foundations and Applications, ECMDA-FA '09, pages 130--145, Berlin, Heidelberg, 2009. Springer-Verlag. Google ScholarDigital Library
- H. Oliveira, L. Murta, and C. Werner. Odyssey-vcs: a flexible version control system for uml model elements. In Proceedings of the 12th international workshop on Software configuration management, SCM '05, pages 1--16, New York, NY, USA, 2005. ACM. Google ScholarDigital Library
- M. van den Brand, Z. Protić, and T. Verhoeff. Fine-grained metamodel-assisted model comparison. In Proceedings of the 1st International Workshop on Model Comparison in Practice, IWMCP '10, pages 11--20, New York, NY, USA, 2010. ACM. Google ScholarDigital Library
- M. van den Brand, Z. Protić, and T. Verhoeff. Generic tool for visualization of model differences. In Proceedings of the 1st International Workshop on Model Comparison in Practice, IWMCP '10, pages 66--75, New York, NY, USA, 2010. ACM. Google ScholarDigital Library
- M. van den Brand, Z. Protić, and T. Verhoeff. Rcvdiff - a stand-alone tool for representation, calculation and visualization of model differences. 2010. ME 2010 - International Workshop on Models and Evolution.Google Scholar
- Z. Xing and E. Stroulia. UMLDiff: an algorithm for object-oriented design differencing. In Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering, ASE '05, pages 54--65, New York, NY, USA, 2005. ACM. Google ScholarDigital Library
Index Terms
- Assessing the quality of model-comparison tools: a method and a benchmark data set
Recommendations
On the importance of model comparison tools for the automatic evaluation of the correctness of model transformations
IWMCP '11: Proceedings of the 2nd International Workshop on Model Comparison in PracticeThe transformation judge is a novel system for the automatic evaluation and comparison of model transformations that have been submitted as solutions for common model transformation tasks. Its most important feature is the correctness check that is done ...
Generic tool for visualization of model differences
IWMCP '10: Proceedings of the 1st International Workshop on Model Comparison in PracticeModel comparison includes three major concerns: presentation, calculation, and visualization of model differences. In this paper we address the concern of visualization of model differences in the context of model configuration management systems. Since ...
Fine-grained metamodel-assisted model comparison
IWMCP '10: Proceedings of the 1st International Workshop on Model Comparison in PracticeIn this paper we consider two major concerns in the process of comparing two models -- representation and calculation of model differences.
Based on previous work, we adopt a set of requirements which a difference representation should satisfy in order ...
Comments