Level of detail in UML models and its impact on model comprehension: A controlled experiment
Introduction
The UML (Unified Modeling Language) is the de facto standard for modeling software systems in industry. Its adoption in industry has become more prominent since the introduction of MDA (Model-driven Architecture), in which UML is one of the key foundation [4]. Apart from the use of UML for MDA or other forms of model-driven development, we have seen a great amount of evidence where UML is still prominently used in conventional manners: UML is mainly used for architecting solution, communicating design decisions, and detailed specification for implementation; and not so much for automatically generating implementation code.
While the use of UML models for automatic code generation generally requires certain level of formality in the models, in conventional model-driven development designers have more freedom to choose the level of formality, styles, and rigor in modeling a system. This freedom consequently leads to the use of various styles and rigor in UML modeling.
In practice, the use of different styles and rigor in modeling can manifest in various form, which includes the use of varying degree of completeness, level of detail, and proportion in the models [3]. In this respect, we aim to explore the effects of using different styles and rigor in modeling on the comprehensibility of the models.
Despite the potential effect of using different styles and rigor in modeling on model comprehension, there has been little study conducted to investigate the issue. Furthermore, many of the previous works that investigated UML model comprehension have been mainly looking at UML diagram visualization or layout (e.g., [5], [6]). Therefore, in this paper we look into level of detail (LoD) in UML models and investigate its impact on model comprehension. In a controlled experiment involving two independent groups of graduate students, we explored the effects of using different levels of LoD in a UML model of a library system on model comprehension. The experiment shows results that confirm the significant effect of LoD in UML models on model comprehension.
The objective of the experiment according to the GQM template [7]:
Analyze level of detail in UML models
for the purpose of investigating its impact
with respect to model comprehension
from the perspective of the researcher
in the context of masters students at the Technology University (TU) Eindhoven
Note that in this paper we do not make a distinction between the term model and diagram. When we say model, we actually refer to a UML model that is visualized by means of any UML diagram types (e.g., class diagram). Therefore, in this paper we use the term model and diagram interchangeably.
The rest of this paper is organized as follows: in Sections 2 Related works, 3 Level of detail we discuss related works and the definition of level of detail, respectively. In Section 4 we discuss experiment planning. In Section 5 Experiment operation, 6 Data analysis and interpretation we discuss experiment operation and data analysis, respectively. Finally, in Section 7 we provide discussion on the results, and in Section 8 we draw some conclusions and outline future work.
Section snippets
Related works
Many studies that looked into UML model comprehensibility have been primarily looking at the layout or visualization aspects of UML models. The work of Purchase et al. reported in [6] for instance, investigated the impact of class diagram notations on model comprehension. In their work, two visualizations of several class diagram notations (e.g., inheritance direction, inheritance arcs, associations cardinality) were applied as treatments to a UML model. The results of their study reveal that
Level of detail
LoD in UML models is defined as the amount of information that is used to represent a modeling element. For example, the modeling element ’message in a sequence diagram’ may be represented by any of the following amounts of information: an informal label, a label that represents a method name, a label that is a method name plus the parameter list. Likewise, in modeling class diagrams, many syntactical features are available to increase the LoD: class attributes and operations, association
Experiment planning
In this section we discuss how the experiment was designed, which include the subjects of the experiment, variables in the experiments, and the types of treatments received by the subjects. Furthermore, all instruments that were used in the experiment will be discussed.
Experiment operation
The experiment was conducted in one day, and started at nine o’clock in the morning. We allocated 120 min to prepare and execute the experiment. Although there was no strict time limitation in performing the experiment tasks, the subjects were advised to finish the whole tasks within 90 min.
We started the experiment by assigning each subject to the treatment groups randomly, as discussed in Section 4.4.1. Subsequently, we briefed the subjects on how to work on the experimental tasks. The
Data analysis and interpretation
In this section we provide the results of the experiment. First, we discuss the analysis procedure. Subsequently, the results of the main statistical analyses will be discussed.
Discussion
In this section we provide further discussions about the results and we identify their implications on both research and practice. Additionally, we discuss validity threats to this study.
Conclusion and future work
In this paper we report our empirical investigation into the impact of level of detail (LoD) in UML models on model comprehension, which was measured in terms of comprehension correctness and comprehension efficiency. This study was based on an experimental study using 53 graduate students majoring computer science at Technology University Eindhoven, The Netherlands. Having applied two versions of a UML model with different LoD to two independent groups, we have found that the group receiving
Acknowledgements
This work was accomplished under the FINESSE (des7015) project supported by the STW (Stichting Technische Wetenschappen), The Netherlands. I would like to thank the students of Software Architecting 2007 at TU/e who have participated in the experiment. I also thank the editors of this Special Issue on Quality in Modeling, as well as the anonymous reviewers for their constructive feedbacks that help improve this paper. Finally, I would like to express my appreciation to Dr. Michel R.V. Chaudron
References (24)
- et al.
Evaluation of the comprehension of the dynamic modeling in UML
Information and Software Technology
(2004) - et al.
Quality and comprehension of UML interaction diagrams – an experimental comparison
Information and Software Technology
(2005) - et al.
Empirical assessment of using stereotypes to improve comprehension of UML models: a set of experiments
Journal of Systems and Software
(2006) - et al.
In practice: UML software architecture and design description
IEEE Software
(2006) - et al.
Managing the quality of UML models in practice
- et al.
A survey into the rigor of UML use and its perceived impact on quality and productivity
- Object Management Group, The Unified Modeling Language....
- et al.
On evaluating the layout of UML class diagrams for program comprehension
- et al.
UML class diagram syntax: an empirical study of comprehension
- et al.
Experimentation in Software Engineering: An Introduction
(2000)
Empirical assessment of UML static object diagrams
International Workshop on Program Comprehension
An experimental investigation of formality in UML-based development
IEEE Transactions on Software Engineering
Cited by (61)
A level-of-details framework for representing occupant behavior in agent-based models
2022, Automation in ConstructionCitation Excerpt :The selection of a particular LoD was based on factors such as mesh simplification and specification of the number of polygons per LoD to reduce the geometrical complexity [55]. Since then, the concept of LoD has been adapted by researchers for various domains such as software development [56,57] virtual reality systems [58], geoinformatics [59], and building information modeling (BIM) [60]. It is important to mention that the LoD should not be confused with a similar concept of “Level of Development” in BIM that determines the degree to which the element's geometry and attached information should be delivered by the various stakeholders during the design and construction stages [61].
An empirical evaluation of the use of models to improve the understanding of safety compliance needs
2020, Information and Software TechnologyCitation Excerpt :Experiments on the comprehension of software code with UML analysis models [37] suggest that the comprehension is not improved when using models. Nonetheless, a higher level of detail in the models seems to ease the understanding of a system [38,39]. The study of the inclusion of stereotypes in UML diagrams [40,41] has shown that it improves model comprehension.
Artifact-based vs. human-perceived understandability and modifiability of refactored business processes: An experiment
2018, Journal of Systems and SoftwareInvestigating comprehension and learnability aspects of use cases for software specification problems
2017, Information and Software TechnologyCitation Excerpt :However, the results of the study might got influenced by the amount of training provided to the subjects participated in the study. Nugroho et al. [46] conducted a study to investigate the understandability of use case models having varied Level of Detail (LoD) in the development phase. The evaluations were made by judging the correctness and efficiency of the subjects’ developed use case models.
Investigating Factors Influencing Students' Assessment of Conceptual Models
2023, ACM International Conference Proceeding Series