Level of detail in UML models and its impact on model comprehension: A controlled experiment

doi:10.1016/j.infsof.2009.04.007

Information and Software Technology

Volume 51, Issue 12, December 2009, Pages 1670-1685

https://doi.org/10.1016/j.infsof.2009.04.007 Get rights and content

Abstract

Previous studies have shown that the style and rigor used in UML models vary widely across software projects [1], [2], [3]. However, notwithstanding the varying use of styles and rigor, little research has been conducted to investigate the drivers and effects of using different styles and rigor in modeling on software development. In this paper, we evaluate Level of Detail (LoD) in UML models as a form of style and rigor in UML modeling. Using a UML model of a library system, we experimentally investigate the impact of LoD on model comprehension. More specifically, we explore whether LoD in UML models affects the correctness and efficiency in comprehending UML models. Using two independent groups of graduate students majoring in computer science, we performed a controlled experiment. The results of the experiment confirm the significant effect of LoD in UML models on model comprehension. Nevertheless, replication of this study is necessary, especially in contexts that involve professional software engineers, to improve the generalizability of the results.

Introduction

The UML (Unified Modeling Language) is the de facto standard for modeling software systems in industry. Its adoption in industry has become more prominent since the introduction of MDA (Model-driven Architecture), in which UML is one of the key foundation [4]. Apart from the use of UML for MDA or other forms of model-driven development, we have seen a great amount of evidence where UML is still prominently used in conventional manners: UML is mainly used for architecting solution, communicating design decisions, and detailed specification for implementation; and not so much for automatically generating implementation code.

While the use of UML models for automatic code generation generally requires certain level of formality in the models, in conventional model-driven development designers have more freedom to choose the level of formality, styles, and rigor in modeling a system. This freedom consequently leads to the use of various styles and rigor in UML modeling.

In practice, the use of different styles and rigor in modeling can manifest in various form, which includes the use of varying degree of completeness, level of detail, and proportion in the models [3]. In this respect, we aim to explore the effects of using different styles and rigor in modeling on the comprehensibility of the models.

Despite the potential effect of using different styles and rigor in modeling on model comprehension, there has been little study conducted to investigate the issue. Furthermore, many of the previous works that investigated UML model comprehension have been mainly looking at UML diagram visualization or layout (e.g., [5], [6]). Therefore, in this paper we look into level of detail (LoD) in UML models and investigate its impact on model comprehension. In a controlled experiment involving two independent groups of graduate students, we explored the effects of using different levels of LoD in a UML model of a library system on model comprehension. The experiment shows results that confirm the significant effect of LoD in UML models on model comprehension.

The objective of the experiment according to the GQM template [7]:

Analyze level of detail in UML models
for the purpose of investigating its impact
with respect to model comprehension
from the perspective of the researcher
in the context of masters students at the Technology University (TU) Eindhoven

Note that in this paper we do not make a distinction between the term model and diagram. When we say model, we actually refer to a UML model that is visualized by means of any UML diagram types (e.g., class diagram). Therefore, in this paper we use the term model and diagram interchangeably.

The rest of this paper is organized as follows: in Sections 2 Related works, 3 Level of detail we discuss related works and the definition of level of detail, respectively. In Section 4 we discuss experiment planning. In Section 5 Experiment operation, 6 Data analysis and interpretation we discuss experiment operation and data analysis, respectively. Finally, in Section 7 we provide discussion on the results, and in Section 8 we draw some conclusions and outline future work.

Section snippets

Related works

Many studies that looked into UML model comprehensibility have been primarily looking at the layout or visualization aspects of UML models. The work of Purchase et al. reported in [6] for instance, investigated the impact of class diagram notations on model comprehension. In their work, two visualizations of several class diagram notations (e.g., inheritance direction, inheritance arcs, associations cardinality) were applied as treatments to a UML model. The results of their study reveal that

Level of detail

LoD in UML models is defined as the amount of information that is used to represent a modeling element. For example, the modeling element ’message in a sequence diagram’ may be represented by any of the following amounts of information: an informal label, a label that represents a method name, a label that is a method name plus the parameter list. Likewise, in modeling class diagrams, many syntactical features are available to increase the LoD: class attributes and operations, association

Experiment planning

In this section we discuss how the experiment was designed, which include the subjects of the experiment, variables in the experiments, and the types of treatments received by the subjects. Furthermore, all instruments that were used in the experiment will be discussed.

Experiment operation

The experiment was conducted in one day, and started at nine o’clock in the morning. We allocated 120 min to prepare and execute the experiment. Although there was no strict time limitation in performing the experiment tasks, the subjects were advised to finish the whole tasks within 90 min.

We started the experiment by assigning each subject to the treatment groups randomly, as discussed in Section 4.4.1. Subsequently, we briefed the subjects on how to work on the experimental tasks. The

Data analysis and interpretation

In this section we provide the results of the experiment. First, we discuss the analysis procedure. Subsequently, the results of the main statistical analyses will be discussed.

Discussion

In this section we provide further discussions about the results and we identify their implications on both research and practice. Additionally, we discuss validity threats to this study.

Conclusion and future work

In this paper we report our empirical investigation into the impact of level of detail (LoD) in UML models on model comprehension, which was measured in terms of comprehension correctness and comprehension efficiency. This study was based on an experimental study using 53 graduate students majoring computer science at Technology University Eindhoven, The Netherlands. Having applied two versions of a UML model with different LoD to two independent groups, we have found that the group receiving

Acknowledgements

This work was accomplished under the FINESSE (des7015) project supported by the STW (Stichting Technische Wetenschappen), The Netherlands. I would like to thank the students of Software Architecting 2007 at TU/e who have participated in the experiment. I also thank the editors of this Special Issue on Quality in Modeling, as well as the anonymous reviewers for their constructive feedbacks that help improve this paper. Finally, I would like to express my appreciation to Dr. Michel R.V. Chaudron

References (24)

M.C. Otero et al.
Evaluation of the comprehension of the dynamic modeling in UML
Information and Software Technology
(2004)
C. Glezer et al.
Quality and comprehension of UML interaction diagrams – an experimental comparison
Information and Software Technology
(2005)
M. Staron et al.
Empirical assessment of using stereotypes to improve comprehension of UML models: a set of experiments
Journal of Systems and Software
(2006)
C.F.J. Lange et al.
In practice: UML software architecture and design description
IEEE Software
(2006)
A. Nugroho et al.
Managing the quality of UML models in practice
A. Nugroho et al.
A survey into the rigor of UML use and its perceived impact on quality and productivity
Object Management Group, The Unified Modeling Language....
K. Wong et al.
On evaluating the layout of UML class diagrams for program comprehension
H.C. Purchase et al.
UML class diagram syntax: an empirical study of comprehension
C. Wohlin et al.
Experimentation in Software Engineering: An Introduction
(2000)

M. Torchiano

Empirical assessment of UML static object diagrams

International Workshop on Program Comprehension

(2004)

L.C. Briand et al.

An experimental investigation of formality in UML-based development

IEEE Transactions on Software Engineering

(2005)

Cited by (61)

A level-of-details framework for representing occupant behavior in agent-based models
2022, Automation in Construction
Citation Excerpt :
The selection of a particular LoD was based on factors such as mesh simplification and specification of the number of polygons per LoD to reduce the geometrical complexity [55]. Since then, the concept of LoD has been adapted by researchers for various domains such as software development [56,57] virtual reality systems [58], geoinformatics [59], and building information modeling (BIM) [60]. It is important to mention that the LoD should not be confused with a similar concept of “Level of Development” in BIM that determines the degree to which the element's geometry and attached information should be delivered by the various stakeholders during the design and construction stages [61].
Agent-based modeling is an advanced computational technique capable of representing complex and dynamic processes of human behavior in building performance simulation. Though the agent-based approach supports diverse applications concerning human behavior modeling within the built environment, there is no consensus on the optimal amount of information or level of granularity needed for occupant information representation. This paper attempts to formalize the level of details (LoD) needed for occupant behavior representation in agent-based environments. A novel framework, grounded on the concept of LoD, is proposed to select the required details in representing occupants in agent-based models. Ten attributes related to occupants' presence, movement, behavioral processes, and repertoire are considered to define the LoD. The framework identifies use case parameters as the guiding principle and allows a hybrid approach for selecting varying degrees of occupant attributes to serve the purpose of simulation. A discussion on the pertinence of different occupant behavior LoDs in relation to the desired objective and simulation context is also presented. The study intends to support the occupant behavior research by advancing agent-based occupant modeling in building performance simulation.
An empirical evaluation of the use of models to improve the understanding of safety compliance needs
2020, Information and Software Technology
Citation Excerpt :
Experiments on the comprehension of software code with UML analysis models [37] suggest that the comprehension is not improved when using models. Nonetheless, a higher level of detail in the models seems to ease the understanding of a system [38,39]. The study of the inclusion of stereotypes in UML diagrams [40,41] has shown that it improves model comprehension.
Critical systems in application domains such as automotive, railway, aerospace, and healthcare are required to comply with safety standards. The understanding of the safety compliance needs specified in these standards can be difficult from their text. A possible solution is to use models.
We aim to evaluate the use of models to understand safety compliance needs.
We have studied the effectiveness, efficiency, and perceived benefits in understanding these needs, with models and with the text of safety standards, by means of an experiment. The standards considered are DO-178C and EN 50128. We use SPEM-like diagrams to graphically represent the models.
The mean effectiveness of 20 undergraduate students in understanding the needs and the mean efficiency were higher with models (22% and 38%, respectively), and the difference is statistically significant (p-value ≤ 0.02). Most of the students agreed upon the ease of understanding the structure of safety compliance needs with models when compared to the text, but on average, the students were undecided about whether the models are easy to understand or easier to understand than the text.
The results allow us to claim that the use of models can improve the understanding of safety compliance needs. Nonetheless, there seems to be room for improvement in relation to the perceived benefits. It must be noted that our conclusions may differ if the subjects were experienced practitioners.
Artifact-based vs. human-perceived understandability and modifiability of refactored business processes: An experiment
2018, Journal of Systems and Software
Business processes modeling has proven to be effective and reverse engineering techniques with which to recover business process models when they are missing or outmoded have therefore emerged. Regrettably, these techniques often lead to models with quality flaws and consequently to models with low levels of understandability and modifiability. Refactoring has been widely used to deal with such flaws, altering the internal structure of models while preserving their semantics. There are several studies concerning how understandability and modifiability are affected by refactoring in terms of several artifact-based measures. However, there is little evidence regarding how refactoring affects quality in terms of human-perceived measures. The goals of this paper are, therefore: to collect further empirical evidence about the influence of refactoring on understandability and modifiability of business process models and to investigate the correlation between artifact-based understandability and modifiability and human-perceived ones. The obtained results are not trivial and show that business process obtained by means of reverse engineering has recurrent quality flaws, and the understandability and modifiability of business process models cannot be assessed by using artifact-based measures only. Human-perceived measures need to be taken in to consideration in order to have a more accurate evaluation.
Investigating comprehension and learnability aspects of use cases for software specification problems
2017, Information and Software Technology
Citation Excerpt :
However, the results of the study might got influenced by the amount of training provided to the subjects participated in the study. Nugroho et al. [46] conducted a study to investigate the understandability of use case models having varied Level of Detail (LoD) in the development phase. The evaluations were made by judging the correctness and efficiency of the subjects’ developed use case models.
Context: Availability of multiple use case templates to document software requirements inevitably requires their characterization in terms of their relevance, usefulness, and the degree of the formality of the expressions.
Objective: This paper reports two experimental studies that separately investigate two usability aspects, namely the comprehension and the learnability of use case templates for software specification problems.
Method: We judged the comprehension aspect by evaluating the subjects’ understanding of the requirements, specified in eight different use case templates, and the ease with which the changes were made by them in the requirement specifications. The learnability aspect was judged by assessing the completeness, the correctness, and the redundancy of the use case specifications developed by the subjects using these eight use case templates for three software specification problems.
Results: Our results suggested that the Kettenis’s use case template was found to be significantly more understandable, and the templates by Tiwari, Yue and Somé were found to be significantly more flexible to adapt to the changes. On the learnability aspect, the way we formulated it, we found different templates to be more complete (Kettenis), correct (Somé), and non-redundant (Tiwari).
Conclusion: The specifications documented using a more detailed use case template with an intermediate degree of formality can be more comprehensible and flexible to adapt to the required changes to be made in the specification. A more formal template seems to enhance the learnability as well.
The Impact of Code Bloat on Genetic Program Comprehension: Replication of a Controlled Experiment on Semantic Inference
2023, Mathematics
Investigating Factors Influencing Students' Assessment of Conceptual Models
2023, ACM International Conference Proceeding Series

View all citing articles on Scopus

View full text

Level of detail in UML models and its impact on model comprehension: A controlled experiment

Abstract

Introduction

Section snippets

Related works

Level of detail

Experiment planning

Experiment operation

Data analysis and interpretation

Discussion

Conclusion and future work

Acknowledgements

Information and Software Technology

Information and Software Technology

Journal of Systems and Software

In practice: UML software architecture and design description

IEEE Software

Managing the quality of UML models in practice

A survey into the rigor of UML use and its perceived impact on quality and productivity

On evaluating the layout of UML class diagrams for program comprehension

UML class diagram syntax: an empirical study of comprehension

Experimentation in Software Engineering: An Introduction

Empirical assessment of UML static object diagrams

International Workshop on Program Comprehension

An experimental investigation of formality in UML-based development

IEEE Transactions on Software Engineering