On the intuitive comprehensibility of contribution links in goal models: an experimental study

Goal models have long been considered to be useful tools for representing and analyzing complex decision problems in various stages of the software development lifecycle. Their usefulness for such tasks lies in their ability to compactly represent large numbers of alternative solutions to requirements problems and to capture the impact of each solution to high-level qualities of interest. In this way, goal models allow identification of optimal choices with respect to such quality priorities. To represent the impact of solutions to qualities, goal models utilize contribution links, a special diagrammatic modeling construct. Researchers of goal modeling languages have introduced various ways to visualize the particular construct and to define formal semantics for it. However, there is little evidence that, during actual use, the proposed visualizations evoke a way of performing diagrammatic inferences that is consistent with the corresponding formal semantics. We conduct an experimental study aimed at comparing two visualization choices for contribution links, symbolic versus numeric, with respect to their ability to evoke inferences that are consistent with their formal semantics. The experiment also explores if individual psychological differences including trait cognitive style, mathematics anxiety, and mental math ability, affect this evocation. Participants are asked to make a series of diagrammatic inferences over two sets of goal models each adopting one of the two competing visualization formats, symbolic vs. numeric. We measure accuracy, that is, the level to which participant decisions are consistent with the formal semantics proposed for each visualization, and investigate the effect to accuracy of various relevant factors – visualization choice, individual differences, and reasoning method adopted. Findings include that most participants adopt specific inference rules instead of working intuitively, that such rules are more consistent with the formal semantics in numeric models, that the utilization of negative contributions and notions of goal denial may hinter accuracy, and that the individual differences considered do not play an important role in either accuracy or choice of inference method.