Comprehension of Procedural Visual Business Process Models

First, we summarize studies that challenge the assumption concerning whether using a modeling notation for process descriptions instead of alternative representation paradigms to maximize model comprehension is always the best choice. Ottensooser et al. (2012) finds that process models improve comprehension accuracy more than written use cases and that they increase comprehension accuracy for users trained in process models, but not for users who have no prior training. In line with this result, Rodrigues et al. (2015) report that experienced users perform better in comprehension tasks with BPMN models than with textual process descriptions, while there was no difference among inexperienced users. In summary, there is a moderate level of evidence of the effect of representation in these two studies – that is, that process models are superior to textual descriptions – since the effect was not significant for all user groups. Users also seem to prefer BPMN diagrams over structured text and textual descriptions when the goal is to understand a process (Figl and Recker 2016).

Regarding other types of representation, users report a slight preference for diagrams with icons attached to activity symbols that express the semantic meaning of the process activities over diagrams without icons (Figl and Recker 2016). One other study, with relatively small sample size, shows that procedural (imperative) process models aid comprehension more than declarative models do (Pichler et al. 2012), perhaps because procedural process models explicate the sequence of activities, while this information is hidden in declarative models. In addition, BPMN3D, a version of BPMN that uses dimensions for data objects is evaluated as best for comprehension, followed by Bubble, which visualizes process tasks as bubbles, and then other uncommon visualization concepts of process models (Hipp et al. 2014).

Several studies compare the composite effects of notations. BPMN, EPC, UML AD, and YAWL were investigated in more than one study, so results can be compared to some degree. These studies show that vBPMN, a configuration extension of BPMN, is easier to comprehend than C-YAWL, a configuration extension of YAWL (Döhring et al. 2014). This finding is also reflected in Figl et al. (2013a), who isolate the perceptual discriminability deficiencies of symbols in YAWL and demonstrate that these symbols lowered comprehension accuracy below that of UML AD and BPMN. Sarshar and Loos (2005) find that EPC scores better than Petrinets in helping users understand XOR, but their evidence of a difference in subjective difficulty is weak. Recker and Dreiling (2007, 2011) compare BPMN and EPC and find no evidence of higher comprehension accuracy of BPMN, although participants performed better with BPMN when the task was to fill in missing words in a cloze test about the process. Two other studies, each of which reports only descriptive statistics, thereby offering only weak evidence, compare EPC to other notations. Sandkuhl and Wiebring (2015) award eEPC – that is, extended EPC using additional symbols – the highest absolute score in subjective perception of notation and the second-highest score in comprehension, following UML AD; however, Weitlaner et al. (2013) find that eEPC is less well understood than either UML AD or BPMN. Two additional studies offer moderate evidence of the lower comprehension accuracy of EPC in comparison to other notations: In the study by Jošt et al. (2016) UML AD statistically significantly outperformed both EPC and BPMN in some cases, although results were not consistent and notations were not presented in consistent flow directions, compromising the study’s validity. This result is in line with Figl et al. (2013a), who show that semiotic clarity deficiencies in EPC reduce comprehension accuracy below that of UML AD and BPMN.

Results on communication-oriented flow diagrams and functional flowcharts are mixed. One study reports that communication-oriented flow diagrams have higher perceived ease of understanding (Kock et al. 2009), but this result is not confirmed in a second study (Kock et al. 2008).

Several notations have been investigated only in a single study, so results can only provide an overview of the notations that have been the subjects of empirical evaluation. Natschläger (2011) provide weak evidence in the form of descriptive statistics for higher comprehension of “deontic” BPMN, which expresses deontic logic like obligations, alternatives and permissions, in comparison to regular BPMN. Similarly, Recker et al. (2005) find weak evidence without using statistical tests that configurable EPCs (c-EPCs) are perceived as more useful than standard EPCs. In Stitzlein et al. (2013), the domain-specific health process notation (HPN) outperforms BPMN in complex tasks, but the effect is reversed for simple tasks.

Some studies investigate notational characteristics like aesthetic symbol design, semantic transparency, perceptional discriminability, and the use of gateways¹ as isolated factors, so their results can be generalized beyond specific notations. These studies do not always adhere to exact syntactic restrictions of modeling notations in their experimental material but focus instead on varying specific notational characteristics. This kind of isolation makes it easier to achieve internal validity and to determine what causes an effect on model comprehension. When comparing notations as a whole, models differ based on many variables (e.g., different numbers of symbols), so it is difficult to suggest how to improve a notation.

For instance, Recker (2013) demonstrates that the use of gateway constructs benefits understanding and explains this effect as resulting from higher perceptual discriminability. Perceptual pop-out and discriminability show their relevance for comprehension accuracy and perceived cognitive load in (Figl et al. 2013a, b). In contrast, characteristics like semantic transparency and aesthetics, which relate to later stages of perceptional processing, affect perceived cognitive load but not comprehension accuracy directly (Figl et al. 2013a, b).

Kock et al. (2009) report that subjectively experienced “ease of generating the models” in a notation is also positively associated with model comprehension.

Reijers et al. (2011b, p. 10) show that modularization is positively related to model comprehension, explaining that modularization in large models “shields the reader from unnecessary information.” In contrast, Turetken et al. (2016) report strong evidence of higher comprehension accuracy for fully flattened models in local comprehension tasks that can be completed based on information in a sub-model. The type of comprehension task seems central to the investigation of decomposition, as no evidence of differences in other tasks have been found. In addition, Johannsen et al. (2014) find that low levels of violated decomposition principles, as described in Wand and Weber’s decomposition model (Wand and Weber 1995), positively influence comprehension.

Another stream of research on secondary notation is concerned with how to incorporate Gestalt theory in model design to make it easier for humans to recognize related elements as belonging together. Gestalt theory deals with principles associated with how humans perceive whole figures instead of simpler, separate elements (Wagemans et al. 2012). Several visual variables can be employed in this context; studies have indirectly investigated the principle of “common region” (Palmer 1992) by researching the use of swim lanes or visual grouping of sub-processes and the principle of “similarity” using colors and syntax highlighting. Although, according to the principle of “common region,” swim lanes are hypothesized to benefit comprehension, Bera (2012) removes the lanes in the “no swim lane” condition and finds evidence of an effect on time taken and performance in problem-solving but none on comprehension. Jeyaraj and Sauter (2014) investigate a “no swim lane” condition in which actors are completely cut from the side of the diagram and redundantly inserted in each activity symbol and find no evidence of a cognitive advantage of swimlanes for process models. In fact, some items on external actors were easier to answer in the “no swim lane” condition.

Turetken et al. (2016) use an experimental condition in which sub-processes are visually grouped in a common region by means of background colors but find that this representation does not significantly enhance comprehension accuracy. On the contrary, the version without visual grouping was rated as easier to understand. Perhaps this result was due to participants’ not needing information about how the process was separated into sub-processes to answer the comprehension tasks.

Color coding with bright colors lowers perceived difficulty for participants from a Confucian culture, but not for participants from a Germanic culture; this result may be related to a culture-dependent preference for bright colors in Asia, although color in general is not related to model comprehension in this study (Kummer et al. 2016). In contrast, Reijers et al. (2011a) find that syntax highlighting with colors for matching gateway pairs is positively related to novices’ model comprehension, as it helped them identify relevant patterns of matching gateways, but the study finds no evidence that syntax highlighting improves experts’ performance. The inconsistency between the findings of Kummer et al. (2016) and Reijers et al. (2011a) may occur because of differences in the kinds of information that the studies highlight with color. For instance, Kummer et al. (2016) color activity symbols, while Reijers et al. (2011a) color-code gateway pairs that belong to each other, information that is more relevant to comprehension tasks. Petrusel et al. (2016) use task-specific color highlighting of regions of model elements that are relevant to a comprehension task and find no evidence of reduced comprehension accuracy, although color highlighting lowers mental effort (measured by fixations and fixation durations in eye-tracking results) and time taken.

Size measures relate to the number of elements, including arcs, gateway nodes, event nodes, and task nodes (Mendling et al. 2012a).

Two studies with large sample sizes find that model size operationalized as the number of nodes is negatively related to model comprehension accuracy (Recker 2013; Sánchez-González et al. 2010), but Mendling and Strembeck (2008) report no effect of the number of nodes on comprehension accuracy. Sánchez-González et al. (2010) find that another measure of model size, the length of the longest path from a start node to an end node, lowers comprehension, while Mendling and Strembeck (2008) find no evidence of such an effect. Relationships between higher size and lower comprehension accuracy are in line with cognitive load theory, as these metrics reflect higher intrinsic cognitive load. Mendling and Strembeck’s (2008) non-significant results may also be due to a lower variance in the numbers of nodes and diameters in their set of models, as they do not mention a systematic variation of these variables, and all models fit on A4-size pages. Another possible interpretation is that only a few elements are relevant for each comprehension task, not the whole model, so the choice of comprehension tasks could influence the results.

Aguilar et al. (2008) report weak evidence of negative correlations between model comprehension and metrics that are related to events (number of events, intermediate events, and number of sequence flows from events). There was further weak evidence for the relevance of the metrics of number of gateways (Sánchez-González et al. 2012) and exclusive decisions (Aguilar et al. 2008), while the number of OR joins (Reijers and Mendling 2011) did not have a significant effect.

Several studies investigate the metrics related to models’ structuredness. Structuredness denotes that, “for every node with multiple outgoing arcs (a split) there is a corresponding node with multiple incoming arcs (a join) such that the subgraph between the split and the join forms a single-entry-single exit (SESE) region” (Dumas et al. 2012, p. 33). Mendling and Strembeck (2008, p. 147), who calculate the metric as “one minus the number of nodes in structured blocks divided by the number of nodes,” find no significant correlation between it and comprehension accuracy. Dumas et al. (2012) find that a comparable metric results in inconsistent results, as structuredness reduces comprehension for two models and heightens comprehension in two other models, which leads the authors to conclude that structuring might be beneficial only when it does not increase the number of gateways.

Research has also identified three other metrics related to model structure of the model as potentially relevant factors. First, Sánchez-González et al. (2010) find that the maximum nesting of structured blocks in a process model is a negative influence factor, which finding is supported by research showing that interactivity of the process activities included in a comprehension task, measured via the process structure’s tree distance metric, is negatively related to comprehension (Figl and Laue 2015). Second, Mendling and Strembeck (2008) find strong evidence that the relationship between a model’s comprehensibility and its cut-vertices, the absence of which would separate the model into two parts, is such that higher separability as a global metric is associated with higher overall comprehension accuracy. However, experiments that measure the presence of cut-vertices for each comprehension task separately find no evidence of such an effect (Figl and Laue 2015). Third, there is weak evidence of the ability of the degree to which a model is constructed out of pure sequences of tasks to heighten comprehensibility (Sánchez-González et al. 2010).

Many studies investigate control structures using measurements beyond simple counting of gateway elements. Studies find weak evidence of the relevance of the metrics of control-flow complexity (Sánchez-González et al. 2012) and sequence flows looping (Aguilar et al. 2008), but no significant effect of concurrency (Mendling and Strembeck 2008). We also rated as weak the evidence of a negative effect on the comprehensibility of gateway mismatch (measured as the sum of gateway pairs that do not match with each other, such as when an AND-split is followed up by an XOR-join), based on Sánchez-González et al. (2010, 2012). Reijers and Mendling (2011) report a non-significant correlation between gateway mismatch and model comprehension. The frequency with which different types of gateways are used in a model (gateway heterogeneity) is investigated in four studies, with two finding no evidence of a correlation (Reijers and Mendling 2011; Mendling and Strembeck 2008), one finding weak evidence of a correlation (Sánchez-González et al. 2012), and the fourth reporting strong evidence of a significant correlation (Sánchez-González et al. 2010).

Seven studies investigate comprehension of control structures by comparing the difficulty of comprehension tasks. At first view, results seem contradictory, as some authors report strong evidence that order/sequence tasks are easiest and repetition/loops tasks are most difficult (Figl and Laue 2011, 2015; Laue and Gadatsch 2011), while Melcher et al. (2010; Melcher and Seese 2008) find that order tasks are most difficult and repetition tasks are easier. While the counterintuitive result from Melcher and Seese (2008) could be explained by the low number of participants (9) for the order task, the second study’s sample was large, and the authors (Melcher et al. 2010) themselves note that order’s being the most difficult is “not directly intuitive” and that it is the only normally distributed variable in their study. According to Weitlaner et al. (2013), order and repetition are easier to comprehend than concurrency, but they present only descriptive statistics. Another study reveals that tasks related to “OR” routing symbols are more difficult than those related to “AND” and “XOR” (Sarshar and Loos 2005), without giving exact numbers. It is not possible to assess whether different studies’ “rankings” of control structures differ statistically, but the differences among studies may also be related to question wording as an influence factor in comprehension accuracy (Laue and Gadatsch 2011). Drawing upon these insights, Figl and Laue (2015) base their analysis on a qualitative coding of the control-flow patterns that must be understood in order to answer a comprehension task correctly, rather than on the task’s wording. Their study finds that order is easiest and repetition most difficult in terms of comprehension accuracy, while concurrency and exclusiveness patterns have a medium level of difficulty.

Some studies look at metrics that relate routing paths (arcs) to model elements. A higher average number of a gateway node’s incoming and outgoing arcs and a higher ratio between the number of arcs in a process model and the theoretically maximum number of arcs both have a negative effect on comprehension accuracy (Sánchez-González et al. 2010, 2012). These results are supported by Reijers and Mendling (2011, p. 9), who comment that “the two factors which most convincingly relate to model understandability both relate to the number of connections in a process model, rather than, for example, the generated state space.” There is additional weak evidence that a decision node’s higher maximum number of incoming and outgoing arcs is related to lower comprehension, but the extent to which all nodes in a model are connected to each other is not reported to be a significant influence factor (Reijers and Mendling 2011).

Only one study, Heggset et al. (2015), has been published on syntax rules. The authors report that improving models by means of syntactic guidelines increases comprehension accuracy. As the guidelines include more than twenty steps, such as those related to symbol choice and routing behavior, and the study reports results for only one model, it is not possible to derive from this study which syntactic guidelines are especially important.

Mendling and Strembeck (2008) also examine soundness in EPCs, which can be violated by, for instance, incorrect insertion of OR-joins, but find no evidence for a relationship between such soundness and comprehension.

Studies have found no evidence of the effect of domain knowledge on model comprehension (Bera 2012; Turetken et al. 2016; Recker and Dreiling 2007; Recker et al. 2014). There are several possible explanations for this influence factor’s lack of statistical significance. For example, researchers have held domain knowledge constant and have chosen homogenous groups of participants (e.g., students) to avoid bias from high levels of familiarity with the domain in their experiments (e.g., Recker and Dreiling 2007), making an effect on model comprehension unlikely because of the low variation in the variable. Studies have also used self-reported scales to measure “perceived” domain knowledge, which might not be able to capture knowledge in the domain, or the “domain-specificity” of the models could have been low.

Measures of modeling experience and familiarity vary significantly in the studies. Categorizing these measures is difficult because some authors distinguish between the place where experience was acquired (university versus practice) and the type of experience (formal training versus use in practice), while others use measures that overlap with these distinctions (e.g., self-rated familiarity with a notation/process modeling in general, self-rated number of processes modeled, knowledge of a notation). Therefore, we distinguish only between self-assessed experience (or familiarity) and modeling knowledge that is measured objectively.

Of the eighteen measures of modeling experience, we categorize four as showing moderate/strong evidence of an effect on model comprehension, while the rest do not report a statistically significant effect. Studies find no evidence of an effect of self-assessment of previous modeling knowledge (Johannsen et al. 2014; Reijers and Mendling 2011; Weitlaner et al. 2013; Recker and Dreiling 2007), duration of involvement with business process modeling (Mendling and Strembeck 2008), self-assessment of process modeling experience (Reijers and Mendling 2011; Turetken et al. 2016), intensity of work with process models (Mendling and Strembeck 2008), or modeling familiarity (Ottensooser et al. 2012; Recker 2013; Kummer et al. 2016) on model comprehension.

However, the research shows significant positive effects on model comprehension of the frequency of use of “flow charts” (Ottensooser et al. 2012), process model working experience (Recker and Dreiling 2011), training on modeling basics at a university or a school (Figl et al. 2013a), and model training at different universities (Reijers and Mendling 2011). Mendling et al. (2012b) report an inverse u-shaped curve as describing the relationship between modeling intensity and duration of modeling experience, on one hand, and comprehension accuracy, in which medium intensity/experience was best, on the other. This finding may explain the mixed results for experience-related variables. Another likely explanation for the low amount of evidence regarding experience measures’ effect on comprehension is that researchers are often more interested in keeping the effect of these variables constant in their study design and focusing on other influence factors instead of selecting samples with larger variance in experience and familiarity with process modeling. This explanation is also reflected in Gemino and Wand’s (2004, p. 258) warning that choosing participants with a higher level of experience, “while seemingly providing more realistic conditions, might create substantial difficulties in an experimental study.”

Eight studies use adapted versions of the process modeling knowledge test Mendling et al. (2012b) propose; of these eight studies, six find a strong positive effect on comprehension accuracy (Figl and Laue 2015; Figl and Strembeck 2015; Kummer et al. 2016; Figl et al. 2013b; Mendling and Strembeck 2008; Recker 2013), while the effect was not significant in two (Figl et al. 2013a; Recker et al. 2014). Additional research on user preferences regarding representations to understand a process shows that knowledge of conceptual modeling heightens the preference for process diagrams over structured text (Figl and Recker 2016).

Other individual factors that positively affect comprehension accuracy include higher education in general (Weitlaner et al. 2013), as academics and high-school graduates perform better than apprenticeship graduates do. Döhring et al.’s (2014) study compares students with post-docs and industry employees and finds no evidence of a difference, possibly because students had more modeling training, while seniors had more practical experience. Recker and Dreiling (2011) show that the use of participants’ native language in labels is significantly positively related to performance in filling out a cloze test on a process but find no evidence of an effect on comprehension accuracy. Cultural background (Germanic versus Confucian) also has no effect on comprehension accuracy, although participants from Germanic cultures rate the models as more difficult to understand (Kummer et al. 2016).

Research also addresses the influence of model readers’ learning styles and cognitive styles on comprehension. The sensing learning style, which characterizes learners who “prefer learning and memorizing facts from a process model bit-by-bit” (Recker et al. 2014, p. 204), and the surface learning strategy, which indicates learning by memorization, are positively related to comprehension accuracy, while abstraction ability and surface learning motive, the last of which indicates extrinsic (instead of intrinsic motivation) and low learning intensity, are negatively associated with comprehension accuracy (Recker et al. 2014). Moreover, the spatial cognitive style heightens the preference for diagrams over text, while the verbal cognitive style lowers it (Figl and Recker 2016).

Comparison of theoretical and empirical work shows that the research discusses the representation paradigm (e.g., text versus model) theoretically and subjects it to empirical evaluation. The main conclusion is that users prefer diagrammatic representations, but prior experience and training is an important precondition if they are to benefit from diagrams more than from textual representations in comprehension tasks.

The research also compares the procedural process paradigm with declarative models both conceptually and empirically.

In addition, the research validates a variety of alternative visualizations empirically, but such research might be pursued in more directions. For instance, animation and narration techniques used to increase the intuitiveness of process representation (Aysolmaz and Reijers 2016) have not been addressed empirically. Newly proposed visualization opportunities like augmenting process tasks in models with storyboard-like shots of a 3D virtual world to simulate the process (Kathleen et al. 2014) or associating activities with user stories (Trkman et al. 2016) could illustrate business activities vividly while improving comprehension. Furthermore, the effect on comprehension of using semantically oriented pictorial elements like icons and images could be assessed empirically in more detail using objective comprehension tests since existing research investigates icons only from a subjective point of view.

A high number of studies in all three article categories use or discuss BPMN as a notation for modeling processes, which reflects the establishment of BPMN as the de-facto standard for business process modeling (Kocbek et al. 2015).

The effect on comprehension of such process modeling notations as BPMN, UML AD, YAWL, EPCs, and Petri Nets is analyzed theoretically and in empirical studies, the latter of which also evaluate a variety of less common notations. While there is still room for future research to clarify their interplay, the extant empirical work makes several attempts to investigate such notational characteristics as semiotic clarity, perceptual discriminability, semantic transparency, and visual expressiveness. For example, Recker (2013) demonstrates that using gateways aids in model comprehension more than implicit splits and joins in a process do because of a perceptual discriminability effect. This finding is in agreement with hints from error analyses of process model repositories that indicate that implicit representation is often misunderstood, an indication that is also explained by low semiotic clarity because BPMN allows more than one way to model the same concept (Leopold et al. 2016). Concerning semantic transparency of BPMN, Genon et al. (2010) propose new symbols with higher levels of intuitiveness, and Leopold et al. (2016) hypothesize that throwing message events are misunderstood as passive instead of active, based on the event symbol. Future research that measures comprehension could empirically evaluate such hypotheses.

Such criteria as semiotic clarity and visual expressiveness can also be determined in expert evaluations like ontological analyses that compare potentially relevant semantic concepts and symbols, in the case of semiotic clarity, or by identifying visual variables used, in the case of visual expressiveness. However, determining the degree to which they affect comprehension is best done in user studies. There are opportunities for scholars to examine such other notational characteristics as graphic economy and restriction and the extension of a notation’s syntax and semantics.

Still more empirical work can be done on the decomposition of models and hierarchical structuring. Zugal et al. (2012) describe on theoretical grounds how decomposition could lead to two opposing effects: abstraction, which would aid comprehension, and a potential split-attention effect, which would lower comprehension. Research questions like this are central to understanding human behavior related to information acquisition in the context of process models to determine benefits of “hiding” irrelevant information in sub-processes. Turetken et al. (2016) report no evidence of increased comprehensibility from using such abstraction; on the contrary, tasks that require information from sub-processes are answered better when this information is not hidden (and, thus, no split-attention effect could occur). Future empirical research should specify a tradeoff curve between the effect of abstraction and split-attention on comprehension and determine how interactive model visualizations can support human information-seeking in larger process models. While Johannsen et al. (2014) are first to have investigated decomposition heuristics experimentally for EPCs, the literature also contains other proposals for decomposition heuristics that have not yet been looked at from an empirical point of view (e.g., Milani et al. 2016).

Some studies address dual coding and highlighting. Reijers et al. (2011a) demonstrates the potential of color use for highlighting elements or syntax structures to influence novices’ information-search behavior, as novices lack the task-specific experience to identify matching gateway patterns. Color use seems to depend on the type of information for which it is used, as it is not always beneficial (Kummer et al. 2016).

Turetken et al. (2016) investigate visual enclosure to highlight elements that belong to a sub-model, and others investigate swim lanes to group activities visually according to their actors (Bera 2012; Jeyaraj and Sauter 2014). Model readers have to use different information-search behaviors that apply to identifying actors, with and without swim lanes, both of which seem to satisfy this particular information need. When they are not directly relevant to the information need, the additional visual information on elements belonging together may even lower perceived ease of understanding (Turetken et al. 2016).

Additional studies on other aspects of dual coding, such as alignment of symbols and textual annotations and highlighting of elements, may provide additional valuable insights. (See, e.g., La Rosa et al. (2011) for ideas on highlighting.)

Empirical studies address few variables related to process models’ layouts. Figl and Strembeck (2015) investigate the overall model direction, but this experiment should be replicated in combination with layout options in order to shed light on why no significant differences are found related to flow direction. Future research could assess whether insights from related fields, such as graph drawing, can be generalized to process models. (For example, Purchase et al. (2000) show that minimizing the number of crossings and maximizing symmetry are related to comprehension.) Some conceptual work has been done: Bernstein and Soffer (2015) identify process models’ key layout features (e.g., symmetry, angles, overall shape and size, alignment of elements) based on users’ perceptions of models’ readability and define measurable metrics to characterize the layout of process models. Schrepfer et al. (2009) propose how model layout might relate to the model’s comprehensibility. Effinger et al. (2011) perform a user evaluation of the criteria for process models’ layout (e.g., bending and crossing of edges, arrangement, overlapping and size of elements, coloring), but did not measure the effect of these criteria on the comprehension of process models. Additional investigation into and experimentation regarding factors related to the model’s layout are required to determine the degree to which these factors are of only aesthetic value or in which cases they hinder information reception from process models.

Articles that provide modeling guidelines give concrete, practical hints for some size measures, such as “do not use more than 31 [nodes]” or “use no more than 2 start and end events” (Mendling et al. 2012a, p. 1195), which are grounded on analyses of errors in large collections of process models. Experimental studies on the comprehension of process models do not compare models beyond or below specific thresholds but find an overall negative correlation between larger size and comprehension. Similarly, Sánchez-González et al. (2010, 2012) calculate thresholds for the levels of comprehension based on data sets of models whose comprehension scores come from user studies. While information about the size at which a process model begins to be difficult to understand is useful for practitioners, more complex characteristics should also be taken into account. These are detailed below.

While block structuredness is advised in modeling guidelines (Mendling et al. 2010a; Rosa et al. 2011), the theoretical literature also warns of drawbacks like the duplication of model elements and gives advice on when to prefer unstructured modeling (Gruhn and Laue 2007). These ideas are in line with Dumas et al.’s (2012) empirical results, which show that structuredness is not always superior.

The metric nesting depth’s tendency to heighten cognitive load is addressed in theoretical discussions (Azim et al. 2008; Storch et al. 2013) and supported by empirical evidence (Figl and Laue 2015; Sánchez-González et al. 2010).

More theoretical and practical discussions than empirical studies address model refactoring – that is, replacing “process model fragments [with] semantically equivalent ones” to achieve higher quality and understandability of models (Fernández-Ropero et al. 2013, p. 1397). In principle, model refactoring lowers the intrinsic cognitive load in understanding a process model without changing the process’s behavior. Weber et al. (2011) collect a catalogue of indicators of bad process model quality to identify refactoring opportunities and provide a set of behavior-preserving techniques for refactoring to avoid redundancies and unnecessary increases in the model’s complexity. Automatic tools can help to identify “syntactical anti-patterns” (Laue and Awad 2011) and exchange them. Overall, the potential of automatic model refactoring and patterns to reduce cognitive load remains to be determined, so there are opportunities for scholars to examine these patterns in comprehension experiments.

While theoretical discussions mention the importance of removing redundant elements (Rosa et al. 2011) and frame rules for how to deal with redundancies (Becker et al. 1995), no empirical studies investigate the potential negative effect of redundancy on comprehension.

Although theoretical discussions advise keeping gateway heterogeneity and gateway mismatch low in models (Mendling 2013), evidence in empirical studies is missing or weak. One problem with relating the global metrics of models to overall model comprehension scores is that, if characteristics are not isolated, they can be confounded with other characteristics of the models. Constructing informationally equivalent models while varying gateway heterogeneity and gateway mismatch is difficult, if not impossible, so empirical studies also use specifically designed comprehension tasks that relate to various control structures. Overall, empirical studies support the theoretical hypotheses that order is easy to understand, while repetition is more difficult to understand (Gruhn and Laue 2006). Gruhn and Laue (2006) also theorize that exclusiveness is easier to understand than concurrency is, but research does not yet report evidence of differences in comprehensibility. Guidelines for avoiding OR gateway elements (Mendling et al. 2010a) might have caused researchers to refrain from including OR in their comprehension studies at all, as only one study includes OR, and it reports weak evidence of OR’s being more difficult to understand than AND and XOR are (Sarshar and Loos 2005).

The recommendation to minimize the number of routing paths per element (Mendling et al. 2010a) is consistent with the results of empirical studies (Reijers and Mendling 2011).

Few empirical studies investigate the effect on model comprehension of adhering to syntax rules; additional research must be undertaken before the association between those variables is more clearly understood.

Aysolmaz and Reijers (2016) mention the idea of tailoring process models to personal factors, an idea that has not been empirically assessed but seems practically and theoretically important. In a similar vein, Becker et al. (2004) present “configurative” process models, which can automatically adapt their information to user groups. Future work may address the extent to which this idea can ease comprehension, for example, by tailoring process models to aspects of cultural context, expert-novice differences, or cognitive styles, all of which are variables for which empirical studies report interaction effects with various forms of representation.

As theoretical discussions have highlighted the importance of domain knowledge (Schrepfer et al. 2009), it was surprising that variables that measure domain knowledge show no significant correlations with model comprehension in empirical studies. As these results may be influenced by variance in the variable domain knowledge in the studies, future work could compare domain experts with users who are unfamiliar with a domain to determine the magnitude of a possible effect of domain knowledge on comprehension.

Despite the wealth of measures used to assess experience and familiarity, the research reports statistical evidence of an effect on comprehension for only a few. The possible reasons for this result have already been discussed. Future researchers must be particularly cautious in interpreting self-rated measures of modeling experience since Mendling et al. (2012b) report that their relationship with comprehension is u-shaped and not linear.

In contrast to self-reported familiarity and knowledge, the process modeling knowledge test Mendling et al. (2012b) propose correlates with comprehension in six of eight studies, demonstrating its content validity. Therefore, this instrument can be recommended to control for individual variation in human information behavior related to the comprehension of process models.

The study of Figl and Recker (2016), which focuses on cognitive styles, and that of Recker et al. (2014), which focuses on learning style, are unique in that user characteristics are the main variables of interest. Kummer et al. (2016) provide the first experimental analysis of the cultural dependency of human information behavior related to process comprehension but find no evidence of an effect.

The theoretical discussions mention additional variables, such as perceptual expertise, ability to recognize patterns (Schrepfer et al. 2009), user attitudes, and self-efficacy (Reijers et al. 2010), none of which are addressed in process modeling research and all of which future studies could take into account.