Visual ppinot: A Graphical Notation for Process Performance Indicators

In Visual ppinot, PPIs are depicted as a rectangle decorated with a gauge icon on its upper left corner, its ID centred at the top, and the measure defining the PPI displayed inside the rectangle. The target value and the temporal scope are displayed together with their corresponding icons in an optional grey bottom compartment as shown in Fig. 4a.

On the other hand, measures can be classified into the three main measure categories present in the ppinot metamodel: base measures, aggregated measures, and derived measures.

Time measures, visually identified by an hourglass

, are used to measure the duration between the occurrence of two events, considering as events not only BPMN events, but also state transitions of BPNM elements such as activities, pools, or data objects. Notice that a time measure has an undefined value until both events have happened, something that is relevant for aggregated measures.

To indicate the two events, time measures use time connectors, represented as dashed lines. As shown in Fig. 8a, the connector for the first event is labelled with “from” and decorated with an empty circle on the end close to the measure icon

, whereas the connector for the second event is labelled with “to” and decorated with a filled circle

. Because the start and end of activities and pools are by far the most relevant events for defining time measures, they have their own graphical representation: the start event is depicted as an empty circle

, whereas the end event is represented as a filled circle

. In both cases, the name label is left as optional and it is usually omitted. The semantics of these two events are token based (OMG 2011), i.e., we consider that a start event happens when a token arrives at a BPMN element and that the end event happens when the token leaves an element. These two events are usually used with pools and activities, but they can also be used with BPMN events (see Sect. 5.6.1 for details). If an event is related to a state transition, the corresponding time connector must be labelled with the target state, i.e., the state to which the BPMN element must change to consider the event as triggered. Any state defined in the BPMN specification (OMG 2011) can be used with pools and activities, as well as any user-defined state can be used with data objects. A summary of the applicability of time connectors is displayed in Table 2. Figures 5 and 7 present several examples of time measures.

Count measures, identified by an ellipse with the numbers 1, 2, and 3 inside

, are used to count how many times a given event happens. Events are linked to count measures using applies-to connectors as shown in Fig. 8b, and their applicability rules are the same as for time connectors, summarised in Table 2. Figure 14 contains two examples of count measures.

State condition measures, decorated with an ellipse containing a checkmark

, generate Boolean values depending on the current state of activities, pools, or data objects. As depicted in Fig. 8c, these BPMN elements are linked to state condition measures using applies-to connectors, which must be labelled with the target state name

. Notice that the start and end event notations cannot be used with this type of measures because they are not actual states but events (see Table 2).

In the case of state condition measures aggregation, Boolean values are mapped to integers, i.e., \(false \mapsto 0,\ true \mapsto 1\). Because of this mapping, the aggregation functions are not the same as those commented on in Sect. 5.1.2, but the following ones are summarised in Fig. 9: (#) number of process instances in which the state condition holds, equivalent to the summation function; (%) percentage of process instances in which the condition holds, equivalent to the average function; (\(\exists\)) true if there exists at least one process instance in which the condition holds, i.e., when the values of the minimum and maximum aggregation functions are 0 and 1 respectively, as shown on the right side of Fig. 9; (\(\forall\)) true if the condition holds in all the process instances in scope, i.e., if the minimum and maximum functions values are both 1; (\(\not \!\exists\)): true if there does not exist any process instance in which the condition holds, i.e., if the values of the minimum and maximum functions are both 0.

Some examples of aggregated state condition measures are shown in Fig. 10.

Data measures, identified by a small data object icon

, are used to obtain the value of a specific attribute of a data object. The applies-to connector linking the measure icon with the data object must specify the attribute reference to be measured and, optionally, the state the data object must be in to actually obtain the value, as depicted in Fig. 11a). If the state were specified and the data object were in a different state, the value of the measure would be undefined. Notice that, to aggregate data measures, the measured attribute must belong to a data type with at least the \(\le\), \(+\), and \(\div\) operators defined to properly apply the usual aggregation functions.

Most elements of the ppinot metamodel have a 1:1 correspondence with the graphical symbols in Visual ppinot, as suggested by the Physics of Notation principle of semiotic clarity. However, some symbol deficit, i.e., leaving some semantic constructs without graphical representation, was introduced to limit the graphic complexity, as suggested by the Physics of Notation principle of graphic economy, which states that the number of graphical symbols should be cognitively manageable.

The decision of which semantic constructs do not have a graphical representation was made according to (1) the frequency they appear in the related literature and the scenarios in which Visual ppinot has been applied and (2) the type of information they convey. Concerning the former, we excluded those semantic constructs that appear with lower frequency. Specifically, the PPI temporal scope and target can be graphically depicted only when they are simple (e.g., monthly, lower than 7, or between 10 and 15) but not when they have more complex semantics (e.g., working days or Christmas holidays). As for the latter, we excluded some attributes of PPIs and measure definitions whose information is provided by means of a free text field such as goals, informed, or comments.

Some symbol redundancy, i.e., having more than one symbol for a single semantic construct, was also introduced to allow the modelling of aggregated base measures in their expanded or abbreviated form (see Fig. 5), thus providing an explicit mechanism for dealing with diagrammatic complexity, as suggested by the Physics of Notation principle of complexity management.

The complete correspondence between the ppinot metamodel and the Visual ppinot graphical symbols is included as Online Appendix B.

The Physics of Notation defines eight dimensions or visual variables of the graphic design space, which can be used to graphically encode information (Moody 2009). They are divided into planar (horizontal and vertical position) and retinal variables (shape, size, colour, brightness, orientation, and texture). In Visual ppinot, only shape, brightness, texture, and position are used, although the last one is only used for enclosing measures inside PPI icons. The unused visual variables can be freely used by the user to emphasise concepts from the business domain. This decision is not aligned with the Physics of Notation principle of visual expressiveness, which pursues the use of the full range and capacities of visual variables, but it maintains consistency with BPMN 2.0.

Shape is the main visual variable of Visual ppinot nodes because of its privileged role in perceptual discrimination (Moody et al. 2009). Therefore, different shapes have been used for different constructs. These shapes are graphic metaphors commonly used for the semantic concepts they represent, following the Physics of Notation principle of semantic transparency.³ Thus, a ruler is used as a metaphor of a measure, a gauge as a metaphor of an indicator, an hourglass as a metaphor of time, and so on. Furthermore, considering also that similar shapes should be used to represent similar constructs (Moody et al. 2009), similar symbols were designed for derived single-instance and derived multi-instance measures.

Shape is also used to distinguish between the main groups of connectors. To distinguish between subgroups, we used brightness for from and to, and texture for aggregates and isGroupedBy. In these cases, we decided to use text to reinforce graphical differences as suggested by the Physics of Notation principle of dual coding. The specific way in which visual variables have been used has been inspired by BPMN 2.0 and other similar notations such as UML because most users of Visual ppinot are expected to be familiar with these notations.

Finally, the use of shading, line thickness, colours, or any other distinction that does not fax/copy well or make symbols difficult to draw by hand was intentionally avoided. This decision has been supported by the experience gained in the evaluation scenarios, in which all the symbols were easily drawn by hand, something greatly appreciated in visual notations (Rumbaugh 1996).

Our research method in this evaluation has been based on case study research. Specifically, we have followed the case study research process proposed in Runeson and Höst (2009) and Runeson et al. (2012), as described below and summarised in Fig. 17.

The established objective for our multiple-case study was refined into five research questions, namely:

Regarding the four selected cases, Table 3 summarises their information in terms of processes, activities, PPIs, and people involved. In the case of the IT Department of the Andalusian Health Service, Visual ppinot was applied to the RFC management process (with 27 activities in the real model) and its 11 associated PPIs. Apart from two Visual ppinot experts, the two managerial roles involved in this case were the ones responsible for processes and continuous improvement, and for SLAs and performance management respectively, both belonging to the quality group of the department.

In the Information and Communication Service of the University of Seville, Visual ppinot was applied to four business processes, with a number of activities between 6 and 23, framed in the context of the IT support to the university staff (email incident management, for instance), and their 16 associated PPIs. In this case, the roles involved were the section manager, the technical manager of the teaching and research support area, and the software development manager together with two researchers and one master’s student who worked as a Visual ppinot assistant for the ICS staff.

In the Andalusian Ministry of Justice and Public Administration, there were 29 PPIs described in 5 processes with a number of activities between 8 and 36, ranging from social and health benefits management to suggestions, complaints, and claims management. In this case, three researchers amongst the authors of this article were the roles involved.

Finally, Visual ppinot was evaluated in two master’s courses of the University of Seville belonging to the Master of Information and Communication Technology Management, and the Master of Software Engineering and Technology, and in one bachelor course (Processes and Services Management) of the same university. In total, 112 students were trained in Visual ppinot and modelled at least two PPIs in the master’s courses, and at least eight PPIs in the bachelor course, for a real business process specified in BPMN. The processes belonged to many different domains: health, justice, university (scholarships, enrolment, research project management, etc.), software development or politics, to name a few. In total, 374 PPIs were modelled in these processes.

With respect to the data collection protocol, interviews, questionnaires and some available archival data were the initially defined data sources, according to the information available at that moment. Later on, they were refined and extended during the course of the study, as detailed in the following section.

The preparation of data collection and its actual collection were performed by means of iterative cycles. We applied data collection techniques from the three degrees defined in Lethbridge et al. (2005). In the following, we describe them, organised by data type. The degree is specified in parenthesis.

Process models and previously defined PPIs (third degree) The three studied organisations had previously undergone a BPM initiative, and, as a result, all processes under study were already modelled either in BPMN (AHS) or in another non-standard notation like flow diagrams (ICS, AMJPA). For each process, PPIs were defined using ad hoc table-based notations written in a natural language. All these available data were collected, and, in the required cases, the process models were translated to BPMN by some of the researchers involved to allow the definition of the corresponding PPIs in ppinot tool suite. In the academic scenarios, the process models were modelled in BPMN by the students using the documentation provided to them from different sources.

PPIs defined in Visual ppinot (second and third degree) The definition of the provided indicators using Visual ppinot involved different roles depending on the unit. In the AHS, the 11 PPIs associated with the RFC management process were modelled with the supervision and support of the one responsible for processes and continuous improvement (the result is presented in Online Appendix C). In the ICS, a preprocessing was needed, since some of the indicator definitions provided were not actually referred to the business processes but to other aspects of the organisation, i.e., they were not actual PPIs. Once filtered out those indicators, the Visual ppinot assistant modelled the 16 PPIs under the supervision of all the other roles involved from the ICS staff, throughout 11 meetings (read Sánchez-Jerez 2012 for more details, including business process and PPI models). In the AMJPA, two Visual ppinot experts were in charge of modelling the 29 PPIs. Finally, in the AS, students were required to define PPIs textually using some templates and patterns provided, whereas their graphical definition using Visual ppinot was optional (just for improving their grades).

Event logs and PPIs computation (third degree) In the AHS unit, a set of event logs containing the information of the business process execution on a period of 24 months was also available. This information was used by two Visual ppinot experts to compute the PPI values in the provided period. This was carried out through the ppinot tool suite, in particular the PPINOT Compute Engine, using as input both the PPI definitions in Visual ppinot and the provided event logs. The result was the set of raw values for the input set of PPIs, which were stored in our data repositories.

Evolution information (third degree) As part of an ongoing collaboration that some of the authors maintain with the AHS, we also had the opportunity of working on an evolution of the RFC management process, serving as a consultant for its re-modelling in BPMN. This scenario provided data to check how the Visual ppinot definition helps in the case of the associated business process evolution. Specifically, we gathered the process models and PPIs defined before and after the evolution.

Interviews and questionnaires (first degree) Direct methods for collecting data were used in three of the cases. In particular, in the AHS, the final graphical PPI definitions together with their computation results were presented by two researchers to the person responsible for SLAs and performance management during an interview. This was a semi-structured interview, more in the form of a discussion. We used the interview questions prepared as a guide of important information to be gathered. Furthermore, the conclusions of the evolution of the process model were also discussed with him in that interview. Most interesting facts and important answers were written down by one of the researchers in the form of notes that were later sent to the interviewee for validation. In the ICS case, the final graphical PPI definitions were presented to the software development manager during a structured interview by the aforementioned Visual ppinot assistant, who also wrote down the answers for their later validation by the interviewee. Finally, questionnaires were conducted with students at the end of their courses in order to gather information regarding identified expressiveness-related limitations and understandability-related weaknesses in Visual ppinot.

The analysis performed in the study is of the deductive type, implying that categories of analysis are imposed prior to the data collection. The main categories of analysis in our multiple-case study correspond with the five research questions and are: expressiveness, precision, automation, understandability and traceability. The specific actions accomplished during this analysis are detailed next.

This section presents the results identified after the analysis performed on the collected data. We structure the findings according to the five categories corresponding to the research questions posed in Sect. 8.2.

RQ₁ – Expressiveness After the analysis of the collected data, some limitations were detected in Visual ppinot and the notation was consequently improved. The first improvement was related to the distinction between linear and cyclic time measures, identified during the definition of PPI-3 in the AHS, which implied measuring an average duration located within a loop. The second was the inclusion of the isGroupedBy connector, used to define different target values according to a certain attribute value of a data object, as required by PPI-9 and PPI-10 from the AHS with respect to RFC objects (see Fig. 14). The third improvement was the optional inclusion of the target values and temporal scopes in the PPI icon, provided they are simple enough to be displayed. Finally, the range of function types used within derived measures was expanded to include Boolean and relational functions in addition to arithmetic ones, to meet the requirements of some PPIs.

After these improvements, more than 400 real-world PPIs from different domains were defined in Visual ppinot. As detailed in Table 4, all Visual ppinot symbols were used at least once, although some of them (e.g., Time or Count Aggregated Measure) were used much more frequently than others (e.g., State Condition Aggregated Measure). Furthermore, most of the PPIs modelled in the case study had simple numeric targets (98.4 %) and temporal scopes (87 %), thus allowing their complete graphical representation.

RQ₂ – Precision The application of Visual ppinot to the PPIs previously defined in other formats revealed three major limitations: (1) the indicator definitions were not clear because they used ambiguous language, making their interpretation and computation difficult; (2) some of the indicators lacked a clear relation to the business processes; actually some of them were not related to any business process and could not be computed on the basis of any business process execution values; and (3) for those indicator definitions related to business processes, many lacked the explicit relationship to the different business process elements, i.e., it was not straightforward to instrument the corresponding business process to obtain the PPI values.

RQ₃ – Automation The computation results obtained via ppinot tool suite from the PPI definitions in Visual ppinot and the collected event logs were presented to the person responsible for SLAs and performance management during the interview. He was asked to validate them and he compared them with the reports they had for that time period and reported them to be correct. During this interview, he recognised the usefulness of Visual ppinot in this scenario because of two reasons: first, it was not necessary to spend time in implementing PPIs using SQL queries to compute their values, and, second, the interpretation that was given to a PPI using Visual ppinot was much easier to understand than finding out the interpretation by going through a set of SQL queries. In addition, Visual ppinot proved to be platform independent since ppinot tool suite was developed without knowledge of the information system that provided the logs for computing PPI values.

RQ₄ – Understandability The first applications of Visual ppinot revealed a couple of limitations related to the labelling of time connectors and the use of data property condition measures. This led us to a twofold improvement of the notation. On the one hand, the labels “start” and “end” of time connectors were changed to “from” and “to”, respectively, since the first labelling seemed to refer to the start and end of activities and pools instead of the events that allow the duration to be measured. On the other hand, we removed the graphical construct for data property condition measures because Visual ppinot users did not clearly distinguish between them and data measures, and the construct was no longer necessary since they can be modelled as a derived single-instance measure with a Boolean function on a data measure.

After these improvements, the users, i.e., organisations’ employees, students and researchers, were able to read and validate PPI graphical definitions as reported on during the interviews. Furthermore, the users from the AS unit (students) were also able to generate them after a proper training, and even more important, more than 80% preferred to define their PPIs graphically rather than textually, even when it was not compulsory in their assignments. These results are encouraging and coincide with the results obtained in the experiment performed in Mora et al. (2011), where graphical and textual models of software measurement were compared, demonstrating the graphical model to be more understandable and modifiable than the textual one.

RQ₅ – Traceability Findings related to this aspect were found in the context of the AHS case, during the evolution scenario described above. In this case, the business process update was meant to change the position of certain activities, to add some new ones to depict some unrepresented exceptions to the normal flow, and to refine some other aspects like the data flow. These changes barely affected the previously defined PPIs, except for the cases in which the relocated activities had connections to PPIs. In these cases, the graphical editor maintained those connections between the PPIs and the relocated activities, which allowed an immediate and automatic update of the PPIs.

Regarding expressiveness, despite the considerable number of PPIs defined in different domains, we cannot state that all possible PPIs can be defined with Visual ppinot. However, extension points were already defined in its underpinning metamodel (del Río-Ortega et al. 2013), and corresponding extensions could also be performed in the visual notation when identified.

Concerning understandability, there is a limitation to generalisation since all the Visual ppinot users in our multiple-case study had technical backgrounds (most of them were engineers, but there were also mathematicians and computer scientists). It might be possible that the results regarding the ability to read and write PPI definitions in Visual ppinot are different if their profiles are non-technical (from social sciences, for instance).

With respect to the automatic computation, a limitation can be found in the information required from the logs. In order to compute most common PPIs, a log with the activities carried out, with the time when they were carried out, and the process instance to which they belonged is necessary. This is the typical information provided by most process-aware information systems in the form of process event logs. However, if the information systems are not process-aware, this information might be harder to obtain. Nevertheless, we do not consider this a serious issue since, according to our experience, many information systems used in organisations nowadays are process-aware. Furthermore, new techniques are being developed to gather this information from non-process-aware information systems (Rodríguez et al. 2012; van der Aalst 2015).

Finally, though the model integration brings some benefits like evolution traceability and the possibility to see business process models together with their associated PPIs, it can also involve some challenges regarding readability. When the number of PPIs increases, the readability of business process models including PPIs decreases. This can be alleviated using technological tools to decide, from the whole set of PPIs defined for a business process, which PPIs to show and which to hide.

The measurement of business process performance has triggered many research efforts, yielding a variety of different approaches. Many of them propose languages and architectures for describing and monitoring PPIs, some from a general point of view such as Castellanos et al. (2005), Popova and Sharpanskykh (2010) or Saldivar et al. (2016), whereas others are specific to certain contexts such as semantic business processes (Pedrinaci et al. 2008; Wetzstein et al. 2008) or service-oriented architectures (Momm et al. 2007).

In general, Visual ppinot improves not only the expressiveness of those works, but also the visual representation of business process-PPI links. Regarding expressiveness, Visual ppinot, i.e., the ppinot metamodel, allows PPI definitions which are not possible to express in other approaches, especially those related to states or data, as analysed in del Río-Ortega et al. (2013) and summarised in Table 5.

Regarding the representation of business process-PPI links, the aforementioned approaches mainly focus on semantics and hardly on syntax details that could ease the understanding of PPI definitions. Costello and Molloy (2009) and González et al. (2009) are some of the authors who have already identified this problem and have made some proposals to improve the comprehension of PPI definitions and bring them closer to non-technical stakeholders. Their proposals include a PPI visual model and “a language for high-level monitoring, measurement data collection and control of business processes”, although they actually present textual (e.g., XML-based) mechanisms that require a certain level of technical knowledge and, in the case of Costello and Molloy (2009), are only focused on time measures.

On the other hand, Korherr and List (2007) have extended the BPMN and EPC metamodels in order to define business process goals and performance measures, including cost, quality, and cycle time measures, although only cycle time measures are explicitly connected to business process elements and visually modelled. In contrast, Visual ppinot provides a visual representation and an explicit connection with the business process for all of the allowed measures and, in addition, considers all the information required to define and calculate them.

With a level of expressiveness similar to Visual ppinot, Friedenstab et al. (2012) have proposed a graphical notation for BAM. When compared to it, Visual ppinot presents some improvements such as the definition of PPIs related to data, the explicit and visual representation of connectors to the BPMN elements, the set of principles for obtaining cognitively effective visual notations taken into account in its development, and the supporting tool and subsequent validation of our proposal.

Finally, another very closely related work is the one presented in Delgado et al. (2014), where an execution measurement model for business processes realised by services is proposed based on an existing software measurement ontology (García et al. 2009). This model provides a predefined set of generic execution measures organised according to the four dimensions of the Devil’s quadrangle (Jansen-Vullers et al. 2008; Dumas et al. 2013), i.e., time, cost, quality, and flexibility, together with measures for lean and service executions. It also proposes a method and a tool to guide and support execution measurement and the subsequent business process improvement.

In contrast, Visual ppinot allows for the definition of domain-specific, user-defined PPIs, apart from the predefined ones proposed in Delgado et al. (2014); these PPIs are visually modelled together with the business process; and Visual ppinot is intended for defining measures on any type of business process, including those realised partially or exclusively by humans. Actually, Visual ppinot can complement the work in Delgado et al. (2014) by broadening the spectrum not only of the business processes to be measured, but also of the measures themselves.

Table 5 summarises this analysis of the related literature. In particular, we have evaluated to what extent the related approaches that are directly comparable to Visual ppinot fulfil or cover the different features identified as desirable and evaluated in our approach. A \(\sim\) sign in a cell indicates that that particular approach addresses that feature partially.

In the context of frameworks for measurement dimensions, a number of works have been proposed, such as Cross and Lynch (2007), Keegan et al. (1989), Brignall et al. (1991), Kaplan and Norton (1992), Brand and Kolk (1995), or Adams and Neely (2002), but the aforementioned Devil’s quadrangle and its four dimensions (time, cost, quality, and flexibility) has proven to be the most suitable for business processes (Jansen-Vullers et al. 2008; Dumas et al. 2013). The main difference between these frameworks and Visual ppinot (and other similar approaches such as the ones discussed in the previous section) is that while Visual ppinot focuses on “how” the indicators are measured – i.e., how the information required for their computation can be obtained from the process – the frameworks focus more on “what” is measured by the indicators. One of the consequences of this difference is the need of using proxies (which can be defined in Visual ppinot) for the operationalization of dimensions that cannot be directly measured such as quality or flexibility, e.g., using the number of complaints received or the number of items returned as proxies of the quality of a purchase process (Jansen-Vullers et al. 2008). This is the reason why Visual ppinot does not include specifically quality or flexibility measures.

Other works are focused on one particular dimension of the Devil’s quadrangle. With respect to the time dimension, there are some timed-BPMN proposals such as Lanz and Reichert (2014), Cheikhrouhou et al. (2013), or Mendoza et al. (2011). However, these approaches are focused on modelling temporal constraints in the process flow instead of defining measures, i.e., they restrict the process behaviour according to certain time constraints. Therefore, they could be seen as an extension of the time event that BPMN and other similar notations include, and the definition of measures in Visual ppinot could be reused as a mechanism to specify those restrictions. In fact, after the analysis of the time patterns presented in Lanz et al. (2014), Visual ppinot covers 8 out of the 10 patterns. The two uncovered patterns are TP4 (Fixed Date Element) and TP5 (Schedule Restricted Element). To cover TP4, we would need an extension to the ppinot metamodel already identified in del-Río-Ortega et al. (2015). As for TP5, it is not covered straightforwardly since schedules are not artefacts present as part of the business process model; however, if this information was provided within a data object, it could then be expressed in Visual ppinot.

Regarding the approaches related to the cost dimension (Magnani and Montesi 2007; Sampath and Wirsing 2009, 2011), their focus is on obtaining cost estimations of processes based on past executions, but they are not intended to compute actual values of the instances that are currently running, which is the goal of Visual ppinot.

Other approaches that are somehow related to the context of this article are those that try to integrate risks with business processes. Risk-aware business process management seeks to reason about the likelihood and the impacts of the occurrence of various types of risks, such as security or regulatory non-compliance (Suriadi et al. 2014). Obtaining undesired values for certain PPIs can be understood as a type of risk, therefore, PPI definitions in Visual ppinot can be used as the input to many of the existing approaches to define risks, such as Jakoubi et al. (2010) or Rosemann and zur Muehlen (2005).

It is also worth mentioning the recently released DMN standard for decision modelling (OMG 2016). Although it serves a different purpose, it is also concerned with calculating process-related measures. Visual ppinot can complement DMN, especially in the context of decision logic modelling. Since decision rules in DMN are defined through a number of expressions that are evaluated using input variables (that can also be themselves expressions), Visual ppinot can be used to define those process-related input variables.

The evaluation of the tool support for PPI definition in current BPMSs is based on the analysis performed and presented in Saldivar et al. (2016) and on further analysis we have carried out. The most representative commercial tools at the time of writing as well as open-source solutions have been considered for the evaluation. In particular, we have considered IBM Business Process Manager (IBM 2009), ARIS Process Performance Manager (Scheer et al. 2006), BizAgi Modeler (BizAgi 2015), Bonita Open Solution (Bonitasoft 2011), Adonis Community Edition (BOC Group 2015, Oracle Business Process Management Suite 12c (Oracle 2014), TIBCO Business Studio (TIBCO 2014), and Camunda (Camunda 2014).

Because not all the tools could be installed for their study, we have also based our analysis on the official documentation published by each solution. Sometimes, the documentation provided insufficient information to draw a conclusion about a particular feature. The results of the analysis are summarised in Table 6 and described in the following paragraphs. Most of the analysed tools provide predefined standard measures such as duration, idle time, cost, throughput, or resource utilisation. IBM BPM and ARIS PPM are the exception, since they allow the user to define their own measures although with some restrictions. Regarding the former, it is only possible to define measures using arithmetic operations on process variables. As for the latter, it is possible to define a wide range of measures except for data measures, as long as it can be deduced from the documentation.

With respect to visual representation, only ARIS PPM offers a partial graphical mechanism for PPI definition through measurement points defined over EPC models. However, a comprehensive graphical definition of PPIs is not possible according to the available documentation, since only measurement points can be graphically depicted, whereas the rest of the elements involved in a PPI have to be described textually using some forms provided by its user interface.

It makes sense to take traceability between business processes and PPIs into consideration only in tools where user-defined measures are allowed, i.e., ARIS PPM and IBM BPM, but as far as their documentations describe, it is not clear whether they support it. Regarding the possibility to automatically compute PPI values, all of the analysed tools offer this feature. In addition, most of them allow the generation of reports, either predefined or user-defined, depending on whether predefined or user-defined measures are available, respectively.