Simulating progressive iteration, rework and change propagation to prioritise design tasks

A design for a system or component is not completed in a single step, but requires a progressive process in which parameters are incrementally defined and frozen, and in which more accurate tools and analyses are gradually brought to bear as increased confidence in the design justifies the increased effort for their application. One way of perceiving design is thus as a gradual process of uncertainty reduction, in which the design description begins as a vague concept that allows for a wide range of final designs. This concept is concretised through a sequence of knowledge-generating activities and decision-making tasks, reducing the space of possibilities until a precise recipe for manufacture is reached (Antonsson and Otto 1995). According to this perspective, the aim of the design process is to generate knowledge that reduces uncertainty and increases design maturity.

There is a rich body of literature aiming to understand, represent and support the progressive nature of design processes. For instance, Antonsson and Otto (1995) distinguish between uncertainty, which represents uncontrolled stochastic variations, and imprecision, which is seen as uncertainty in choosing among alternatives. They propose the Method of Imprecision (MoI), based on the mathematics of fuzzy sets, for formal representation and manipulation of imprecision in engineering design. It is intended to support decision making and to facilitate co-ordination of concurrent engineering. Wynn et al. (2011) describe a process simulation model that captures iterative progression as uncertainty reduction in design, using the term ‘uncertainty’ to refer to everything that contributes to a lack of definition, lack of knowledge or lack of trust in knowledge. Their model considers the effects of design process tasks on up to five aspects of uncertainty associated with information that is iteratively developed during design: imprecision, inconsistency, inaccuracy, indecision and instability. The ‘level’ of each uncertainty aspect associated with a given piece of design information is modelled without reference to the information ‘content’ and is represented as a numeric value between predefined extremes. O’Brien and Smith (1995) consider progression in terms of design maturity, arguing that progress of immature designs should be prevented and mature designs should be progressed in concurrent design. They write that ‘a design is mature when it is complete enough to allow the release of details to downstream activities, knowing that the release of further details from the current activity will not lead to redesign in any downstream activities. Knowing when a design reaches maturity would reduce the risk of releasing immature details to a following activity, and remove the delays caused by the need to check, through other methods, the validity of the design before its release’ (O’Brien and Smith 1995). According to these authors, assessment of design maturity can prevent unnecessary delays and wasteful redesign. Grebici et al. (2007) also focus on maturity as a driver of design progress, presenting a framework to facilitate the exchange of knowledge about information maturity during design.

As hinted above, the fundamental process enabling progression in engineering design is iteration (Smith and Eppinger 1997b). Design is inherently iterative because of the cyclic interdependencies between parts of any design problem and because loops of synthesis, analysis and evaluation are fundamental to the design process (Braha and Maimon 1997). Although iteration is therefore necessary to progress a design, it is also considered to be a major source of delays and budget overruns (Eppinger et al. 1994; Browning and Eppinger 2002). Smith and Eppinger (1997a) recommend two general strategies for accelerating an iterative design process: Faster iterations and fewer iterations. Both approaches require comprehension of the whole process, especially the coupling between its tasks. Considering the effects of hidden information in product development, Yassine et al. (2003) identify three main causes of churn leading to PD delays: interdependency, concurrency and feedback delays. They propose resource-based, rework-based and time-based strategies to mitigate these issues. In the context of complex product development networks, Braha and Bar-Yam (2004a, b, 2007) show that the propagation of defects or rework can be contained by prioritising resources at central components or tasks.

To summarise, modelling iteration is critical in any simulation of the design process (Wynn et al. 2007); ideally, the fundamental modes of progressive iteration and unnecessary rework should both be considered.

A high degree of interconnection between parts of a product leads to complex interactions during the design process (Eckert et al. 2004). In particular, whenever a component is changed during design (often due to iteration, although change can also arise from other sources), this can cause knock-on changes in other components so the design can continue to ‘work together’ as a whole (Lindemann and Reichwald 1998). This is termed change propagation. Change can propagate through different paths, depending on the connectivity between components of the product. It is thus important ‘to be aware not only of individual change chains but of complex change networks’ (Eckert et al. 2004). Almost all design activity includes engineering changes (Jarratt et al. 2011); Lindemann and Reichwald (1998) conclude from an extensive study that effective management of change propagation could provide a big advantage. This can be achieved through effective change prediction, which involves two activities: predicting the causes for change and predicting its knock-on effects (Eckert et al. 2004).

Numerous change prediction models have been proposed in the research literature, focusing mainly on understanding knock-on effects (Jarratt et al. 2011). One of the most established approaches is the Change Prediction Method (CPM) by Clarkson et al. (2004). The basic principle common to almost all the methods is that change can only propagate between two components if they are somehow connected. In CPM, for instance, the product architecture must be modelled as two DSMs, respectively, indicating the direct impact and likelihood of change propagation between each pair of components in the design. An algorithm accounts for the possibility that change may propagate between two components via several intermediate paths, thus increasing the possibility that the propagation may occur. Koh et al. (2012) build on CPM to develop a method which aims to support prediction and management of undesired engineering change propagation incorporating parameters and change options as well as component linkages. Another recent approach to change prediction is discussed by Yang and Duan (2012), who explore change propagation paths using a parameter linkage-based approach.

Numerous articles have applied simulation to understand the impact of interdependencies and rework on the schedule and risk of design processes. Many are based on DSMs that represent the network of dependencies between tasks (Smith and Eppinger 1997b; Browning and Eppinger 2002; Yassine 2007). Approaches differ, among other things, in their simulation method, treatment of concurrency and treatment of task duration and rework (Karniel and Reich 2009). The main differences are discussed in the following subsections.

A range of techniques and models have been developed to increase understanding and enable the prediction of the behaviour of social, biological and technological networks (Newman 2003). Braha and Bar-Yam (2004a, b, 2007) show that large-scale product development networks have comparable structural properties and display similar statistical patterns to such systems. These structural properties can provide information on the dynamics of the product development process, for instance the characteristics of rework and its propagation. Braha and Bar-Yam (2004a, b) find that complex product development networks are dominated by some highly central tasks and characterised by an uneven distribution of nodal centrality measures and asymmetry between incoming and outgoing links. Braha and Bar-Yam (2007) consider priority rules based on the in-degree and out-degree of nodes and show that significant performance improvements can be achieved by focusing efforts on central nodes in the design network. They see the ‘close interplay between the design structure (product architecture) and the related organisation of tasks involved in the design process’ (Braha and Bar-Yam 2007) as a potential explanation for the characteristic patterns of product development networks. Although their analysis focuses on organisational networks rather than product architecture, they argue that the statistical properties should be similar.

Braha and Bar-Yam (2007) develop a stochastic product development model and analysed its behaviour. They show that if you run the PD process on top of a random network, a threshold behaviour that depends on the average degree of the network determines whether the PD is stable or unstable, and how much time it takes.

In particular, they show that the dynamics of product development is determined and is controlled by the extent of (1) correlation among neighbouring nodes, and (2) correlation between the in-degree and out-degree of individual tasks. For weak correlations, a network with any topology was found to exhibit the same behaviour as a random network.

Recalling that the objective of this article is to assess priority rules in design, another relevant area of research is the resource-constrained project scheduling problem (RCPSP). The objective is to minimise the lead time of a project by appropriately scheduling activities which are subject to precedence and resource constraints, but not iteration. Overviews of the many approaches to solving this problem are provided by Brucker et al. (1999) and Hartmann and Briskorn (2010). Because the problem itself is well defined, standard problem sets have been developed and widely used to compare the performance of different algorithms.

Commonly studied solution techniques include exact and heuristic algorithms as well as meta-heuristics. One of the most important solution techniques is priority rule-based scheduling (Kolisch 1996; Buddhakulsomsiri and Kim 2007; Chtourou and Haouari 2008). Browning and Yassine (2010) combine the resource-constrained multi-project scheduling problem (RCMPSP) with the use of task-based DSMs to analyse the performance of priority rule heuristics.

The RCPSP and RCMPSP literature demonstrates that priority rules provide a powerful approach to support managerial decision making. However, because these two problems do not include iteration or rework, it is not clear whether the insights are applicable to design processes.

Progression in design cannot be achieved through a linear process, due to cyclic interdependencies between parts, parameters and interfaces in the system being designed. This necessitates an iterative process of progressively increasing maturity, in which work must begin based on incomplete or imprecise information, in the knowledge that this may later require corrective changes. Such changes propagate through the design via interfaces, creating rework. The rework must be scheduled and prioritised; inappropriate prioritisation may increase the total amount of work that needs to be done.

Models have been developed to simulate the design process considering different aspects of this behaviour and to generate insights for scheduling and prioritisation. Most task-based models capture the effect of rework, but do not directly account for change propagation due to structural connections within the developed product. Increasing progress of individual components’ design maturity towards a final solution is usually not considered explicitly. Propagation effects have been extensively studied in change prediction models based on product architecture (component) DSMs; however, such models do not consider the time and cost of change completion as the process simulation models do. Finally, most scheduling algorithms consider the impact of priority rules but do not account for iteration and rework. No simulation model exists that explicitly combines all the following characteristics: rests on a component-based DSM; accounts for maturity progression in components; integrates the design process characteristics of iteration, rework and change propagation; and can evaluate the impact of task prioritisation in this context.

The tasks that can be started at any time depend on the maturity levels of each component, availability of suitable resource and the relative priorities of any components that require the same resource. These aspects of the model are discussed in Sects. 3.1.1, 3.1.2 and 3.1.3 respectively.

After choosing the task to be started by each idle resource, as explained above, the durations are calculated and the model state updated accordingly. These steps are explained below.

After starting the selected tasks, the simulation is advanced to the next event in \(\mathbf {E}\) at which a task is due for completion. The model state is updated and the possibility of change initiation and propagation is considered. Once the task completion has been processed as explained below, the state has changed and new tasks may become possible to start. The model thus returns to the first step (Sect. 3.1).

The model integrates existing concepts in the literature, namely iteration as a progression between maturity levels, rework initiation as an uncertain event that occurs on the completion of design tasks and propagation of initiated changes through the design according to likelihood and impact values. The model can be classified using the scheme referred to in Sect. 2.3 (Karniel and Reich 2009). It uses Monte Carlo simulation methods to account for occurrence and propagation of changes. Concurrency is treated using a multipath approach to execute activities in parallel with possible overlap. The model assumes rework to require less time than the first execution and thus accounts for learning effects to otherwise deterministic task durations.

Table 3 summarises the simulation algorithm. As well as the likelihood-impact DSMs \(\varvec{L}\) and \(\varvec{I}\) and the resource mapping matrix \(\varvec{Q}\), a number of parameters defining model assumptions are required. Suggested values are provided in Table 4. Some of these values may be justified by reference to prior studies as shown in Table 4. Even where such justification is not possible, making assumptions explicit in this way is useful because it allows their impact to be evaluated using sensitivity analysis. This is done in Sect. 5.

This subsection illustrates the model logic by presenting an application to a product model of the Westland Helicopters EH101. The input for the simulation is a DSM indicating impact and likelihood of change propagation between components in the helicopter. In this context, the term component refers to a helicopter subsystem, such as hydraulics or transmission. To generate the likelihood and impact matrices, a workshop and interviews were held with company personnel [see Clarkson et al. (2004) for details]. For the present article, this information was supplemented with an approximate duration for designing each component,³ thus completing the input data required for the simulation model (Fig. 1).

In this example, policy 6 [highest active sum (binary)] and the assumption parameter values in Table 4 are used. For illustrative purposes, three identical resources are used and assumed to be able to work on all components. Simulating 10,000 runs results in a histogram of total process duration (Fig. 2). Each bin of this histogram represents a number of possible process outcomes. One of these outcomes is plotted as a Gantt chart in Fig. 3. Figure 4 shows the performance of all policies relative to the mean of random task selection.

The Gantt chart reveals the mechanics of the simulation. Up to three subsystems can be worked on at the same time, because this is the number of available resources. The four green bars in every row reflect that maturity has to be increased to a new high four times to complete a component’s design. If changes cause rework in a component, a previously accomplished maturity level has to be reattained. As explained in sect. 3.2.1, such tasks are subject to learning effects. This is visible in Fig. 3 as a gradual decline of task duration between two green bars. The chart also shows that rework due to propagated changes occurs more frequently than rework caused by initiated changes, because of the fan-out of change through the product architecture.

The order of task execution is determined by the priority policy and is independent of the sequence in the DSM. In this case, under policy 6 the first component to be worked on has the most entries in its DSM column and the last component to be started has the fewest. For example, ‘cabling and piping’ and ‘auxiliary electrics’ both only have one outgoing relation to another component—‘bare fuselage’. The design of both components is therefore started towards the end of the project. Because the design of ‘bare fuselage’ is dependent on the other components, it can only be advanced to a certain level once both predecessors have gained a certain maturity.

The experiments used 12 product architecture DSMs drawn from the literature (Fig. 5). Each of these models was created through analysis of real products by the respective authors as listed in the figure caption. Three of the models include asymmetric input and likelihood data. The other nine models provide only symmetric binary DSMs. For simulation, these were transformed into impact and likelihood DSMs by assuming all likelihoods are 0.5 and all impacts are 1.

To study the influences of symmetry and sparsity as well as the sensitivity of the model to changes in the input data, additional input models were created based on the DSM of a chainsaw by adding entries. Ten or 20 entries were either randomly added to the upper and lower triangle (to create less sparse, less symmetric architectures) or symmetrically to both triangles (to create less sparse, more symmetric architectures).

To study the impact of a given priority rule across multiple product models, it is necessary to reduce the DSMs to a set of metrics that can be easily compared on one or more scales that ideally should be cardinal. Four metrics were used to characterise product models in the simulation experiments:

The sets of process traces resulting from simulation must also be comparable on ideally cardinal scales to evaluate policy performance under different conditions. The following metrics were used to compare performance of processes across simulation setups:Because the simulation results in a profile of possible processes, for each configuration 10,000 runs were used and the mean values of each metric calculated. The lower these two values are for a policy, the better its performance is assumed to be. Because of the simplifications made by the simulation model, relative performance is assumed to be more significant than the absolute results for total duration and tasks attempted. For each configuration studied, the performance of each policy is therefore presented as a percentage improvement or reduction in the metric values obtained from the random task selection policy for that configuration.

The objective of the simulation experiments was to reveal general insights regarding whether the choice between policies in Table 2 has a significant impact on process performance, and what contextual factors influence which policy performs best. The factors that can be studied are determined by the parameters of the model, which were in turn constructed from important issues identified during the literature review. Considering the available parameters, the following questions were identified for analysis:Each of these eight questions was investigated through systematic variation of the simulation parameters followed by in-depth study of the outcomes. Table 5 summarises the experiments that were undertaken. In each case, 10,000 runs were performed using the indicated combinations of parameters and inputs. For most experiments, all input models and policies were initially included, although Table 5 only lists the configurations that are used to present the findings.

The parameter \(\varDelta m_{max}\) (see Table 4) constrains the activity sequence by specifying the maximum maturity level difference allowed between connected components for a task to be attempted. Using the chainsaw model, \(\varDelta m_{max}\) was varied (\(\varDelta m_{max}: \; 1-4\)) to investigate the effect maturity level constraints have on policy performance. All other parameter values for this experiment remained as shown in Table 5.

Figure 12 shows that in most cases, differences between mean policy performance increase while the variance of results decreases if \(\varDelta m_{max}\) is assumed to be higher. In most cases, this has no effect on qualitative policy performance. It has to be noted that the absolute duration to finish the design increases with decreasing \(\varDelta m_{max}\). Based on \(\varDelta m_{max} = 4\) total duration for random task selection increases by 5, 13 and 21 %, respectively.

It can be concluded that the more strongly interfaces between parts constrain the activity sequence, the less the choice of policy affects design completion time. The choice between policies has the greatest impact if the activity sequence is not constrained at all. As the allowed maturity level difference between interconnected parts is reduced, policies become less important as they have fewer alternatives to choose among. In this case, the total duration to finish the design generally increases.

Analysis of the results revealed the three most effective policies under a range of conditions to be:These policies were found to be more effective than others tested across most of the scenarios studied. However, the specific situation does influence how well each policy performs. Therefore, this ranking is rather qualitative and should be understood as a suggestion of policies that perform well in most situations.

The performance of these policies can be explained in terms of the logic of the model. Lowest passive sum first can be interpreted as ensuring that tasks that could receive propagated change from many dependencies are not attempted until after those dependencies are finalised. Highest maturity level first and least reworked first can be interpreted as ensuring that the components which are thought to be least susceptible to change at any given point in the design process (because they have been reworked less than other tasks) are treated first. Work done to progress these components is less likely to be invalidated than work done to progress components that are historically more likely to receive change. These latter two policies may be valuable from a pragmatic point of view, because their use requires only knowledge of the history of the design process. Unlike lowest passive sum first, an explicit model of the product architecture is not required.

The least-recommended policy is lowest maturity level first. This policy always prioritises components that are lagging behind the rest. It may be interpreted as promoting firefighting, a phenomenon that often occurs in practice (Repenning 2001). The simulation results reported here confirm again that excessive firefighting may be detrimental to successful product development. The reason is that problems which keep occurring should usually be left until as late as possible, because repeated rework implies they are highly sensitive to changes elsewhere in the design.

This article makes two main contributions. Firstly, a new task-based model shows how design process simulation can account for the combined effects of progressive iteration, rework and change propagation in concurrent engineering. This is the first time these specific effects have been combined in a simulation study of the resource-limited design process. In future, widening the model’s scope and combining its features with other approaches to simulating the design process could lead to a more realistic and thus more accurate representation of design practice.

Secondly, the model was used to study the impact of selected priority rules on design process performance, under different assumptions and for different products. The results, summarised in Table 6, show that certain priority rules are generally effective in reducing the impact of change propagation and unnecessary rework during design. Results also confirm that effective management of the jobs competing for attention becomes more important as the interconnectedness and concurrency of design projects increases.

Several of the conclusions drawn from the simulation results are implied by prior theoretical and empirical results of Yassine et al. (2003) and Braha and Bar-Yam (2007), as noted in the text. The article thus confirms those prior results and their applicability to the specific context of engineering design as progression between maturity levels, set back by stochastic initiation and propagation of changes. The article also contributes new insights regarding progressive iteration through increasing maturity levels, a feature which is not explicitly incorporated in those prior models.

The box plots throughout this article reveal two insights about the variance in the results, which arises from the stochastic nature of the model. Firstly, the effect of the policies shifts the mean and also the whole distribution by an amount that is meaningful relative to the spread shown by the box. Secondly, in most cases, it is clear that the policies, which result in a reduction in time and effort, also result in a reduction in spread. Thus, the most effective policies can be expected to reduce variability (risk) as well as the expected (mean) performance. It may be noted that a differentiation of roughly 10 % in design process duration, as provided by comparing some of the policies studied in this article, can be significant in real terms. For instance it might imply avoiding a delay of 6 months on a 5-year aircraft program.

A key issue regarding any simulation model, and especially in cases where general insights are derived from simulation, is to understand the validity of the model and its input data.

In this case, the model is thought to have high face validity because it is synthesised from concepts that are well established and accepted in the research literature (Browning and Eppinger 2002; Clarkson et al. 2004; Cho and Eppinger 2005). The findings are developed directly from these assumptions, which might be refined in future studies but provide a reasonable basis for the current article. Additionally, certain results of the simulation were found to reflect prior work in which more general models were used to analyse the impact of priority policies in PD (Braha and Bar-Yam 2007). This alignment of results with prior research further support the validity of the new model.

In terms of input data, the model was deliberately constructed in such a way that existing product DSM models can be used directly to provide most required data. Of the assumptions that are not captured in a product DSM, several of the required parameter values (maximum number of change propagation steps, average learning curve steepness, etc.) were sourced from previous widely cited research studies. Finally, the effects of most assumptions embedded in the model were evaluated through a series of sensitivity analyses. Most of these assumptions were found to have little or no impact on the qualitative results as presented in Table 6. A critical assumption was found to be the probability of change initiation. This value was calibrated by reference to existing reports on the amount of rework found in typical development projects.

The findings are therefore believed to be well justified and relatively robust, although there is still scope for improvement of the model and for further sensitivity studies. Some suggestions for enhancement are identified below.

The approach presented in this article, in common with all models, simplifies actual conditions in a number of ways. Some of these simplifications and opportunities for further work are discussed below.