Pareto-optimization of complex system architecture for structural complexity and modularity

Theory of complexity has received a lot of attention in the field of engineering system architecting and design (Asikoglu and Simpson 2010; Braha and Maimon 1998; Dolan and Lewis 2008; Malik 1992; Tamaskar et al. 2014). There are multiple aspects of complexity aspects in the context of a complex engineered system design and development effort. Complexity can be categorized into (1) internal and (2) external complexity. There are three main dimensions of internal complexity that emerge from the existing literature in the context of system design and development—(1) structural complexity; (2) dynamic complexity and (3) organizational complexity (Braha and Bar-Yam 2007; Lindemann et al. 2009; Malik 1992; Riedl 2013; Weber 2005).

Structural complexity arises when a system is composed of many components and whose interaction is difficult to describe or understand. Structural complexity relates to the notion of the architecture of a system and, therefore, represents complexity determined by the form. Given a set of basic functions, there are multiple forms that can perform specified set of functions. An example is the function of slowing food spoilage. Over millennia humans have developed numerous concepts for achieving this particular function—cooling, freezing, irradiating, salting, canning and vacuum-packing the food. Achieving the desired functionality and system behavior with minimal structural complexity becomes an important criterion for down-selection of design/architectural concept. The system architecture impacts the complexity of the system during its initial design phase, during the implementation phase and during the changes that will occur in its lifetime.

Dynamic complexity is driven by what system does (i.e., its functions as well as the chosen concept). A system is deemed dynamically complex if its external behavior/dynamics is difficult to describe and predict effectively. Since the system behavior is described over an operational envelope of the system, dynamic complexity is an implicit function of this envelope. There are two primary sources of dynamic complexity: (1) interactivity among functional attributes and (2) uncertainties in their interactions. Note that the system behavior is often bounded by the underlying system architecture and, therefore, dynamic complexity possesses a strong positive correlation with structural complexity for well-engineered systems (Conway 1968; Sinha 2014) and they relate to the interplay between form and function/system behavior.

Finally, organizational complexity relates to the system development organization’s structure and the system development process. Organizational structure often mirrors the system’s architecture and is thereby closely related to structural complexity. Figure 1 describes a complexity typology for engineered complex systems from a system design and development perspective. In general, it has been observed that structural complexity strongly correlates with organizational complexity (Conway 1968; MacCormack et al. 2012).

The external complexities relate to facets that are usually not under control of the system development organization. They typically include complexities associated with funding mechanisms, market dynamics, political and institutional complexities, etc. They are much larger in scope and usually tend to encompass very large-scale projects (Sussman 2000).

From different complexity categories mentioned, structural complexity of the system architecture is the focus of this paper. The structural complexity is a characteristic that can be measured, and is influenced by the number of components, interfaces and the connectivity structure of the system. More components and intricate connectivity structure increase the design, development and operation effort (Maier and Rechtin 2009) and it also has been observed that complexity is strongly correlated with the development effort and cost (Barton et al. 2001; Braha and Bar-Yam 2007; Sinha 2014). Increasing complexity also leads to the phenomena of cost-complexity spiral (Tamaskar et al. 2014). This issue was explicitly addressed in the defense sector, where numerous scholars observed the growing cost of system complexity (Arena et al. 2008; Augustine 1997). There is consensus among academia and industry that there is a need for framework to manage system complexity. For the rest of this article, we will focus our attention to structural complexity, which is related to the architecture of system. In this context, the term structural complexity and architectural complexity are used interchangeably throughout the paper.

To address the management of system complexity, the complexity of the system needs to be measured in a quantitative way. Previous literature on system complexity has often stopped short of prescribing an objective and quantifiable metric, which is an essential element in enabling complexity-inclusive system architecting and active complexity management. Various system complexity metrics are developed and proposed since 1970s, with early works originating from the software engineering sector, such as cyclomatic metric (McCabe 1976) and interconnectivity-based metric (Kafura and Henry 1981). As time progressed, numerous scholars introduced metrics based on different theories. According to literature survey conducted by Tamaskar et al. (2014), complexity metrics developed are based on information theory, network theory and empirical data. Information theory-based metrics determine system complexity using system information content, and metrics developed by Maimon and Braha (Braha and Maimon 1998; Maimon and Braha 1996), Hornby (2007) and Allaire et al. (2012) are in this category. Network theory-based metrics determine system complexity using overall topology of the system represented by the network. Metrics proposed by Mathieson and Summers (2010), Sinha and de Weck (2013) and Chen and Li (2005) are based on network theory. Another notable complexity metric based on empirical data is proposed by Bearden (2003). Application of proposed complexity metrics can be found in numerous literature. Tamaskar et al. (2014) demonstrated the use of their metric on space satellite application. Complexity metrics developed in Sinha and de Weck (2013) were utilized for system architectural configuration complexity assessment (Min et al. 2016), technology infusion impact on system complexity (Min et al. 2015), and product platform complexity assessment (Kim et al. 2016).

Modularity is an important property of a system where it represents the degree to which a system can be divided into several strategic chunks called modules. A module can be defined as “a relatively independent chunk of a system that is loosely coupled to the rest of the system”, (Holtta-Otto et al. 2012) where a chunk may be a subsystem or a single element. In many engineering systems, a module consists of a form where a specific function of the system is embedded during the system synthesis process, enabling system architects to add, remove or enhance a specific system function with ease.

A modular architecture enables relatively independent development of modules due to minimal systemic interaction among them. This aids localized development and technology infusion within the system while limiting the perception of underlying system complexity (Baldwin and Clark 2000). This helps to manage the complexity and the development process better while minimizing the “design churn effect” (Yassine et al. 2003). A highly modular system decomposition strategy might ease the development process by enabling efficient application of divide and conquer principles and by impacting the perception of underlying complexity. This helps manage the actual, underlying system complexity more efficiently. A modular system is deemed to favor flexibility for future modifications and novel technology infusion into the system over a period of time by limiting the degree of interaction between system modules (Baldwin and Clark 2000). Modularity benefit depends on the system decomposition strategy and is a measure of relative cohesion within and across the modules (Newman 2010).

Modular system has several advantages such as ease of assembly (Holtta-Otto and de Weck 2007), ease of maintenance (Suh and Kott 2010) and ease of architecture adaptability (Engel and Browning 2008; Engel et al. 2016). Due to such advantages, modular design strategy is widely used in engineering community to design complex system architectures using manageable and independently upgradable modules, such as fiber placement system and truck powertrain (Engel and Reich 2015). There is a whole field of research concerning modular system design and analysis methodologies. Early works by Alexander (1964) and Simon (1996) set the tone for complex system design by hierarchical decomposition, followed by the seminal work of Baldwin and Clark (2000), which established design rules for modular systems. Numerous methods to determine modularity are proposed in the literature, ranging from more engineering and functionality focused (Guo and Gershenson 2004; Martin and Ishii 2002; Mikkola and Gassmann 2003; Sosa et al. 2003) to more network science centric methods (Blondel et al. 2008; Newman 2010). These works are supplemented by development of other methods, such as modular function deployment (Ericsson and Erixon 1999), flexible platform development (Suh et al. 2007), modular product family concept development (Otto et al. 2016), and modular system design based on architecture options theory (Engel and Reich 2015) to just name a few.

To support modular system design, various modularity metrics have been proposed in time. According to Holtta-Otto et al. (2012), there are two types of modularity metrics. First type of metric measures the degree of coupling in order to determine degree of independence of a module. Several modularity metrics of first kind are introduced over time. Examples include singular value-based metric proposed by Holtta-Otto and de Weck (2007), coupling-based metric proposed by Martin and Ishii (2002), design interface-based metric proposed by Sosa et al. (2003) and the minimum description metric (MDL) proposed by Yu et al. (2007), which measures the sum of model description length and mismatched data description of the architecture. Second category of metric measures the similarity of modules in the system. Primary consideration for modular metrics of this type is connective relationship between elements within the system, being sensitive to tight and loose coupling among them. This metric has been proposed by the network science community and is context-agnostic and solely based on iterative graph partitioning techniques (Newman 2010). This methodology is based on splitting a graph into sub-graphs such that interfaces within sub-graphs are maximized while minimizing those between the sub-graphs. In this paper, we use this approach for finding optimal decompositions from modularity maximization point of view.

Based on comprehensive literature survey in system complexity management, modular system design methodology, complexity metrics and modularity metrics, there are many works published in their respective fields. However, there is virtually no work that approached complex system design problem from the quantitative management of modularity and structural complexity concurrently. Only work that authors are aware of is by Tamaskar et al. (2014), who applied their complexity and modularity measuring framework to the space satellite application. This paper addresses this research gap in complex system design and complexity management through the introduction of Pareto-optimization framework for system architecture from the viewpoint of system modularity and module-level structural complexity allocation. The optimization is based on the assumption that the actual complexity of the system is a property of the system and is independent of the system decomposition strategy employed. However, system decomposition does influence the allocation of the total complexity to individual modules and those tied to the system integration aspect. Using the proposed framework, complex systems can be optimized for degree of modularity, while variation of structural complexity among system modules is minimized. This leads to more equitable allocation of module-level complexity and such allocation strategies enable more stable and efficient planning of development schedule for module-level design and development.

The objective of this research is to introduce a system architecture optimization framework that simultaneously optimizes the allocation of structural complexity to its modules, while maximizing the overall system modularity. To accomplish this, multiple system representation and analysis steps were performed. First, appropriate metrics for measuring structural complexity and system modularity are selected. Second, using selected complexity and modularity metrics, a multi-objective optimization problem for system architecture design is formulated. The formulated optimization framework is then applied to a system, where the number of different modularized concepts of the system is generated and evaluated for its optimality in modularity and structural complexity. Using the optimization result, Pareto-optimal frontier of system’s structural complexity versus modularity is generated for decision maker’s evaluation and final concept selection. In this section, complexity and modularity metrics selection and optimization framework formulation are presented in detail.

Now we are in position to formulate the multi-objective optimization problem to find the optimum system decomposition Gr(.) that minimizes the structural complexity variation among system modules while maximizing the overall system modularity: Let us consider the distribution of module structural complexity, C ^(module) = [C ⁽¹⁾, C ⁽²⁾,…, C ^(k)] where k stands for the number of modules. An approximate measure of standard deviation of module-level structural complexity is given by

The widely used approximation of standard deviation (Mood et al. 1973) above was used in lieu of the sample standard deviation formula to account for sample size effects, since the number of modules in the system being analyzed may be relatively small (Hozo et al. 2005). As stated earlier, an equitable distribution of module structural complexity (i.e., with minimal standard deviation in module-level structural complexity) is desirable from system architecting and design perspective. This will ensure minimal asymmetry in the module-level complexity that needs to be handled by the development groups responsible and there will be no module-level complexity bottleneck that could jeopardize the system development effort and subsequent operation and continuous availability of the system. Such an equitable distribution of module-level structural complexity tends to have a negative impact on the overall modularity of the system, and reduction in variation of module complexity is accompanied with reduction in the degree of modularity of the system. This requires us to make a trade-off between the two aspects of system architecture with the intention of achieving the optimal system architecture. Once the complexity and modularity metrics are available, our aim is to find system decompositions (i.e., association of components to modules/groups) Gr(.) with minimal variation in module structural complexity while maximizing the degree of modularity (i.e., maximization of the modularity index). For the modularity maximization strategy in this research, a two-phase community detection strategy used by Blondel et al. (2008) is implemented. In the first phase of the mentioned strategy, each node (or a component) is assigned a community (or a module) based on modularity-delta maximization using local search. During the second phase, it creates a hierarchical module structure. In the next iteration, the current community structure is the output from the second phase of the previous iteration. Iteration continues until there is no further improvement in overall modularity score, measured by the Q metric. Finally, the associated multi-objective optimization problem can be stated as:

In this case, there is no other constraint to be satisfied for the system decompositions to be feasible. There may be circumstances where additional context-specific constraints are imposed for system decompositions to be feasible. Notice that this is a combinatorial optimization problem and, therefore, outside the realm of applicability of gradient-based optimization methods and we, therefore, adopt the heuristics-based, evolutionary computational approach. Our implementation of multi-objective optimization (i.e., minimization) is based on the concept of domination. A solution Gr _i is said to dominate Gr _j if ∀ M ϵ 1, 2,…, k, f _M(Gr _i ) ≤ f _M(Gr _j ), such that f _M(Gr _i ) < f _M(Gr _j ). Now, from the solution set P, Pareto-optimal solution set P ₀ are those that are not dominated. The Pareto-optimal set in the entire search space S is the non-dominated set. A multi-objective optimization algorithm yields a set of solutions that are not dominated by any solution that it encounters. In this study, we have developed and used a combinatorial variant of AMOSA algorithm (Bandyopadhyay et al. 2008) with adaptive cooling schedule to improve the efficiency of the algorithm (Ingber 1996). The details of this combinatorial, adaptive multi-objective simulated annealing approach will be part of a future manuscript.

To extract necessary information required for the optimization, an abstract model of the system needs to be created. The system model should be easy enough to create with minimal information available about the system. This is due to the timing of design and development phase that the optimization must take a place. The ideal timing for applying proposed optimization is during the preliminary system concept development stage, when there is very little information available for the new system in consideration. Given the constraints, it was decided to use the design structure matrix (DSM) (Eppinger and Browning 2012) as the abstract modeling methodology of choice. Modeling system elements as simple row and column entries of the matrix, and specifying connections between system elements using a simple binary entry allows system model to be created without much of detailed system information, making it ideal for the preliminary concept design phase. Additionally, information required for optimization can be extracted and the optimization results can also be displayed using the DSM.

Once the system DSM is created, several different modular configurations of the system can be generated using several clustering algorithms available, such as the ones proposed by Newman (2010) and Blondel et al. (2008). Additionally, using clustering algorithms mentioned, one can generate population of system configurations, and evaluate them for optimality through the use of heuristic-based optimization algorithm, such as genetic algorithm (Holland 1992) or simulated annealing (Kirkpatrick et al. 1983). For the research presented, we have used various clustering algorithms to generate population of modularized system architecture configurations and applied multi-objective optimization approach presented in the previous section.

In this section, the selection of complexity and modularity metric, formulation of multi-objective optimization problem, and description of system model creation and modular system concept generation are introduced. The proposed framework is demonstrated using a real-life complex system example, which is optimized for structural complexity and modularity.

The train bogie system is the chassis in the railroad locomotive. It is responsible for supporting the train body, keeping the train on railroad track, and provides ride comfort through suspension’s noise, vibration and harshness control. The bogie system used in the case study was specifically designed for the tilting train conceived by Korean Railroad Research Institute (KRRI) Consortium. When the train corners a curve, the train body as well as passengers inside it shift towards the outer side of the curve due to inertia. To prevent the train from going off the track, the train is forced to travel at a slower speed, increasing the overall travel time. To alleviate this problem, a tilting mechanism is added to the bogie to counter the inertia force. By counteracting the force, the train can travel through curves faster, while the inertia force felt by the passengers is minimized. Figure 5 shows the train, the computer-aided design (CAD) representation of a bogie system, and the constructed DSM model of the bogie system shown. In this case study, the train bogie system shown will be used to demonstrate the optimization framework introduced.

To construct the DSM model of the train bogie, relevant engineering information was collected. The engineering firm that designed and developed the particular bogie system provided the engineering data. Complete bill of material (BOM), bogie installation drawing and module drawings were provided to aid the DSM construction. Table 2 shows the list of modules and number of components for all modules.

There are 24 modules that make up the complete train bogie with 153 components in total. It should be noted that some modules are symmetric copies of the other about the center of the train, with left side and right side being symmetric. There are modules with just one component listed. This is due to the engineering firm delegating the actual designs to another supplier and receives specified subsystems in one piece. In this case study, these modules are treated as modules with a single component.

The next task is to construct the actual DSM for the entire bogie system, comprising current modules and components listed in Table 2. This was accomplished using the bogie system installation drawing and individual module engineering drawings, which show connective relationship between components. Resulting DSM of the bogie system is shown in Fig. 5 with 153 rows and columns, each row and corresponding column representing 153 components in the bogie. In the figure, a bold box marks each module. Connections between components are represented by black marks. To turn this into analysis-ready system model cells with connections (marked black) are filled with number “1” while other cells are filled with number “0”.

In addition to information obtained from engineering drawings, the engineering firm provided individual complexity rating for all components shown in the DSM. A rating scale between 1 and 5 was used to rate complexity of a component, 1 being the least difficult to design, and 5 being the most difficult. Due to symmetry in design, nearly identical components were used in front and rear of the bogie (or on the right and left sides of the bogie). For these components with multiple instances, individual component complexity value was divided by the number of instances of that component used in the bogie system. If an axle bearing assembly has complexity of 5, and if 4 of them are used to complete the system, the complexity of each assembly is assigned a component complexity of 1.25. This way, the total component complexity is conserved and estimation of interface complexity is not adversely impacted. Table 3 shows examples of component complexity allocation.

Having estimated the component complexities (α _i ∈ [1,5]), the next step is to estimate the interface complexities (β _ij ) for each interface in the bogie system model. The interface complexities depend on the associated component complexities and are estimated using the following relationship:

The proposed interface complexity estimation process is depicted in Table 4 below.

It can be noted from the above description that the range of interface complexity lies in [ε, 0.5], where ε = 0.1*min(MaxList). Once the component and interface complexities are estimated, we have the overall estimates for the component and interface complexity components (C ₁, C ₂) to be plugged into Eq. (1) for estimating the structural complexity of the bogie system and related bookkeeping.

Table 5 shows module-level structural complexity distribution for each module, along with the bogie system level modularity and overall structural complexity values. The values of integrative complexity (IC) and its normalized counterpart are also provided at the bottom of the table. The integrative complexity is reflective of the amount of integration effort that one might expect at the time of system integration/assembly.

The current system decomposition strategy divides the bogie system into 24 modules. Based on this decomposition, the resulting degree of modularity, Q, is 0.56, indicating a system that is not very modular. We applied modularity maximization strategy used by Blondel et al. (2008) to find out the system decomposition that leads to maximization of Q. This resulted in decomposition strategy which divides the bogie system into 11 modules. This increased the degree of modularity by more than 30%, from 0.56 to 0.74. A DSM view of the modularity-optimized decomposition for the bogie system is shown in Fig. 6, and the corresponding module complexity values and other system metrics are shown in Table 6. Observing the results, the integrative complexity (IC) for the bogie system has dropped from a value of 22.93 for the original decomposition to 9.16 for the modularity maximizing decomposition.

It is interesting to note the changes in distribution of module complexity in this case as compared to the current system decomposition. The modularity maximizing decomposition led to creation of modules with a larger dispersion of module complexity, with three most complex modules taking up a large share of the total module complexity. The standard deviation of module-level complexity distribution has gone up from 4.86 (in case of original decomposition) to 8.32 for the modularity maximizing system decomposition. Modular decomposition structure shown in Fig. 6 and module-level complexity distribution is shown in Table 6 for the modularity maximizing decomposition on the Pareto front, which has 11 modules, as labeled in Fig. 7. One can notice from the table that modularity maximizing decomposition has led to smaller number of highly complex modules (i.e., module 1 and 2 in this case). This asymmetric distribution of module-level complexity can have serious impact on the system development planning and effort where modules with larger complexity require increased attention and possibly lead to schedule slippage in the system development process.

This is formulated as a multi-objective optimization problem stated in Eq. (8), and solved using a simulated annealing-based multi-objective optimization algorithm. The generated Pareto front and a subset of the generated modular decompositions are shown in Fig. 7. It shows the optimal Pareto front with a subset of dominated points obtained by minimizing the spread of the module complexity and maximizing the modularity index. Non-dominated points are traced to show a piece-wise Pareto front, with associated annotations, for clarity. The current decomposition structure of the bogie system, which is located very close to the Pareto front, is also plotted on the same graph.

It is observed that increasing the balance in module-level complexity distribution leads to partial destruction of modularity and vice versa. These results show that there is no free lunch and one has to look for an acceptable compromise between the two aspects. This observation also alludes to an interesting system architecting decision that one needs to take while trying to balance these two attributes. For example, if an original equipment manufacturer (OEM) has great competency in handling higher degree of integrative complexity (i.e., lower modularity), they can opt for any decomposition with simpler and balanced modules (i.e., lower and equitable module-level complexity). On the other hand, having very capable module suppliers might influence OEM’s to focus on decompositions with low interface complexity (i.e., high degree of modularity). This decision is a critical one for complexity management strategies that might be put in place during the system development process.

Another interesting observation is the negative correlation between modularity (Q) and the number of modules (k) in the decomposition shown in Fig. 8. Corresponding to a given number of modules (k), there are points with varying degree of modularity (Q). For a fixed number of modules (k), the average modularity value is computed and plotted in the figure. This negative correlation between the number of modules and modularity indicates that higher degree of complexity leads to a smaller number of highly separable modules.

Combining the observations from the multi-objective optimization study, we see that modularity maximizing decompositions tend to have smaller number of modules with a higher degree of asymmetry in module-level complexity distribution. This can lead to a likely negative impact where a few number of highly complex modules can act as bottlenecks and results in system development delay. On the other extreme, highly equitable and balanced distribution of module-level complexity can lead to increased integrative complexity (i.e., lowering of modularity).

Pareto-optimization results shown in Fig. 7 provide valuable information regarding the modularity-complexity trade-space for complex system design. As the number of modules (k) in the system decreases, the standard deviation for module complexity (Sc) tends to increases in a super-linear manner. It means that with fewer numbers of modules, there are likely some very complex modules with large portion of overall system complexity allocated, while some modules are relatively simple, as shown on the right side of the graphs in Fig. 7. This would lead to severe imbalance in individual module development time, which will be impeded by the more complex module. On the other hand, decomposing the system into many smaller complexity modules will balance the allocation of complexity among modules and minimize Sc, but at the cost of reduced system modularity (i.e., results in higher integrative complexity).

Assessing system decompositions that lies on the Pareto front requires evaluation from several viewpoints. First viewpoint is that of the system design and development team, whose main task is the initial system design development, mapping key system functions to forms. One of the key issues from this viewpoint is that each module in the system is responsible to perform a specific function, which enables coherent initial design and ease of system upgrade when a new technology is developed to enhance that specific function. This will influence the choice of design from the Pareto front. The second viewpoint is that of the system production team, responsible for the initial system assembly. In Figs. 7 and 8, system decompositions with fewer modules were shown to be more modular compared to designs with larger number of modules. This results in smaller integrative complexity, which can have a positive impact on system assembly time, influencing the variable production cost of the system.

The final viewpoint is related to the system operation and maintenance team, responsible for operating and maintaining the system after it is deployed in the field. In some instances, modules consist of collection of components that have roughly equivalent maintenance interval. This allows ease of system inspection and maintenance, minimizing the number of modules to inspect and maintain per scheduled maintenance. When selecting candidate modular design, these factors, along with associated cost data, should be taken into consideration. All these viewpoints should be taken into consideration when the choice of final design is made.

In this section, the proposed multi-objective optimization framework is demonstrated through an actual complex system case study, where a train bogie was used to create optimum modular designs, while attempting to minimize the standard deviation between individual modules. Using the optimization algorithm, a Pareto front was generated, which showed the trade-off between the degree of modularity and standard deviation of module to module complexity. From these modular designs generated, the system architect can assess designs on Pareto front from different perspectives to select top candidates for further concept evaluation.