In this paper, we address the issue of whether to optimize or satisfice in model-based engineering design. When dealing with operations research problems in the context of engineering design, one may encounter (i) nonlinear, nonconvex objectives and constraints, (ii) objectives with different units, and (iii) computational models that are abstractions of reality and fidelity, Seeking a single-point optimal solution that meets the necessary and sufficient Karush–Kuhn–Tucker (KKT) conditions makes it impossible to obtain a solution that satisfies all the targeted goals. Instead, a method to identify satisficing solutions that satisfies necessary KKT condition but not the sufficient condition is proposed. These solutions are relatively robust, easy to acquire, and often good enough. In this paper, we demonstrate the combined use of the compromise Decision Support Problems and the adaptive linear programming algorithm, as proposed by Mistree and co-authors. This method is appropriate in formulating design problems and obtaining solutions that satisfy only the necessary KKT condition. Further, the use of the proposed method circumvents complications associated with the use of gradient-based optimization algorithms typically used to solve optimization problems. We discuss the efficacy of our proposed method using four test problems to illustrate how the satisficing strategy outperforms the optimizing strategy in model-based engineering design.
Hinweise
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations
1 Frame of reference
Engineering design of complex systems typically requires a team of designers from multiple disciplines or backgrounds to discover intertwined alternatives and select among them (Reich et al. 2010). When selecting the most appropriate solution among alternatives using well-defined or fuzzy criteria, most designers cannot avoid relying on the utility function, no matter how the utility function is abstracted into a specific mathematical equation or tactically formed. There are different ways to identify and refine a utility function that facilitates decision making: through game theory (Vincent 1983), analytic hierarchy process (AHP) (Saaty 1994), expert-team discussion tools with evaluation matrices such as the Pugh controlled convergence method (Frey et al. 2009), voting theory (Arrow 2012), Pareto comparisons (Sen 2018), etc. Hazelrigg has pointed out the limitations of these methods and made clear why these methods are inappropriate for use in design (Hazelrigg 2010). Hazelrigg advocates the use of utility theory and the use of voting and social choice, the effectiveness and accuracy of shortcuts in engineering, possible risks of poor decision making, and the role of the nature of preferences, etc. Among the utility theory-based methods, optimization is one method that enables designers to select among alternatives. For example, using the simplex algorithm to identify vertex solutions and selecting the optimal one.
The simplex algorithm, proposed by Dantzig (Dantzig et al. 1955; Dantzig 1990), is used to identify solutions at the vertices as possible solutions. The vertices being tested are the vertices of a convex polyhedral set, usually bounded by the constraints and bounds of variables, which is known as the feasible space of a linear programming problem. The simplex algorithm is used to search vertices in this way: given the current vertex, identify the adjacent vertex that on the trajectory that makes the greatest contribution in optimizing the objective function. The algorithm performs this search iteratively until none of the adjacent vertices can be used to further optimize the objective, then the current vertex is returned as the optimal solution.
Anzeige
Not all design problems can be solved using utility theory-based optimization, especially the gradient-based optimization methods. In this paper, we focus on the problems that can be managed using mathematical programming, that is, the problems that can be abstracted and solved by minimizing an objective through determining the value of decision variables while system constraints and bounds on the variables wherein the utility function serves as the objective. In modeling the objective using utility theory we assume that all decision makers are rational and that the objective is perfect. Frequently this is not the case in engineering design.
The mathematical models of engineering design problems often have: (i) nonlinear, nonconvex objectives and constraints, (ii) multiple objectives with different units and scales, and (iii) computational models that are abstractions of reality and fidelity,. In spite of this, these problems are frequently addressed using gradient-based optimization techniques, although the mathematical requirements for optimization, the Karush–Kuhn–Tucker (KKT) conditions are not met. Further, an engineering system comprises of several subsystems. Even if a solution converges for a subsystem, it is unlikely to be a true optimal solution for the system.
Recognizing that mathematical models are abstractions of reality, and the objective function may not be exact in design, in this paper, we offer an alternate approach that involves satisficing. Therefore, instead of using an optimization strategy, we propose using a satisficing strategy that results in solutions that meet an acceptable threshold and are satisfactory, but not necessarily optimal (Simon 1956, 1996). With the satisficing strategy, we account for the inaccuracy of the utility function, and we do not rely on utility theory (Byron 1998). We recognize the mathematical difficulties with optimization and instead use the compromise decision support problem (cDSP) construct which is solved using the adaptive linear programming algorithm (ALP), as developed by Mistree et al. (1993). The ALP algorithm is incorporated in the DSIDES software (Hajihashemi 2023). To obtain a satisficing solution, we relax the requirement for satisfying both the necessary and sufficient KKT conditions and require only the necessary condition to be satisfied. This approach allows us to obtain fairly rapid solutions to problems that are characterized by the difficulties described above (Guo and Chen 2023). The speed with which answers can be obtained allows us to search large regions of design space. Further information is available in the work of Guo (Guo 2021). The satisficing strategy is also used to select a solution from alternatives, but the criteria are different from optimization so that the pool of solution finalists are larger, diverse, and easier to acquire, because a satisficing solution only meets the necessary KKT conditions. The foundation of the satisficing strategy is no longer maximizing the utility of a design but identifying “good enough” solutions that are feasible, enable the goals to achieve their targets to an acceptable extent, and are relatively insensitive to uncertainties due to the nonlinear, multi-objective nature of the design. In essence, our focus, in line with many engineering designers’ interests, is in identifying a range of solutions that are useful, feasible, and relatively insensitive to uncertainty (this results in changes in model structure, boundary, feasibility, and dimension). If a designer wants to find a single-point optimal solution using models that are abstractions of reality, then the alternate approach proposed in this paper should not be used.
Optimizers and satisficers in engineering design are from different cultures. They view engineering design from two very different perspectives. They subscribe to different premises, and, in our opinion, the twain shall never meet. Hence, the title of this paper Optimize or satisfice in engineering design?
Anzeige
In Section 2, we describe the formulation of an optimization problem and a satisficing problem, the KKT conditions, and the assumptions underlying the KKT conditions. In Section 3, four test problems are presented to illustrate the efficacy of adopting the satisficing strategy in the model-based design of complex engineering systems. We end with some closing remarks in Section 5.
2 The formulation of optimization problems and satisficing problems
There are differences between the optimizing and satisficing strategies These include differences in problem formulation, approximation, exploration, and evaluation which are important activities in engineering design (Guo 2021). These differences are embodied in the formulation construct, (the compromise decision support problem or cDSP), the approximation algorithm (the adaptive linear programming algorithm or ALP), exploration mechanism (the XPLORE function), and the evaluation method and are implemented in the DSIDES software (decision support in the design of engineering systems).
2.1 The differences and similarities in the formulation of the two strategies
For a given function \(f(x)\), the Euler–Lagrange equations, developed by Leonhard Euler and Joseph–Louis Lagrange (Euler 1755, 1766), form a system of second-order ordinary differential equations \({\nabla }_{xx}^{2}f(x)\). The solutions to \({\nabla }_{xx}^{2}f(x)\) are stationary points of \(f(x)\). Using the Euler–Lagrange equations, one can solve optimization problems which consist of variables that are to be minimized or maximized within set \(\mathcal{F}\), where \(\mathcal{F}\) is the feasible set of solutions to the problem, bounded by constraints and bounds. An optimization problem \({\mathbb{O}}\) is represented as follows.
The mathematical form of an optimization problem \({\mathbb{O}}\):
$$x^{*} :{ }f\left( {x^{*} } \right)\underline{ \succ } f\left( x \right),\forall x \in \mathcal{F}$$
Any point \(x\) that is a local extrema of the set mapped by multiplying active equations with a non-negative vector is a local optima of \({\mathbb{O}}\) (Courant and Hilbert 1953), denoted as \({x}^{*}\). The elements of such a non-negative vector are Lagrange multipliers, \(\mu\) and \(\lambda\). An optimization strategy is, “formulating an optimization problem and obtaining optimal solutions.”
One variant of optimization is goal programming. The format of a goal programming problem \({\mathbb{O}}^{goal}\) is represented as follows. A target value \(T\) is predefined for the objective function \(f(x)\) as the right-hand side value, so the objective becomes an equation, and we call it a goal (Charnes et al. 1955). \({d}^{-}\) and \({d}^{+}\) are deviation variable measuring the underachievement and over-achievement of the goal toward its target. The problem is solved by minimizing the deviation variables, which is minimizing the difference between \(f\left(x\right)\) and \(T\). In other words, goal programming is aimed at finding \(T\)’s closest projection on \(\mathcal{F}\).
The mathematical form of a goal programming problem \({\mathbb{O}}^{goal}\):
Another strategy, a satisficing strategy, is to formulate a problem using the compromise decision support problem (cDSP) and obtain “good enough” but not necessarily optimal solutions (Mistree et al. 1993). For a nonlinear cDSP, we first linearize the nonlinear equations, including nonlinear constraints and nonlinear goal. Therefore, the nonlinear cDSP is approximated as a linear problem with a linear goal \({Goal}^{li}\), a linear feasible space \({\mathcal{F}}^{li}\) bounded by linear constraints \({g\left(x\right)}^{li}\ge 0\) and \({h\left(x\right)}^{li}=0\). Thus, using the cDSP, we seek the closest projection from the linear goal set onto a linear feasible set. We define the solution as a satisficing solution, and we use \({x}^{s}\) to denote it. A cDSP, \({\mathbb{C}},\) is represented as follows.
The mathematical form of a compromise decision support problem (cDSP) \({\mathbb{C}}\):
A solution to a cDSP \({\mathbb{C}}\) is a satisficing solution denoted by \({x}^{s}\). To understand the difference between \({x}^{*}\) and \({x}^{s}\), the KKT conditions are used. The relationship among optimal solutions, satisficing solutions, and near-optimal solutions is illustrated in Fig. 1.
Fig. 1
Relationship among optimal, satisficing, and near-optimal solutions
×
2.2 The necessary and sufficient Karush–Kuhn–Tucker (KKT) conditions
In 1951, Harold W. Kuhn and Albert W Tucker proposed the Kuhn–Tucker conditions that are foundational to optimization (Kuhn and Tucker 1951). Later it was found that William Karush had summarized the necessary conditions in 1939 (Karush 1939). So, the Kuhn–Tucker conditions are now called the Karush–Kuhn–Tucker (KKT) conditions. The KKT approach is a generalization of the method of Lagrange multipliers (Lange 2013). A nonlinear problem with constraints can be represented as a Lagrange equation. The optimal point of the Lagrange function is a saddle point, so the KKT approach is also known as the saddle-point theorem (Hartley 1960).
The KKT conditions include first-order necessary conditions and second-order sufficient conditions. For an optimization problem \({\mathbb{O}}\), suppose that \({x}^{*}\) is an optimal solution and \({\mu }_{i}\) and \({\lambda }_{j}\) are Lagrange multipliers, the first-order necessary KKT conditions are represented in Eqs. 1–5.
These first-order conditions are necessary conditions for optimality; at an optimal solution \({x}^{*}\), Eqs. 1–5 are always met. However, meeting Eqs. 1–5 does not guarantee that a solution is optimal. To guarantee optimality, the second-order sufficient conditions must also be met (Eqs. 6–8):
Equations 1–8 hold at an optimal solution \({x}^{*}\); whereas only Eqs. 1–5 hold for a satisficing solution \({x}^{s}\).
2.3 The analytical geometric meaning of the Kuhn–Tucker conditions
The analytical geometric meaning of the necessary conditions is that at the solution point \({x}^{*}\), where both the primal and the dual are feasible, the gradient vector of the objective \(\nabla f\left({x}^{*}\right)\) can be represented as the non-zero linear combination of the gradient matrix of all equality constraints \(\nabla {h}_{j}\left({x}^{*}\right)\) and all the active inequality constraints1\(\nabla {g}_{i}\left({x}^{*}\right)\) for \(i\) iff \({g}_{i}\left({x}^{*}\right)=0\); Fig. 2.
Fig. 2
The first-order necessary Kuhn–Tucker conditions are satisfied at \({x}^{*}\)
×
The analytical geometric meaning of the sufficient condition is that at the solution point \({x}^{*}\), there exists a non-zero vector, \(s,\) that is orthogonal to the gradient matrix of all active inequality constraints and equality constraints, such that the second-order matrix of the Lagrange equation with respect to decision variables, \({x}^{*},\) is conditionally positive semidefinite: \({s}^{T}{\nabla }_{xx}^{2}{\text{L}}({x}^{*},{\lambda }^{*},{\mu }^{*}){\text{s}}\ge 0\), \(\forall s\in \mathcal{S}\). In other words, the sufficient condition requires that the Lagrange equation is convex at \({x}^{*}\), that is, in a relatively small range containing \({x}^{*}\), the degree of convexity2 of the objective should not exceed the degree of convexity of the constraints combined by Lagrange multipliers; see Fig. 3. However, the sufficient condition significantly reduces the possibility of achieving a solution for problems with an objective with a relatively higher degree of convexity, that is, problems in which the degree of convexity of the objective is higher than the degree of convexity of any non-zero linear combination of constraints, Fig. 4.
Fig. 3
The convexity requirements for satisfying the second-order sufficient Kuhn–Tucker condition—when the degree of convexity of the objective does not exceed at least one non-negative linear combination of constraints. a Is when a linear combination of constraints is convex, and the contours of f(x) are concave. b Is when both of them are convex, but the degree of convexity of a linear combination of constraints is larger than the degree of convexity of the contours of f(x)
Fig. 4
An optimal solution cannot be identified by Lagrange multipliers—when the convexity degree of the objective is higher than any non-zero linear combination of constraints
×
×
Design problems typically embody a highly convex objective, or a nonconvex and non-concave objective making it virtually impossible to find a solution using gradient-based optimization algorithms. To improve the design to: (i) closely achieve the target of the goals, and (ii) be robust to uncertainties that may cause variations to the boundary of the feasible set, \(\mathcal{F}\), designers need to explore satisficing solutions. That is why we propose a satisficing strategy to deal with engineering design problems.
2.4 Assumptions when using the KKT conditions
There are assumptions when applying the necessary and sufficient KKT conditions to solve an optimization (nonlinear programming) problem. These assumptions also provide mathematical requirements for the problem in order to use the KKT conditions to obtain an optimal solution.
Assumption 1: The mathematical models of the constraints and goals used to model a design problem are complete and accurate representations of reality, namely the physical problem. This is to ensure that the optimal solution to a design problem is also optimal for the corresponding physical system. To use Lagrange functions to solve nonlinear problems, one must assume that the mathematical models accurately capture all the necessary details of the physical problem. Thus, an optimal solution to an engineering design problem is also optimal for its corresponding physical problem. As George Box said, “all models are wrong, but some are useful,” (Box 1979), models are sometimes simplified, or, perhaps, not simplified correctly, and this may not be noticeable. When designing a system, there also may be information that cannot be captured and abstracted, such as hidden variables, implicit functional relationships among variables, factors defined as decision variables which cannot be fully controlled by designers, etc. Therefore, the optimal solution to an engineering design problem formulated as an optimization problem is unlikely to be optimal for the physical system.
Assumption 2: All equations of the problem are differentiable. Although according to the KKT conditions, it is only required that, within a small finite area around the optimal solution, the objective function, active constraints, and equality constraints should be differentiable, but, as we need to solve the Lagrange function globally, it is required that all equations should be differentiable.
Assumption 3: The degree of convexity of at least one non-zero linear combination of all constraints is higher than the degree of convexity of the objective. This assumption is critical to meet the KKT sufficiency condition. This condition guarantees finding an optimal solution to the design problem shown in Fig. 3 but cannot identify an optimal solution to the problem in Fig. 4.
When using an optimizing strategy, all three assumptions are applied, whereas when using the satisficing strategy, only the second assumption is applied; see Fig. 5.
Fig. 5
The assumptions when using the optimizing strategy and the satisficing strategies
×
2.5 Definition of robustness
There are several definitions of “robustness” in the engineering design literature. Carlson and Doyle define robustness as a system’s capacity to maintain some typical features under variations in the behavior of its components and environment (Carlson and Doyle 2002). After reviewing more than 170 papers, Beyer and Sendhoff summarize that, “the appeal of robust design is that its solutions and performance results remain relatively unchanged when exposed to uncertain conditions” (Beyer and Sendhoff 2007). Magni and co-authors indicate that, “robustness is the capacity of a controlled system working satisfactorily under the circumstances it may encounter in practice” (Magni et al. 1997). The performance of a solution algorithm is usually assessed by optimality, diversity of solutions, and the computational complexity of the algorithm (Soltani et al. 2002; Guo et al. 2022). Based on these ideas, in this paper, we consider robustness in model-based engineering design to have three components: (i) the capability of a method to identify feasible solutions given certain complexities, (ii) the capability of a model to maintain a solution under specific uncertainties, and (iii) a diversity of solutions for various representative design scenarios. Explanations of each robustness goal are as follows.
Robustness goal 1: Minimize the probability of having an empty satisficing set \(\mathcal{S}\ne \mathrm{\varnothing }\), given complexities ç; see Eq. 9. ç may include nonlinear and nonconvex equations, multiple goals (objectives) with different units or scales, and the targets of the multiple goals having different levels of achievability.
Robustness goal 2: Maximize the probability of maintaining a good enough (satisficing) solution \({{\text{x}}}^{{\text{s}}}\), under uncertainty \(P\). The representation or result of uncertainty \(P\) is the variation of the feasible set \(\widetilde{\mathcal{F}}\). Unlike some other methods that focus on eliminating uncertainties or minimizing model variations under uncertainties, that is to minimize \({\text{Pr}}\left(P\right)\) or \({\text{Pr}}\left(\widetilde{\mathcal{F}}|P\right)\), we acknowledge that model variations due to uncertainties may not be under a designers’ control or awareness, and the optimal solution, \({x}^{*},\) may not be optimal with model variations, that is \({x}^{*}\notin \widetilde{\mathcal{F}}\) or \(f({x}^{*})\ngeqq f(x)\), so we want to have a relatively high chance of maintaining a satisficing solution \({x}^{s}\); see Fig. 10.
Robustness goal 3: Maximize the diversity of solutions. Why should the diversity of solutions be associated with robustness? “Considering the goodness of a solution set regarding its distribution quality is key to multi-objective problems” (Farhang-Mehr and Azarm 2002). It is important to balance the convergence of the search algorithm and the diversity of solutions for multi-objective problems (Xu et al. 2022). Engineering designers often need different solutions under multiple design scenarios that represent their various design preferences among the multiple goals, meanwhile the diversity of solutions brought about by different solution processes, such as different search starting points, should be minimized. When design preferences evolve, solutions can be adapted. High solution diversity means that range and richness of solutions has been expanded therefore accommodating evolving design preferences. Therefore, we desire a large the sum of squares of solutions under all design scenarios \({SS}_{DS}\) but a small sum of squares of solutions for all starting points \({SS}_{SP}\); see Eq. 11. This idea is based on ANOVA F-test (Wu and Hamada 2011).
In Section 3, we use four test problems to demonstrate using the cDSP and the ALP (Mistree 1993) to realize the satisficing strategy by removing Assumptions 1 and 3 (Fig. 5) and better achieving the robustness goals (Eqs. 9–11).
3 Advantages of using the satisficing strategy in engineering design
We use four test problems TP-I, TP-II, TP-III, and TP-IV (Table 2 and Sections 3.3–3.6), to demonstrate differences between the satisficing strategy and optimizing strategy. The satisficing strategy, in this paper, is accomplished by using the cDSP (the formulation construct) (Mistree et al. 1993), ALP (the solution algorithm) (Mistree et al. 1981, 1993), and DSIDES (the computational platform) (Ming et al. 2018). In this section, we introduce the challenge surmounted by using the cDSP and ALP in four stages of engineering design listed in Table 1, using test problems. We discuss the advantage of surmounting each challenge in the subsection and table association with the challenge.
Table 1
Challenges in the model-based design of complex engineering systems and discussion of advantage
Challenge surmounted
Discussion of advantage
Formulation
Using Goals and Minimizing Deviation Variables Instead of Objectives
3.1 Some typical features of engineering design problems
Key challenges in the model-based design of complex engineering systems? Janet K. Allen and Farrokh Mistree’s design experience includes the co-design of materials (Sinha et al. 2013; Nellippallil et al. 2020; Messer et al. 2010), products (McDowell et al. 2010; Simpson et al. 2001; Pedersen et al. 2013; Choi et al. 2005), and associated manufacturing processes (Nellippallil et al. 2017, 2019, 2020), the design of mechanical, thermal, and structural systems (Nellippallil et al. 2019; Samadiani et al. 2010; Panchal et al. 2005); ships (Mistree et al. 1990) and aircraft (Chen et al. 2000; Simpson et al. 2001) and engineered networks (Milisavljevic-Syed et al. 2020) Based on this experience, in Table 1, we list 5 challenges for consideration. The test problems are very simple, in other words, they are just toy problems (Table 2). Why do we use toy problems instead of more complicated problems? Our rationale for using simple toy problems is: i) to easily visualize the goals and constraints on a two-dimension plane, ii) to demonstrate that some hyperparameters of NSGA II may impact the results (but we do not focus on accurately quantifying how each hyperparameter affects the results), and iii) these toy problems, although quite simple, do have typical complexities that engineering design problems may have3.
Table 2
The features of the test problems (TP)
Test problem (TP)
TP-I
TP-II
TP-III
TP-IV
Feature
Multiple objectives
*
*
*
*
Nonlinear
*
*
*
*
Nonconvex
*
*
*
Objectives with various units (scales)
*
*
Goal targets with various degrees of achievability
*
The methods used in optimizing and satisficing strategies are listed in Table 3. For optimizing methods, we used the “SciPy.optimize” package4. There are ten algorithms in the package, but only three are used to solve the test problems. The algorithms that are not used are: the Nelder–Mead, Powell (Powell’s conjugate direction method), (Powell 1964), CG (conjugate gradient methods), (Straeter 1971), and the BFGS, (Broyden–Fletcher–Coldfarb–Shannon algorithm) (Fletcher 1987) cannot easily manage problems with constraints. Newton-CG (Newton conjugate gradient method) (Khosla and Rubin 1981; Nash 1984), L-BFGS-B (an extension BFGS for large-scale, bounded problems) (Zhu et al. 1997), and TNC (truncated Newton method or Hessian-free optimization) (Zhu et al. 1997). These methods either cannot deal with problems without Jacobians5 or return infeasible solutions without recognizing that they are infeasible. Therefore, in SciPy we use COBYLA, (the constrained optimization by linear approximation algorithm) (Powell 1989, 2007), Trust-constr6, and SLSQP, (sequential least squares programming)7 to solve the test problems.
Table 3
Methods for comparison of the optimizing and satisficing strategies
Strategy
Optimizing
Satisficing
Item
Model formulation construct
Mathematical programming
Goal programming
Compromise Decision Support Problem
Solution algorithms
Constrained Optimization by Linear Approximation (COBYLA) algorithm
Adaptive Linear Programing (ALP) algorithm
Trust-region constrained (trust-constr) algorithm
Sequential Least Squares Programming (SLSQP) algorithm
As NSGA II (non-dominated sorting genetic algorithm II) (Deb et al. 2002) can be used to solve problems with all five features listed in Table 2, therefore, we use NSGA II in MATLAB, as NSGA II can converge near-optimal solutions to engineering design problems with the five typical features we identified (in Table 2). Why is it necessary to study a satisficing algorithm to solve engineering design problems? The reason is that we observe that NSGA II has the following drawbacks that may prevent engineering designers from acquiring insight to improve the design formulation and exploring the solutions space:
NSGA II cannot give designers insight into possible ways to improve the model. NSGA II uses metaheuristics to search for solutions that generationally improve the optimality and diversity of the solutions, but information such as the bottlenecks in the model, the sensitivity of each segment of the model, the rationality of the dimensions of the model, or anything else that may indicate model improvement, are not provided with an NSGA II search.
The performance of NSGA II, including convergence speed, optimality of solutions, and diversity of solutions, is sensitive to hyperparameter settings. Typical hyperparameters, such as the population size and generation number, must be predefined. However, usually designers assume that a larger population size or a larger number of generations returns better solutions, but they may not know how large is adequate, see TP-II and TP-III.
NSGA II requires more computational power than the satisficing algorithms.
Why do we use NSGA II as a verification algorithm if its performance depends on hyperparameter settings? By using a sufficient number of scenarios for hyperparameter settings, we may find the best result, and using that, we can verify the solutions obtained from the ALP are also near optimal. However, although NSGA II can be used to verify a test problem, when managing engineering designs, designers may be not able or willing to invest the effort in running a number of scenarios. The sensitivity to hyperparameter setting is a weakness of NSGA II since not every designer is aware of how sensitive it can be and how many scenarios are sufficient as the size, complexity, and nature of each problem may have different demands for determining the sufficient number of scenarios. Therefore, given that not all designers have the ability or patience to try a number of scenarios, we suggest that there is some uncertainty in NSGA II as to whether it can deliver an optimum solution. NSGA II has been used to verify the solutions to the test problems and we have invested the necessary effort to ensure that the solution is indeed an optimum.
3.3 Advantage of problem exploration: combining interior-point searching and vertex searching
We use a multi-objective, nonlinear test problem (TP-I), to demonstrate the advantages of problem exploration using the cDSP and ALP, which makes designers less reliant on Assumption 1 and are better achieve Robustness Goal 3.
Formulation: The optimization model and the corresponding cDSP of TP-I are shown in Table 4. For the cDSP, a target value is assigned to each objective. Solving the problem requires minimizing the deviation between the achieved value and the goal target while satisfying all constraints and bounds.
Weights (weight vectors or weight scenarios): the problem has two goals (objectives) but the preferences for the goals are unknown. So, we use an Archimedean strategy (scalarization) (Jahn 1985; Lin 1976; Mistree et al. 1993) to combine the goals and use different weights to explore various preferences and understand the tradeoffs among the goals. We apply five weight scenarios, as listed in the “Weight” column in Table 5. Why do we use multiple weight scenarios? Suppose that we do not know how to set the weight—this happens quite often in multi-goal engineering designs—so we need to explore different weight scenarios and associate them with design preferences. We apply the weight generation method in (Messac and Mattson 2002) and use the version customized to engineering design problems (Guo et al. 2021) to obtain weight scenarios. For example, for a two-goal problem, due to the non-orthogonality between the two goals and the variation in sensitivity of each goal along each dimension, the weight scenario (0.5, 0.5) may neither allow the two goals to be equally prioritized, nor allow the two goals to be achieved halfway through to targets. A weight scenario (0.7, 0.3) does not mean that Goal 1’s target can be achieved 70% whereas Goal 2’s target can be achieved 30%. In short, the weight for each goal is not linear with the rate of achieving their targets. Therefore, we try multiple weight scenarios to realize different design preferences. Why do we use five scenarios? First, we start with two weight scenarios, (0, 1) and (1, 0) as “parents.” If the solution to the two weight scenarios are different, we choose another weight scenario as a “child” weight scenario, laying between the parents, for example (0.5, 0.5). This process is repeated until the new child weight scenario has the same solution as one of its parents. For Test Problem 1 (TP-I), using this method, we stop introducing more weight scenarios when five scenarios have been used.
Table 5
Solutions to TP-I (dominated solutions for each weight are in italics)
Weight
Starting point
COBYLA
Trust-constr
SLSQP
ALP
NSGA II
Solution
\({\sum }_{i=1}^{2}{w}_{i}\bullet {f}_{i}(x)\)
Solution
\({\sum }_{i=1}^{2}{w}_{i}\bullet {f}_{i}(x)\)
Solution
\({\sum }_{i=1}^{2}{w}_{i}\bullet {f}_{i}(x)\)
Solution
\({\sum }_{i=1}^{2}{w}_{i}\bullet {f}_{i}(x)\)
Solution
\({\sum }_{i=1}^{2}{w}_{i}\bullet {f}_{i}(x)\)
(1, 0)
(0.5, 1)
(2, 0.5)
4.25
(2,0.5)
4.25
(2,0.5)
4.25
(2,0.5)
4.25
(2,0.5)
4.25
(0, 0)
(0, 0)
2
(0, 0)
2
(0, 0)
2
(2, 0.5)
(2, 0.5)
4.25
(2,0.5)
4.25
(2,0.5)
4.25
(0, 1)
(0.5, 1)
(0.5, 2)
3.13
(0, 0)
6
(0, 0)
6
(0, 0)
6
(0, 0)
6
(0, 0)
(0, 0)
6
(2, 0.5)
(2, 0)
4
(0.5,0.5)
(0.5, 1)
(0.5, 2)
3.69
(0.5 2)
3.69
(0.5,2)
3.69
(0.5,2)
3.69
(0, 0)
4
(0, 0)
(0, 0)
4
(0, 0)
4
(0, 0)
4
(2, 0.5)
(2, 0.5)
3.5
(2,0.5)
3.5
(2,0.5)
3.5
(0.7,0.3)
(0.5, 1)
(0.5, 2)
3.91
(0.5,2)
3.91
(0.5,2)
3.91
(0.5,2)
3.91
(0.5,2)
3.91
(0, 0)
(0, 0)
3.2
(0, 0)
3.2
(0, 0)
3.2
(2, 0.5)
(2, 0.5)
3.8
(2,0.5)
3.8
(2,0.5)
3.8
(0.3,0.7)
(0.5, 1)
(2, 0)
3.4
(0, 0)
4.8
(0, 0)
4.8
(0, 0)
4.8
(0, 0)
4.8
(0, 0)
(0, 0)
4.8
(2, 0.5)
(2, 0.5)
3.2
(2,0.5)
3.2
(2,0.5)
3.2
Starting points: To test the sensitivity of each solution algorithm to starting points, we intuitively select three starting points that are vertices of the feasible space bounded by the constraint and bounds.
Results are listed in Table 5. Using five weight scenarios and trying three starting points, for every solution algorithm, three solutions are obtained (only the best solutions among all starting points are shown in Fig. 6), as visualized on objective space in Fig. 6 and on \(x\)-\(f\left(x\right)\) Space in Fig. 7. As TP-I is to maximize \({f}_{1}\left(x\right)\) and \({f}_{2}\left(x\right)\), in Fig. 6, ideal solutions should be close to the upper right corner of the objective space, and in Fig. 7, the value of the objective function should be as large as possible. For TP-I, the solutions using different algorithms are identical.
Fig. 6
The solution points to TP-I on the objective space using five algorithms: all the five solution algorithms return the same three solution points
Fig. 7
The solution points to TP-I on the x-f(x) space. a Is the 3D illustration of \({f}_{1}(x)\). b Is the 3D illustration of \({f}_{2}(x)\)
×
×
Observation: For a test problem with nonlinear constraints and multiple objectives, the same solutions are obtained using the optimizing and satisficing strategies. The results of the optimizing solution algorithms (COBYLA, Trust-constr, and SLSQP) are sensitive to the starting point. In other words, using the satisficing strategy, Robustness Goal 3 is better achieved versus using optimizing strategy; Eq. 12. If the model is incomplete or of low fidelity, avoiding being trapped in a local optimum allows an engineering designer to have a better chance of obtaining a solution for the physical system, which means that Assumption 1 is less important.
Why are the solutions insensitive to the starting point when using the cDSP and ALP? This is the result of the combination of interior-point searches and vertex searches in the “XPLORE” module in the DSIDES software. In Fig. 8, we illustrate the “XPLORE” procedure to identify appropriate starting points for approximation and solution search. “m” points within the bounds of each variable are chosen randomly. Then, the value of the goal and the feasibility at “m” points are computed. The feasible points are ranked and “n” of them with the lowest deviation from the goal are selected as starting points and used sequentially. The nonlinear equations are approximated as linear equations around a starting point using second-order sequential linearization, and then the linear problem is solved using the revised dual simplex algorithm (Ignizio 1985). Usually “m” is chosen to be a number in [800, 1200], and “n” is chosen to be a number in [5, 20]. Therefore, using “XPLORE,” the feasible points with relatively good goal-achievement values are selected as starting points. To some extent this reduces the chances of returning a local optimum.
Fig. 8
Using the “XPLORE” module in DSIDES to identify the best starting points
×
We summarize the advantages in exploration and fill in the corresponding blank in Table 1 which becomes Table 6.
Table 6
Summary of advantages in exploration
Stage
Feature
Advantage
Assumption removed
Test problem (TP)
Robustness goal
Introduction
Exploration
Combining interior-point search and vertex search
Designers can avoid getting trapped in local optima to some extent and identify satisficing solutions which are relatively insensitive when the starting points changing
3.4 Advantage in approximation: using second-order sequential linearization and constraint accumulation
The test problem II (TP-II), a multi-objective problem with a nonconvex equation, is used to demonstrate how Assumption 3 is removed and robust goals 1 and 3 are achieved better. The formulations of TP-II using the different strategies are in Table 7. The solutions are listed in Table 8. The best solutions (among different starting points) of each design scenario are visualized on objective space and \(x\)-\(f(x)\) Space in Figs. 9 and 10, respectively. When using NSGA II to verify the results, we use two values of the population size hyperparameter. With a population of 50, solutions are closer to the upper right corner, so only the results of using NSGA II with a population of 50 are presented here.
Table 7
The optimization model and compromise DSP of the TP-II
Solutions to TP-II—dominated solutions of each weight are in italics; for each method, the best solution of each design scenario is marked using a capital letter (A, B, C, D, E, F, G, H, and J)
The solution points to TP-II on the objective space—solutions returned by ALP are relatively close to the upper right corner and diverse
Fig. 10
The solution points to TP-II on the x-f(x) space—using trust-constraint and SLSQP fall into local optima. a Is the 3D illustration of f1(x). b Is the 3D illustration of f2(x)
×
×
Observation: Using the cDSP and ALP, we obtain more diverse solutions closer to the upper right corner, that is, more satisficing solutions are obtained when changing design scenarios. Using the satisficing strategy, Robustness Goal 1 is better achieved for nonconvex problems; see Eq. 13. Therefore, designers can manage nonconvex problems, especially when the degree of convexity of a goal exceeds the degree of convexity of the linear combinations of constraints in a local area, which makes designers rely less on Assumption 3.
NSGA II solutions are sensitive to hyperparameter setting but can return non-dominated solutions if the hyperparameter is set appropriately; however, the solutions are not diverse. DSIDES can return satisficing and relatively diverse solutions and robustness goal 3 is better achieved using satisficing; see Eq. 14.
Why does the cDSP and ALP manage nonconvex problems and return solutions close to the non-dominated solutions (the solutions returned by NSGA II)? Two mechanisms in the ALP make it possible to linearize the nonconvex function relatively accurately and converge with diverse solutions—using second-order sequential linearization and the accumulation of the linear constraints.
The ALP algorithm incorporates a local approximation algorithm (Mistree et al. 1981, 1993; Chen et al. 1996), in which a secant plane of the paraboloid (with the second-order derivatives at the starting point as the coefficients) replaces the original nonlinear function, as in Fig. 11, in which two dimensions of a problem being approximated in two iterations (synthesis cycle) is shown. The weighted-sum of the goals is \({\sum }_{k\in K}{W}_{k}\bullet {G}_{k}\). The starting point \({X}_{0}^{0}\) may not be in the feasible region. A random search or a Hook-Jeeves pattern search can be invoked to identify a point \({X}_{1}^{0}\) in the feasible area. In the first iteration, the problem is linearized at \({X}_{1}^{0}.\)
Fig. 11.
The approximation and solution using ALP in two iterations. a Is Iteration i. b Is Iteration i+1
×
In iteration i, Fig. 11a, a nonlinear constraint NFj is linearized at \({X}_{i}^{0}\), so an approximated linear constraint LFi,j is obtained. Doing this for all nonlinear functions and framing a linear model, the revised simplex dual algorithm is used to obtain Solution \({X}_{i}^{*}\). Using the reduced move coefficient (RMC) to linearly combine \({X}_{i}^{0}\) and \({X}_{i}^{*}\), we find the starting point of the next iteration \({X}_{i+1}^{0}\). In Iteration i+1, Fig. 11b, the approximated linear constraints of all previous iterations (in Fig. 11, we only show LFi,j and LFi+1,j) are accumulated, and a solution \({X}_{i+1}^{*}\), is returned and the starting point of iteration i+2 ,\({X}_{i+2}^{0}\), is obtained using the RMC to linearly combine \({X}_{i+1}^{0}\) and \({X}_{i+1}^{*}\). This procedure is iterated for n times.
We use Fig. 12a to illustrate the second-order sequential linearization used in the ALP. In Iteration i, first, NFj (Paraboloid ABC) is approximated to NFi,j” (Paraboloid AB*C*) with the diagonal terms of its Hessian matrix at \({X}_{i}^{0}\) as coefficients. Then, NFj” is approximated to a secant Plane LFi,j (Plane AB*C*). NFj” and LFi,j are computed as follows.
Fig. 12
The two-step linear approximation methods using the ALP. a Illustrates the situation when the second-order paraboloid of the original nonlinear constraint has intersection with plane x1 − x2. b Illustrates the situation when the second-order paraboloid of the original nonlinear constraint has no intersection with plane x1 − x2
×
NFi,j” is obtained using the second-order full derivatives at \({X}_{i}^{0}\), Eq. 15, because the second-order partial derivatives have limited impact on the gradient (Mistree et al. 1981).
If Fig. 16 has real roots, as is the situation in Fig. 12a, by solving Eq. 16 and selecting the root between Eqs. 17 and 18 with the smaller absolute value for each dimension, we obtain an intersection that is closer to the paraboloid in each dimension, that is B* and C*.
If Eq. 16 has no real roots, as is the case in Fig. 12b, NFi does not intersect with Plane x, and the first-order derivative at \({X}_{i}^{0}\) is used, as in Eq. 19.
If the degree of convexity of NFj is positive or slightly negative (greater than − 0.015) at the starting point of the ith iteration, and if the constraint is active in the (i-1)th iteration, that is, \({X}_{i-1}^{*}\) is on the surface of \({NF}_{j}\),, then the accumulated constraints replace \({NF}_{j}\), Eq. 21; otherwise, the single linear constraint in the ith iteration replaces \({NF}_{j}\), Fig. 22.
Then, the revised simplex dual algorithm is invoked to solve the linear problem \({P}_{i}^{L}\), so a solution \({X}_{i}^{*}\) is obtained. A point \({X}_{i+1}^{0}\), which is a point between the starting point \({X}_{i}^{0}\) and the solution \({X}_{i}^{*}\), becomes the starting point of the next iteration.
Constraint accumulation is only used when the local convexity of an equation is greater than − 0.015, which is a value determined by experiments and heuristics. When the degree of convexity of an equation around the starting point is greater than − 0.015, the equation at the linearization point is convex or slightly concave, as the example shown in Fig. 13a and b, or as the contours of \(f(x)\) in Fig. 3a and b, so the approximated linear equations from multiple iterations are accumulated to replace the nonlinear equation. However, if the local convexity of an equation is below − 0.015, as shown in Fig. 13c, or as the contours of \(f(x)\) in Fig. 3, the equation is nonconvex, we only use a single linearized constraint in each iteration to avoid cutting off too much of the feasible space. Using accumulated constraints, designers can manage nonconvex problems accurately.
Fig. 13
Using accumulated constraints from multiple linearization iterations for a convex equation (a) or slightly nonconvex equation (b) and using a single linearized constraint for a significantly nonconvex constraint (c)
×
The advantages in the approximation are summarized and filled into the corresponding blank in Table 1 to become Table 9.
Table 9
Summary of advantages in approximation
Stage
Feature
Advantage
Assumption removed
Test problem (TP)
Robustness goal
Introduction
Approximation
Using second-order sequential linearization
Designers can balance between linearization accuracy and computational complexity
Designers can manage nonconvex problems and deal with highly convex, nonlinear problems relatively more accurately
3.5 Advantage in formulation: minimizing deviation functions instead of objectives
We adjust the scale of \({f}_{2}\left(x\right)\) in TP-II and add the lower bounds to the two objectives to avoid negative values, so this becomes TP-III; see Table 8. We use TP-III to show how one can manage problems with large differences in the scale of multiple objectives using the cDSP and ALP, this removes Assumptions 1 and 3 and better achieves Robust Goals 1 and 2. The solutions are given in Table 10 and visualized on objective space and \(x\)-\(f(x)\) Space in Figs. 14 and 15. The three optimization algorithms cannot return any feasible solutions. Due to the difference in the scale of the two objectives, one cannot use optimizing algorithms to solve the problem by linearly combining them, because: i) the objective with the largest scale dominates the other objective(s), and ii) a linearized function of the weighted-sum objective at a local area can be singular (Table 11).
Table 10
The optimization model and compromise DSP of TP-III
The solution points to TP-III on objective space using two algorithms – solutions returned by NSGA II are more diverse but sensitive to parameter settings and increasing the population and iterations do not always produce better results
Fig. 15
The solution points to TP-III on the \(x\)-\(f(x)\) space. a Is the 3D illustration of f1(x). b Is the 3D illustration of f2(x). All solutions’ \({f}_{1}(x)\) values are close to 1 except B’ and H’; all solutions’ \({f}_{2}(x)\) values are between close to 18 except A’
Table 11
Solutions to TP-III—dominated solutions of each weight are in italics; for each method, the best solution of each design scenario is marked using a capital letter (A’, B’, C’, D’, E’, F’, G’, H’, and J’)
Weight
Starting point
COBYLA
Trust-constr
SLSQP
ALP
NSGA II/III–Population (P)=20/50
Solution
\({\sum }_{i=1}^{2}{w}_{i}\bullet {f}_{i}(x)\)
Solution
\({\sum }_{i=1}^{2}{w}_{i}\bullet {f}_{i}(x)\)
Solution
\({\sum }_{i=1}^{2}{w}_{i}\bullet {f}_{i}(x)\)
Solution
\({\sum }_{i=1}^{2}{w}_{i}\bullet {f}_{i}(x)\)
Solution
\({\sum }_{i=1}^{2}{w}_{i}\bullet {f}_{i}(x)\)
(1, 0)
(0.5, 1)
Cannot manage nonconvex equations with bounds
All solutions violate one or more constraints
All solutions violate one or more constraints
A’ (0.51,1.82)
1
P=20:
G’ (0.55, 1.85)
P=50:
A’ (0.51, 1.82)
P=20:
0.99
P=50:
1
(0, 0)
(2, 0.5)
(0, 1)
(0.5, 1)
B’ (0.51, 1.96)
15.27
P=20:
H’ (0.52, 1.92)
P=50:
B’ (0.51, 1.96)
P=20:
14.86
P=50:
15.36
(0, 0)
(2, 0.5)
(0.5, 0.5)
(0.5, 1)
C’ (0.55, 1.82)
7.5
P=20:
H’ (0.52, 1.92)
P=50:
D’ (0.53, 1.87)
P=20:
7.69
P=50:
7.78
(0, 0)
(2, 0.5)
(0.7, 0.3)
(0.5, 1)
C’ (0.55, 1.82)
4.9
P=20:
C’ (0.55, 1.82)
P=50:
E’ (0.53, 1.88)
P=20:
4.9
P=50:
5.01
(0, 0)
(2, 0.5)
(0.3, 0.7)
(0.5, 1)
C’ (0.55, 1.82)
10.01
P=20:
J’ (0.54, 1.85)
P=50:
F’ (0.53, 1.89)
P=20:
10.49
P=50:
10.6
(0, 0)
(2, 0.5)
×
×
When using NSGA II to solve the problem, we use two values to the population size, 20 and 50 - one of the hyperparameters, because when we use these values, the solution returned is also changed. There are multiple hyperparameters that need to be tuned when using NSGA II. However, since we are not aiming to identify all the hyperparameters that may affect solutions and demonstrate how and how much each hyperparameter affects solutions, we only use “population” as an example to illustrate that different values of a hyperparameter have an impact on the solutions.
Observation: For a multi-objective (multi-goal) problem with nonconvex functions, when the scale of the objectives varies largely, optimizing algorithms cannot return feasible solutions, whereas a satisficing strategy may allow designers to identify adequate solutions. This is explained using the KKT.
When using optimizing algorithms to solve optimization problems, the first-order derivative of Lagrange equation with respect to decision variable x, which is a function of the parameters \(\mathcal{P}\) of the model (the coefficients in objectives and constraints), decision variables \(x\) (if any objective or constraint is nonlinear), Lagrange multipliers \(\mu\) and \(\lambda\), and weights \(\mathcal{p}\) that combining the multiple goals is shown in Eq. 25.
For a satisficing strategy, the first-order derivative of the Lagrange equation contains only the coefficients of the deviation variables in the objective, since only deviation variables \({d}^{\mp }\) constitute the objective (not the decision variables, \(x\)). For a \(\kappa\)-goal cDSP with \(m\) inequality constraints \(g(x)\) and \(\mathcalligra{l}\) equality constraints \(h(x)\), if we use weights to combine the \(\kappa\) goals \(G\left(x,d\right)\), i.e., using the Archimedean strategy to manage a multi-goal cDSP, then the coefficients in the first-order Lagrange equation would be only the weights and \(\tau\) – \(\tau\) is the Lagrange multiplier of the goal functions; see Eq. 26.
When using optimization, the second-order Lagrange equation may still have parameters and decision variables due to nonlinearity; Eq. 27. For satisficing, the second-order Lagrange equation with respect to deviation variables degenerates to zero because the objective of a cDSP is a linear combination of deviation variables; see Eq. 28. That is why satisficing solutions do not need to meet the second-order KKT conditions.
Although both optimal solutions \({x}^{*}\) and satisficing solutions \({x}^{s}\) meet the first-order KKT conditions, the chance of maintaining the first-order KKT conditions for the two strategies under uncertainty varies. If any uncertainty with probability P takes place to an item \(\mathfrak{I}\) in the first-order equation that destroys its equilibrium, we denote it as \(Pr\left(\widetilde{\mathfrak{I}}|P\right)\). For a N-dimension, Q-parameter8, and \(\kappa\)-goal problem, using optimizing strategy, the source of \(\widetilde{\mathfrak{I}}\) can be decision variables \(\widetilde{{x}_{n}}\), Lagrange multipliers \(\widetilde{{\mu }_{i}}\) and \(\widetilde{{\lambda }_{j}}\), and weights \(\widetilde{{\mathcal{p}}_{k}}\); for the satisficing strategy, the source of \(\widetilde{\mathfrak{I}}\) can only be the weights \(\widetilde{{\mathcal{p}}_{k}}\) and the Lagrange multipliers for the goals \(\widetilde{{\tau }_{k}}\). If and only if none of the items under the uncertainty breaks the equilibrium of first-order equation, then the optimal/satisficing solution is still optimal/satisficing under this uncertainty. For a N-dimension, Q-parameter, and \(\kappa\)-goal problem, the probability of maintaining an optimal solution and a satisficing solution under Uncertainty \(P\) are given in Eqs. 29 and 30, respectively.
As the value of any probability is in the range of [0, 1], the more items on the right-hand side we multiply (the more items the probability depends on), the lower the probability becomes. The items in Eq. 30 are fewer than those of Eq. 29. Hence, the chance of maintaining an optimal solution under Uncertainty \(P\) is often smaller than the chance of maintaining a satisficing solution with the same uncertainty; Eq. 31.
In summary, using the satisficing strategy, robustness goals 1 and 2 are better achieved for nonconvex problems with multiple objectives that have various scales; see Eqs. 32 and 33. Using satisficing, designers can deal with nonconvex, multi-objective problems that may be incomplete or inaccurate and with uncertainties, which helps remove Assumptions 1 and 3.
The advantages in formulation are summarized and filled into the corresponding blank in Table 1 so it becomes Table 12.
Table 12
Summary of advantages in formulation
Stage
Feature
Advantage
Assumption removed
Test problem (TP)
Robustness Goal
Introduction
Formulation
Using Goals and Minimizing Deviation Variables Instead of Objectives
At a solution point, only the necessary Kuhn–Tucker conditions are met, whereas the sufficient Kuhn–Tucker conditions do not have to be met
Therefore, designers have a higher chance of finding a solution and a lower chance of losing a solution due to parameterizable and/or unparameterizable uncertainties
3.6 Advantage in evaluation: allowing violations of soft requirements while avoiding violations of rigid requirements
The advantages in a formulation using a satisficing strategy are shown in Section 3.5. One may wonder if optimizing algorithms are used to solve a cDSP, will the results be the same as the ALP? The answer is no, if the problem is nonconvex and the difficulty of achieving the multiple goals varies greatly. To illustrate the advantage in evaluation using a satisficing strategy we use TP-IV—in which the difficulty of achieving each goal’s target varies enormously. This allows designers to rely less on Assumption 1 and better achieve robust goal 1. We formulate TP-IV into a cDSP and use both the optimizing and satisficing strategies to solve it and compare results. The cDSP, visualization of the two goals, solutions, and visualization of solutions on objective space and \(x\)-\(f(x)\) space of TP-IV are given in Table 13, Fig. 16, Table 14, Figs. 17 and 18, respectively.
The left-hand side (objective function) and the right-hand side (target) of the two goals of TP-IV on the \(x\)-\(f(x)\) space. a Is the 3D illustration of f1(x). b Is the 3D illustration of f2(x)
Table 14
Solutions to TP-IV (close-to-non-dominated solutions or good enough solutions)
The solutions to TP-IV using the ALP and NSGA II—NSGA II finds more non-dominated solutions, whereas ALP finds solutions with better Z values (the weighted combination of deviations)
Fig. 18
The solutions to TP-IV on the \(x\)-\(f(x)\) space—there is little difference between ALP solutions and NSGA II solutions (the arrows are used to identify the solutions and their directions have no meaning). a Is the 3D illustration of f1(x). b Is the 3D illustration of f2(x)
×
×
×
Observation: cDSP and ALP are designed to formulate and explore engineering design problems with various levels of achievability of the goals, in other words, when \({{\varvec{f}}}_{{\varvec{i}}}\left({\varvec{x}}\right)/{{\varvec{T}}}_{{\varvec{i}}}\boldsymbol{ }\boldsymbol{ }\gg \boldsymbol{ }\boldsymbol{ }{{\varvec{f}}}_{{\varvec{i}}}\left({\varvec{x}}\right)/{{\varvec{T}}}_{{\varvec{i}}}\) for Goal i and Goal j, which often happens in engineering design, feasible solutions can be identified.
Using the ALP to solve the cDSP, satisficing solutions (compared with the solutions from NSGA II) can be obtained, whereas using optimizing algorithms, no feasible solutions are returned. Why does using the ALP allow designers to obtain satisficing solutions to a cDSP with various degrees of achievability of the goals, but optimizing algorithms do not?
When \({{\varvec{f}}}_{{\varvec{i}}}\left({\varvec{x}}\right)/{{\varvec{T}}}_{{\varvec{i}}}\boldsymbol{ }\boldsymbol{ }\gg \boldsymbol{ }\boldsymbol{ }{{\varvec{f}}}_{{\varvec{i}}}\left({\varvec{x}}\right)/{{\varvec{T}}}_{{\varvec{i}}}\), especially when \({d}_{j}^{-}\) cannot meet its upper bound of \({d}_{j}^{-}\le 1\), its upper bound must be violated. However, optimization algorithms do not have a mechanism to violate the deviation bounds but not to violate other constraints or bounds. Because some optimization algorithms, such as Trust- constr and SLSQP, treat all constraint and bound priorities equally, if at least one of the constraints and bounds is violated, the solution point is considered to be infeasible.
Unlike those optimizing algorithms, using the ALP, the constraints, and the bounds of the decision variables \(x\) are the highest priority, and the bounds of the deviation variables \({d}_{i}^{\pm }\) are assigned second priority. If the violation of any deviation bound allows a point \({x}^{s}\) in the feasible area bounded by the constraints and the bounds of system variables to be found, then \({X}^{S}\) is returned as a solution. So, for an n-dimension, m-constraint (with p inequality constraints and m-p equality constraints), k-goal cDSP: \({x}^{s}\) is a satisficing solution,
if and only if
\(\frac{{f}_{i}\left({x}^{s}\right)}{{T}_{i}}+{d}_{i}^{-}-{d}_{i}^{+}=1,\)\(\forall i=1, \dots k\) //Goal functions hold at \({x}^{s}\), and
\({g}_{i}\left({x}^{s}\right)\ge 0,\)\(\forall i=1, \dots p\)//Inequality constraints are satisfied at \({x}^{s}\), and
\({h}_{i}\left({x}^{s}\right)=0,\)\(\forall i=p+1, \dots m\)//Equality constraints are satisfied at \({x}^{s}\), and
\(lower bound\le {x}^{s}\le upper bound\), and
Minimize the violation of deviation bounds
In summary, for multi-objective, engineering design problems with nonlinear, nonconvex functions, goals with various scales, and the target of the goals with various degrees of achievability, using the cDSP and ALP allows designers to obtain satisficing solutions that are close to the non-dominated solutions obtained by using NSGA II; see Eq. 34. Multi-objective problems can be designed, especially when the scale of the goals varies largely, which helps remove Assumptions 1 and 3.
$$\begin{gathered} \left[ {Pr\left( {{\mathcal{S}} \ne \emptyset |\left\{ {\begin{array}{*{20}c} {Nonconvexity} \\ {Goals with various scales} \\ {Goals^{\prime}targets with various achievabilities} \\ \end{array} } \right\}} \right)} \right]_{Satisficing} \ge \hfill \\ \left[ {Pr\left( {{\mathcal{S}} \ne \emptyset |\left\{ {\begin{array}{*{20}c} {Nonconvexity} \\ {Goals with various scales} \\ {Goals^{\prime}targets with various achievabilities} \\ \end{array} } \right\}} \right)} \right]_{Optimizing} . \hfill \\ \end{gathered}$$
(32)
The advantages in the formulation are summarized and filled into the corresponding blank in Table 1 so it becomes Table 15.
Table 15
Summary of advantages in evaluation
Stage
Feature
Advantage
Assumption removed
Test problem (TP)
Robustness goal
Introduction
Evaluation
Allowing some violations of soft requirements, such as the bounds of deviation variables
Designers can manage rigid requirements and soft requirements in different ways to ensure feasibility
As a result, goals and constraints with different scale can be managed
We demonstrated the advantages of the cDSP and ALP in the four stages using four test problems. We summarize them by adding information to Table 1, so it becomes Table 16.
Table 16
The advantages of the cDSP and ALP in the four stages of engineering design
Stage
Feature
Advantage
Assumption removed
Test problem (TP)
Robustness goal
Introduction
Formulation
Using Goals and Minimizing Deviation Variables Instead of Objectives
At a solution point, only the necessary Kuhn–Tucker conditions are met, whereas the sufficient Kuhn–Tucker conditions do not have to be met
Therefore, designers have a higher chance of finding a solution and a lower chance of losing a solution due to parameterizable and/or unparameterizable uncertainties
Designers can manage nonconvex problems and deal with highly convex, nonlinear problems relatively more accurately
Exploration
Combining interior-point search and vertex search
Designers can avoid getting trapped in local optima to some extent and identify satisficing solutions which are relatively insensitive when the starting points changing
How does the computational efficiency of the ALP compare to the simplex algorithm and interior-point searching algorithms such as NSGA II?
The ALP algorithm is used for linearizing the compromise Decision Support Problems (cDSP) through second-order sequential linearization and solving them with the revised dual simplex algorithm. Second-order sequential linearization requires multiple iterations. In each iteration, the nonlinear equation is linearized. Nonlinear constraints and nonlinear goals are included in a second-order equation by using the diagonal terms of the Hessian matrix of the equation at the linearization point as coefficients. This is used to solve the second-order equation and the solution with the lower absolute value is selected. Using this value and the linearization point as two points to determine a line, this line then becomes the linear equation. Then the problem becomes linear and is solved with the revised dual simplex algorithm. Then, the solution point becomes the linearization point for the next iteration. The linear equations in each iteration are accumulated as linear equations in the linear problem in the latest iteration. The algorithm converges when the difference among the solutions of several iterations is smaller than a predefined threshold, or when the number of iterations reaches the predefined maximum number of iterations.
Suppose there are \(m\) equations in an \(n\)-dimensional design problem, and \({m}{\prime}\) of the equations are nonlinear. In Iteration \(i\), the computational complexity of linearizing the \({m}{\prime}\) equations into second-order equations is \({\rm O}({m}{\prime}\bullet n)\). Suppose that there are \({m}_{i}^{{\prime}{\prime}}\) linear equations – including the \((m-{m}{\prime})\) equations that are linear in the original design problem and \(({m}_{i}^{{\prime}{\prime}}-m+{m}{\prime})\) equations that are linearized and accumulated from Iteration one to the current iteration, so the computational complexity of solving the linearized problem in Iteration \(i\) is polynomial on average (Kelner and Spielman 2006) although the worst-case is \({\rm O}({2}^{n})\). Therefore, the computational complexity of the ALP is polynomial.
As the computational complexity of the simplex algorithm is polynomial for the average-case (Spielman and Teng 2004), the ALP has the same computational complexity as the Simplex algorithm. The computational complexity of NSGA II is \({\rm O}(k\bullet {N}^{2})\), where k is the number of objective functions and N is the population size. The computational complexity of ALP and NSGA II depend on different parameters—the former depends on the dimension and the latter depends on the population size. Thus the computational cost of the ALP does not exceed that of NSGA II. As users usually set a population much larger than the dimension of a problem when using NSGA II, more often than not, NSGA II requires greater computational power than the ALP does.(Spielman and Teng 2004).
5 Closing remarks
In this paper, in the context of the assumptions that are foundational to the KKT conditions we describe the differences between adopting an optimizing strategy and a satisficing strategy when design complex engineering systems; see Section 2.4 To use the optimizing strategy, to obtain optimal solutions, three assumptions are foundational to meet both the necessary and sufficient KKT conditions. To use the satisficing strategy, to obtain satisficing solutions, only the necessary KKT condition is satisfied. We use four test problems to illustrate the different outcomes between the two strategies regarding three robustness goals (Section 2.5), namely, the chance of identifying satisficing solutions given certain complexities, the probability of maintaining a satisficing solution under uncertainties that result in feasible space boundary change, and the diversity of solutions under multiple design scenarios. Given the characteristics of engineering design problems—often having nonlinear, nonconvex goals and constraints, goals which may have different units, and the targets of goals may have different degrees of achievability, we find a satisficing strategy is more robust with respect to our three robustness goals. We verify the satisficing solutions by comparing them with NSGA II solutions. Thence, we conclude that adopting a satisficing strategy more robust and practical for use in model-based engineering design. To reinforce our conclusion, we offer the following references for consideration: Gautham et al. 2017; Nellippallil et al 2018; Pedersen et al. 2013; Samadiani et al. 2010; Simpson et al. 2001.
Acknowledgements
Lin Guo and Janet K. Allen gratefully acknowledge the financial support from the John and Mary Moore Chair at the University of Oklahoma. Lin Guo and Farrokh Mistree gratefully acknowledge financial support from the L.A. Comp Chair at the University of Oklahoma. Lin Guo acknowledges the financial support from the Pietz Professorship and Start-Up Fund at the South Dakota School of Mines and Technology
Declarations
Conflict of interest
The authors declare no competing interests.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations
There are practical, complicated engineering design problems dealt with by the cDSP and ALP, and they cannot be solved using optimization, for example, hot rod rolling design ***(Nellippallil et al. 2017), reconfigurable manufacturing systems, architecting fail-safe supply networks (Rezapour et al. 2018). Those problems incorporate one or more of the typical features as listed in Table 1. We choose not to use those complicated problems as test problems in this paper because describing the problems clearly would require a lot of space and it is difficult to visualize the objectives, feasible space, and solutions for those high dimension problems.
Here, we define a coefficient or an intercept of a constraint or an objective as a parameter. A parameter has a given value (either a constant value or a stochastic value) and the value does not depend on any decision variables.
Die im Laufe eines Jahres in der „adhäsion“ veröffentlichten Marktübersichten helfen Anwendern verschiedenster Branchen, sich einen gezielten Überblick über Lieferantenangebote zu verschaffen.