Adaptive density-based robust topology optimization under uncertain loads using parallel computing

Topology optimization aims to find the optimal distribution of material within a design domain by minimizing a cost function subjected to a set of constraints. It is a powerful tool for engineers and scientists providing innovative and high-performance conceptual designs at the early stages of the design process without assuming any prior structural configuration. For this reason, it applies successfully to a broad spectrum of applications under deterministic conditions [16, 45]. However, ignoring the different sources of uncertainty in the design process can degrade the optimal design performance under real-world engineering conditions. These sources of uncertainty may affect the final structural design performance, safety, and reliability. We usually differentiate between epistemic and aleatory uncertainties. The former is due to things one could know but does not know in practice, such as inaccurate measurements due to limited data and unknowledge. The latter includes the natural randomness in a process, such as manufacturing imperfections [49], uncertain loading conditions [36], and variations of the material properties [37], to name but a few. Introducing such sources of uncertainty in the optimization process provides more robust and reliable designs for real-world conditions.

Many topology optimization approaches incorporate sources of uncertainty in the design process. We can broadly classify such methods into non-probabilistic and probabilistic approaches [52]. The former [40] does not require statistical information about the uncertainty of the phenomenon but a qualitative notion about its magnitude. We can include the worst-case approach [12, 19] and the fuzzy techniques [35] in such a category. The worst-case methods formulate topology optimization as a min–max optimization problem, whereas the fuzzy techniques use fuzzy set theory for decision-making and uncertainty propagation. We can employ these techniques when we only have an approximate notion about the measure of the unknown variables, which we usually model using an interval where it could be. The latter approaches use the stochastic characterization of the uncertainty of the phenomenon. There are different stochastic formulations differing in the form uncertainty is taken into account by the functional and in the way is incorporated into the optimization process. We can mention robust topology optimization (RTO) and reliability-based topology optimization (RBTO). The RTO methods consider the statistical moments during optimization minimizing the sensitivity of designs to perturbations of the input data [6]. The RBTO approaches introduce reliability constraints into the deterministic topology optimization formulation to ensure an admissible failure probability to the optimized design [28]. We adopt the RTO formulation considering uncertain loading conditions [11, 21].

The efficient and accurate calculation of statistic moments is crucial for addressing real-world problems. The computational complexity of computing the statistical moments in RTO makes it a computational challenge. Adopting the stochastic collocation method (SCM) allows us to solve the integrals of such statistic moments considering loading uncertainty. We transform the RTO formulation into a weighted multiple-loading deterministic problem. The number of collocation points needed for accurately estimating the statistic moments becomes then the bottleneck of the RTO problem. The integration based on full tensor products suffers from the curse of dimensionality. The Smolyak sparse grid method significantly reduces the number of collocation points without losing much accuracy [64]. However, computing such collocation points still represents a computational challenge due to the memory consumption and the processing time when dealing with finite-element models of real-world applications. RTO can then become intractable due to the computational cost.

We can adopt different strategies to reduce the computational cost of calculating the set of deterministic problems used to compute the integrals of the statistic moments using sparse grid SCMs. We can mention rescaling large systems of equations to reduce the ill-conditioning [60], approximate reanalysis [3] to avoid many analyses of the modified design, using low-accurate approximations of the analysis solution [4], and adopting efficient preconditioning techniques to improve the performance of iterative methods [5]. We also have to mention the multiresolution schemes, which decouple analysis and design discretizations. These methods use a coarse mesh for the analysis and a fine grid for the design variables [33, 42] to improve the computational performance. However, we have a limitation in the maximum resolution of the design variables depending on the order and type of finite elements [22]. We also can improve the computational performance using High-Performance Computing (HPC) to address the computationally intensive tasks of the topology optimization process using multi-core [2, 9, 34] and many-core [24, 36, 44] computing.

Adaptive mesh refinement (AMR) techniques [29] using h-refinement and coarsening permit us to save computational cost and reduce the error by increasing and decreasing the number of finite elements needed in the regions of interest. We usually determine these areas of interest using an error estimator, which indicates either coarsening the unnecessary finite elements or refining them in the zone of interest. Such error estimators can focus on different magnitudes from the analysis and the design discretization. We can mention solution-based error estimators to enhance the approximation of a concerning measure in topology optimization, such as the stress estimation in stress-constrained topology optimization using AMR [47, 48]. We also can mention refining the interface in topology optimization for a better interface definition [41] and sensitivity calculation [7]. Since void material regions contribute meaningfully to the computing burden of topology optimization but little to the accuracy of the optimized design, we adopt an error estimation for detecting weak material regions to coarse them. This strategy improves the computational cost and provides an equivalent material distribution to the one generated on a uniformly fine mesh.

The early work of [14] uses the h-adaptive finite-element method in topology optimization following a recursive strategy. They assume the converged solution on a coarse grid to refine it and start a new optimization on the fine mesh. They do not revisit coarse meshes after the generation of the new fine mesh. For this reason, this approach generates a smoother solution of the resulting topology optimization in the coarse mesh instead of the counterpart solution in the fine mesh. In other words, the topology optimization in the fine mesh can result in a very different design from the solution refining the design obtained in the coarse meshes. Therefore, this strategy may lead to suboptimal designs. That is why, [15] introduce the coarsening in the adaptive method to generate a similar result to the one generated on a uniformly fine mesh. They obtain significant computational improvements when the topology optimization requires relatively small volume fractions, since the design domain is highly void after a few iterations of the optimization process. It is well known that the computation of regions with weak material contributes significantly to the overall computational cost but little to the accuracy of the optimized design.

Some recent works use dynamic AMR with parallel computing to address high-resolution topology optimization using the minimization of structural compliance. [32] propose an adaptive shared-memory topology optimization framework using solid isotropic material with penalization (SIMP) technique. They restrict the region to analyze elastic deformations to a narrow band surrounding the high-density zone. They use a matrix-free implementation of the conjugate gradient solver preconditioned with a geometric multigrid (GMG) method for Cartesian grids. This strategy allows them to obviate the computational effort on large void regions on the fly using an assembly-free instance. [31] propose an adaptive distributed-memory topology optimization framework using the level-set method for evolving a high-resolution unstructured mesh. They use an algebraic multigrid (AMG) method as the preconditioner of the Krylov subspace iteration solver for evaluating the objective function and the reaction–diffusion equation used for the level-set update stage. The proposal achieves clear and high-resolution solid-void material boundary reducing the computational cost by coarsening the mesh distributed in the void domain.

[7] address large-scale stochastic topology optimization problems using adaptive mesh refinement and coarsening with a parallelization scheme distributing the computation through the nodes of large supercomputers. They use sparse grid SCM for calculating the stochastic moments included in the objective function. They group the computing resources into physical computing nodes using each group of threads to calculate the problems at the collocation points. This scheme minimizes inter-node communications reducing bandwidth problems. However, the number of required computing nodes increases as we use more collocation points. This work also uses a level-set method for evolving a high-resolution mesh focusing the computational effort on the zero level-set representing the design interface. A significant drawback of the approach is that it uses a progressive refinement approach similar to the scheme proposed by [14], performing a fixed number of topology optimization iterations with a maximum refinement level and increasing it in the next interval. Thus, this scheme can result in a refined design of the one obtained in the coarse mesh, i.e., it can generate suboptimal designs, as mentioned above.

The proposed work aims to reduce the computational cost of density-based RTO under uncertain loads by combining dynamic AMR techniques and parallel computing. We aim to obtain an equivalent design to the one generated on a uniform fine mesh using distributed memory computing but at a much cheaper computational cost. The bottleneck of RTO using sparse grid SCMs to integrate the statistical moments of structural compliance is the finite-element analysis (FEA) of the set of deterministic problems at the collocation points. We use a fine mesh for regularizing and optimizing to obtain a similar design than using such a fine mesh. For performance reasons, we use a dynamically coarse grid based on the thresholding of design variables for the FEA of the deterministic problems at the collocation points.

We propose the incremental calculation of the contribution of each collocation point in the dynamically coarse mesh using computing buffers [25]. This scheme can take advantage of reusing the displacement solution of the previous FEA [24], the assembled matrix of coefficients, and the setup stage of the multigrid method used for preconditioning the parallel solver. It also permits us to exploit the embarrassing level of parallelism of sparse grid SCMs. Besides, it provides adaptability by fitting the computational resources to the requirements of the RTO problem. On the other hand, we use parallel computing to accelerate the resolution of the system of equations of each deterministic problem. The main issue using this approach is how to organize the computing buffers. We follow the proposal of [7] by grouping the computing resources into physical computing nodes using each group of threads in a buffer to calculate the problems at the collocation points. This incremental calculation by the computing buffers and the thread organization by physical computing nodes allows us to use a similar memory allocation to the one used by the deterministic counterpart. In addition, it minimizes inter-node communications reducing bandwidth problems, and adapts the computing requirements of the RTO problem to the available computational resources.

We organize the remainder of the manuscript as follows. We devote Sect. 2 to reviewing the basis and theoretical background of density-based RTO. Section 3 introduces the parallel AMR technique and the error estimation criteria for coarsening the regions of interest. Section 4 presents the parallel scheme for addressing the RTO problem using parallel computing with distributed memory, detailing the partitioning, solving, incremental calculation of the stochastic moments, regularization, and optimization approach. Section 5 shows the numerical experiments evaluating the performance, feasibility, and scalability of the techniques adopted for taking advantage of parallel multi-core computing in density-based RTO using dynamic AMR techniques. Finally, Sect. 6 presents the conclusion and future works of the proposal.

Let \((\Omega ,\mathcal {F},\mathbb {P})\) be a complete probability space, and \(D\subset \mathbb {R}^{d}\) is a bounded Lipschitz domain with its boundary decomposed into three disjoint parts \(\partial D=\Gamma _u \ \cup \ \Gamma _t \ \cup \ \Gamma _0\), where \(\Gamma _u\) is the part of the boundary with prescribed displacements, \(\Gamma _t\) is the part of the boundary with traction forces, and \(\Gamma _0\) is the part of the boundary with traction-free boundary conditions. Consider the linearized elasticity system under random input datawhere u is the stochastic displacement field, \(\sigma\) is the Cauchy stress tensor, p and \(\bar{t}\) are the body and surface forces, \(\bar{u}\) is the prescribed displacement field, and n is the unit outward normal vector to \(\partial D\). Note that the body and surface forces and the displacement field depend on a spatial variable \(x \in D\) and a random event \(\omega \in \Omega\). The stress tensor \(\sigma\) and the symmetric gradient of the displacement field \(\varepsilon\) relate by the constitutive equationwhere \(\mathbb {C}_{ijkl}\) represents the fourth-order constitutive tensor. Considering the mechanical design as a body occupying a domain \(D^m\) which is part of the reference domain D, we can split the design into two subdomains with the following characteristic function:We can formulate the stochastic topology optimization problem under random input data as the minimization of the structural compliance over admissible design and displacement fields subjected to the maximum material allowed, satisfying the equilibrium equation in its weak form as follows:where \(u=u(x,\omega )\), \(v=v(x,\omega )\), \(\bar{t}=\bar{t}(x,\omega )\), \(p=p(x,\omega )\), \(\Theta =\Theta (x)\), \(\mathcal {V}\) denotes the space of kinematically admissible displacement fields, \(\mathbb {C}_{ijkl}^{0}(\omega )\) is the stiffness tensor for an elastic material, \(V^*\) is the volume target considering the pointwise volume fraction \(\Theta (x)\) for a black-and-white design, and \(a(\cdot ,\cdot )\) and \(l(\cdot )\) are the bilinear and linear forms, respectively, as follows:The formulation of the stochastic topology optimization problem provides a different solution for each realization of the random event \(\omega \in \Omega\). To successfully deal with the problem, we transform the stochastic formulation into a deterministic one. We can then use the conventional optimization algorithms to address the stochastic topology optimization problem under random input data. We adopt the formulation of the RTO as a two-objective optimization problem where we consider the expected value and standard deviation of the structural compliance as a measure of structural robustness. We use a weighted approach to scalarize the multi-objective problem into a single-objective one as follows:where \(\mathbb {E}[\cdot ]\) denotes the expectation operator, \(Var[\cdot ]\) stands for the variance operator, and \(\alpha \ge 0\) is a weighting parameter balancing the first two stochastic moments of the performance functional, which we obtain as follows:The SIMP method relaxes the integer-based problem (6) by introducing an interpolation scheme that penalizes a continuous density variable \(\rho\) \(\in\) [0, 1] characterizing composite materials and allows us to use gradient-based approaches in the optimization. The following power-law interpolation function permits us to rewrite the constitutive tensor equation as:where \(\rho =\rho (x)\), p > 1 is the penalization power, and \(\mathbb {C}_{ijkl}^{min}\) is the fourth-order constitutive tensor of the soft material. This penalization function relates the design variable \(\rho (x)\) and the material tensor \(\mathbb {C}_{ijkl}\) in the equilibrium analysis, satisfying \(\mathbb {C}_{ijkl}(0,\omega )\) \(=\) \(\mathbb {C}_{ijkl}^{min}\) and \(\mathbb {C}_{ijkl}(1,\omega )\) \(=\) \(\mathbb {C}_{ijkl}^{0}(\omega )\). We can select p sufficiently big to penalize intermediate densities. According to [8], we usually require p \(\ge\) 3 to ensure that we do not violate the Hashin–Shtrikman bounds. We can then state the problem aswhere \(\rho =\rho (x)\) is the design variable (constant within each finite element) ranging from solid (\(\rho =1\)) to void (\(\rho =0\)). However, density-based topology optimization is prone to numerical instabilities due to checker-board patterns appearing as penalizing intermediate material densities, mesh-dependency as refining the tessellation of the continuum, and the presence of local minima in the design space [51]. We can introduce a density measure \(\tilde{\rho }=\tilde{\rho }(x)\) that tends to regularize the problem addressing these numerical instabilities. Besides, we use projection techniques to project the filtered designs \(\tilde{\rho }\) into solid/void space \(\bar{\tilde{\rho }} = \bar{\tilde{\rho }}(x)\), which produces designs with a clear physical interpretation [62]. We can then state the problem aswhere \(\bar{\tilde{\rho }}=\bar{\tilde{\rho }}(x)\) is the projected and regularized design field.

Assuming finite-dimensional noise and ignoring the uncertainty in the constitutive tensor \(\mathbb {C}_{ijkl}\), we can state that the random input data of the stochastic partial differential equation (PDE) of (1) depends on a finite number N of real-valued random variables \(\left\{ Y_n \right\} _{n=1}^{N}\) mapping \(J(\bar{\tilde{\rho }},y):\Omega \rightarrow \mathbb {R}\). The problem can be then stated aswhere \(\Lambda = Y\left( \Omega \right) \subset \mathbb {R}^N\) and the mean and variance of the stochastic objective function are as follows:where \(\phi (y)\) is the joint probability density function of \(\left\{ Y_n \right\} _{n=1}^{N}\).

We can approximate the integrals of the stochastic moments of (13) and (14) equations employing numerical quadrature as follows:where the points \(\left\{ y_i \right\} _{i=1}^{N}\) and the weights \(\left\{ \Phi _i \right\} _{i=1}^{N}\) depend on the one-dimensional rule and the selection of tensors. We use the Smolyak sparse grid [53] collocation method for the numerical approximation of the multi-dimensional integrals. Such a technique significantly reduces the number of collocation points while preserving a high level of accuracy compared to the full tensor product quadrature [43, 54]. The Smolyak formulas are linear combinations of the full tensor product quadrature formulas. We refer the reader to [18] for the details of the Smolyak quadrature formula \(\mathcal {A}\left( \ell ,N\right)\) with the level \(\ell\) that is independent of the random dimension N and the generation of the corresponding \(\left\{ y_i \right\} _{i=1}^{N}\) points and \(\left\{ \Phi _i \right\} _{i=1}^{N}\) weights used by (15) and (16).

We use the density measure introduced in [10], the so-called density filter, to regularize the density field \(\rho (x)\). In particular, we perform the filtering operation by the convolution product of the filter F and density \(\rho\) functions as follows:where \(B_R\) denotes the open ball of radius \(R>0\) and the filter function satisfies \(F \ge 0\) \(\forall\) x \(\in\) \(B_R\). The filter requires that the volume is the same for the filtered and unfiltered fields, and thus, we can impose the volume constraint on the unfiltered field [30]. We usually replace the expression (17) bywhere \({\tilde{\rho }}\) is the filtered design field, \(N_e\) is the neighborhood set of elements lying within the radius R, \(w(\cdot )\) is the weighting function \(w(x_i)=R-\Vert x_i-x'\Vert\), and \(v_i\) is the volume of each element associated with the design variable. This filtering technique allows us to cope with numerical problems in topology optimization, in particular, checker-board patterns and mesh-dependent designs. Besides, [10] proved the existence of the topology optimization solution mathematically using such a filter in topology optimization.

We also use the volume-preserving Heaviside filter proposed by [62] to prevent blurred boundaries in the material interface projecting \({\tilde{\rho }}\) to full or empty design variables. In particular, it projects the filtered density values \({\tilde{\rho }}\) above a threshold \(\eta\) to solid and the ones below such a threshold to void using the following smooth function:where the projection parameter \(\beta\) allows us to control the smooth function. We can obtain a similar projected field with the threshold value \(\eta = 0\) than using the Heaviside step filter introduced by [20], ensuring a minimum length scale on the solid phase. The threshold value \(\eta = 1\) performs similar filtering to the modified Heaviside filter introduced by [50], giving rise to a minimum length scale on the void phase. We can write the expression (20) aswhich provides an efficient alternative to calculate the projection avoiding the conditional statements [59].

We can derive the sensitivity of the objective function (12) to the design variable \(\rho (x)\) using the chain rule asobtaining the different terms are as follows:We update the design variables using the method of moving asymptotes (MMA) proposed by [57] for its excellent parallel scalability [1]. The MMA method is suitable for addressing inequality-constrained optimization problems, such as the formulation of (12), by generating and solving a series of approximate convex subproblems instead of the original non-linear problem. The algorithm stops when we reach the maximum number of iterations or when the change variable \(||\rho _{e_{k+1}}-\rho _{e_{k}}||\) and the difference in the objective function \(||f_{k+1}-f_{k}||\) falls below a prescribed value.

Let G be an element containing vertices and edges in two-dimensional cases, including faces in three-dimensional cases. We define a mesh \(\{G_i\}_{i=1}^{N_g}\) as the union of the \(N_g\) elements, such that \(D = \cup _i G_i\) is a connected and bounded region. Let us denote boundary elements as \(\partial G\) and interior elements as \(\langle G \rangle\). Assuming interior elements do not intersect, \(\langle G_i \rangle \cap \langle G_j \rangle = \emptyset\) \(\forall\) \(i \ne j\), we state that a mesh is a conforming grid if the set \(\partial G_i \cap \partial G_j\) \(\forall\) \(i \ne j\) is a shared vertex, shared edge, shared face, or an empty set. We define a hanging node as a vertex lying in the interior of an edge or in a face in three-dimensional cases. We call non-conforming meshes the ones containing at least one hanging node.

Refinement of one parent element \(G_i\) consists of replacing it with at least two child elements \(G_{ik}\), such that \(G_i = \cup _k G_{ik}\). We remove parent elements after refining them but storing the links to the child elements in a refinement tree. This information facilitates the coarsening by reintroducing the parent element and removing the child elements. We say that a non-conforming mesh is consistent if lower-dimensional mesh entities (vertices, edges, and faces) are either identical or a proper subset of the other. We refer to the smaller entities as slaves and the larger ones containing other entities as masters. Otherwise, we call them conforming entities.

We adopt the method proposed by [58] for non-conforming AMR of unstructured meshes. This method cast the elimination of constrained degrees of freedom (DOFs) as a form of variational restriction decoupling the AMR method and the governing equation, facilitating the integration of the AMR functionalities in the physics simulation. We can discretize the weak variational problem of (12) constructing an approximate finite-dimensional subspace \(\mathcal {V}_h \subset \mathcal {V}\) on the \(\{G_i\}_{i=1}^{N_g}\) mesh assembling the stiffness matrix \(A(\bar{\tilde{\rho }})\) and the vector of load forces \(f(y_i)\), such thatwhere d is the dimension of the problem and \(y_i\) is the corresponding stochastic collocation point. In non-conforming meshes, we have a larger partially conforming space \(\widehat{\mathcal {V}}_h\) \(\supset\) \(\mathcal {V}_h\) with \(\widehat{\mathcal {V}}_h \in \mathbb {R}^{\hat{d}}\) and \(\hat{d} > d\) since slave entities containing hanging nodes have DOFs independent of master DOFs. We can restore the conformity in the interfaces containing hanging nodes constraining slave entities (slave DOFs) by interpolating the finite-element functions of their masters. We obtain the solution vector \(\hat{u}_h\) in the partially conforming space \(\widehat{\mathcal {V}}_h\) aswhere \(w_h\) represents all slave DOFs that we can evaluate by the linear interpolation \(w_h = w u_h\) using the interpolation operator w. We can write (27) aswhere I is the identity matrix and \(P_c\) is the conforming prolongation matrix. We can assemble the stiffness matrix \(\hat{A}(\bar{\tilde{\rho }})\) and load vector \(\hat{f}(y_i)\) in the space \(\widehat{\mathcal {V}}_h\) obviating the hanging nodes in the mesh. The corresponding linear system \(\hat{A}(\bar{\tilde{\rho }}) \hat{u}_h = \hat{f}(y_i)\) provides a non-conforming solution where the slave DOFs are not constrained. Using the expression (28), the variational formulation of equation (26) on \(v_h\) becomesThus, we can solve the smaller problem \(P_c^T \hat{A}(\bar{\tilde{\rho }}) P_c u_h = P_c^T \hat{f}(y_i)\) for the corresponding stochastic collocation points \(y_i\) where the slave DOFs are eliminated and then prolonging \(u_h\) to obtain the conforming solution \(\hat{u}_h \in \widehat{\mathcal {V}}_h\). We remark that factoring \(P_c\) out of the AMR governing equation facilitates the incorporation of AMR in different problems. The challenge is then the efficient construction of this \(P_c\) operator. We refer to the work of [58] and the implementation of the AMR technique using the mfem [39] and hypre [27] libraries for the details of the serial and parallel construction of the \(P_c\) operator.

The AMR technique requires an error estimator to determine the regions of interest. We choose the areas with void material as regions of interest, because they contribute significantly to the overall computational cost but little to the accuracy of the optimized design. Besides, such areas contribute meaningfully to the ill-conditioning of the elasticity system to solve using FEA. Thus, the error estimators identifying sets of design variables with void material for coarsening them can meaningfully increase the performance [61]. Since we focus on finding the same solution as the one obtained in a fine mesh but at a cheaper computational cost, we use the following error estimator:where \(\varepsilon _v\) is the error estimator based on the design variable, and \(D_{rt} \subset D\) is the part of the domain in the different levels of the refinement tree: the parent element for refinement and the set of child elements for coarsening to the parent element. We can choose the threshold \(\varepsilon _v \le E^{min} + \varepsilon\), with \(E^{min}\) the elastic modulus used for the constitutive tensor \(\mathbb {C}_{ijkl}^{min}\) of void material and \(\varepsilon\) a small value for coarsening the void elements by thresholding.

Parallel computing approaches divide complex problems into smaller and more workable subproblems by distributing their computation across computational resources. The performance of such methods depends on the workload of each subdomain and the communications needed between them. We can optimize the latter using efficient partitioning techniques. We do this by minimizing the number of interface elements and enforcing contiguous partitioning in the tessellation [26]. One key advantage of density-based topology optimization methods is that they operate on a fixed mesh. Thus, the domain partitioning and communications between subdomains are the same during the optimization.

Figure 1 shows the flowchart of the parallel framework using distributed memory for density-based RTO using AMR techniques. This framework aims to increase the performance addressing the computational bottleneck of RTO formulation: the calculation of the stochastic moments of the structural compliance of (12) using sparse grid SCMs. Such a calculation requires the evaluation of N collocation points \(\left\{ y_i \right\} _{i=1}^{N}\) needing an intensive use of computational resources, including computing time and large memory allocation. We evaluate the deterministic problems at the collocation points \(y_i\) and their contribution to the sensitivity functional in a coarse mesh, which requires the projection of the design variables to penalize the constitutive tensor \(\mathbb {C}_{ijkl}\) of the corresponding deterministic models.

First, we tessellate the model using the optimization criteria mentioned above. We then initialize the communications needed to perform the distributed operations: the coarsening and refinement, the regularization of design variables, and the solving of the system of equations of the deterministic problems at the collocation points \(y_i\) using FEA. We use computing buffers to organize and adapt the parallel computing resources to the RTO problem. This strategy permits us to exploit the embarrassing parallelism of SCMs as it increases the number of computing nodes. It also allows computing buffers to reuse information and operators from the FEA of previous collocation points.

Computing buffers make intensive use of intra-node communications during the analysis of the deterministic problems using FEA and the calculation of the contribution to the stochastic sensitivity of the objective function. We also use the intra-node communications on one computing buffer to regularize the design variables and update them using the parallel implementation of the MMA optimization algorithm proposed by [1]. We minimize inter-node communications to gather contributions from computing nodes to calculate the stochastic objective function and sensitivities, which reduces bandwidth problems significantly.

The following sections present the ingredients adopted to exploit the different levels of parallelism of the RTO formulation. Section 4.1 describes the techniques adopted for parallel solving the elasticity system of the deterministic problems at the collocation points \(y_i\) using FEA. Section 4.2 introduces the strategy adopted using computing buffers to adapt the available resources to the RTO problem. We also describe the incremental calculation approach for computing the stochastic moments of the objective function and the contribution of each deterministic problem at the collocation points \(y_i\) to the functional sensitivity. Section 4.3 details the parallel method for regularizing the design variables.

The parallel framework of density-based RTO using AMR schemes aims to reduce the computational bottleneck of the optimization, the numerical approximation of the multi-dimensional integrals using sparse grid SCM evaluating each collocation point using FEA. We test the performance of preconditioning using AMG and GMG methods during the optimization with the incremental calculation of the stochastic moments of the structural compliance obviating AMR approaches. We also check the improvement in performance using AMR strategies in two- and three-dimensional topology optimization problems. Besides, we evaluate the performance using a different number of computing buffers. We present the cumulative wall-clock time for the RTO process to show the computing benefits while providing similar results to obviating the use of AMR techniques.

We run the numerical experiments using two computational systems: one workstation and two computing nodes connected with a 10 Gigabit Ethernet network working as a computer cluster. The former allows us to evaluate the performance using a modern workstation with modest computational resources, whereas the latter shows the scalability increasing the parallel computational resources. We equip the former workstation using an AMD \(\text {Ryzen}^{\text {TM}}\) 9 3950X CPU with 16 cores (32 simultaneous multithreading) at 3.5 GHz (turbo core speed of 4.7 GHz) and 64GB of RAM, whereas we use two Intel Xeon E5-2687W v4 CPUs with 12 cores (24 hyper-threading) at 3.0 GHz (turbo boost speed of 3.5 GHz) per CPU connected with dual LGA2011-v3 and 256GB of RAM per computing node in the computer cluster. Thus, the computing nodes can run 24 processes in parallel.

The battery of experiments consists of four topology optimization problems using structured meshes. In particular, the experiments consist of the topology optimization of a short cantilever in two and three dimensions, a 2D Michell-type structure, and a 3D double-hook structure. We parameterize the geometry of all the experiments using \(H=1\). All the experiments use a tolerance \(10^{-6}\) for the parallel conjugate gradient method we use for solving the deterministic problems at collocation points \(y_i\) using FEA. We use a penalization power \(p=3\) for the power-law interpolation function of (12). We set Young’s modulus \(E^0=1\) for the constitutive tensor \(\mathbb {C}_{ijkl}^{0}\) of solid material and \(E^{min}=10^{-9}\) for the constitutive tensor \(\mathbb {C}_{ijkl}^{min}\) of void material. We use an error estimator \(\varepsilon _v = 2E^{min}\) to determine the void regions. The coarsening of void areas allows us to obtain similar solutions to the one obtained in the fine mesh but at a cheaper computational cost.

The numerical experiments using the AMG preconditioner use a strength threshold \(\theta =0.5\) for the coarsening, a truncation factor \(\tau =0.0\) (no truncation) for constructing the interpolation operator, and the HMIS algorithm for parallel coarsening. The numerical experiments using the GMG preconditioning use the damping factor \(w_s\) for the Jacobi smoothing that we calibrate to ensure convergence in the different experiments [38]. Both multigrid approaches use \(\varphi _1=\varphi _2=4\) pre- and post-smoothing steps in the preconditioning of the iterative solver. We follow a continuation strategy for the \(\beta\) parameter of the projection of (21), following a heuristic strategy for each experiment.

We solve the experiments using multi-core computing with distributed memory with standardized and portable MPI mechanisms. We regularize and update the design variables using the fine mesh, whereas the computing buffers use the dynamically coarse mesh for the incremental evaluation of the collocation points \(y_i\) using FEA and calculating contributions to the functional sensitivity. This strategy requires adaptive coarsening and refinement up to the fine mesh at all topology optimization iterations. We test the increment in performance of the whole process with the best reference implementation without using AMR. We present below the numerical results of the four density-based RTO experiments evaluating the computational aspects of the parallel AMR strategy using multi-core computation.

We combine parallel computing using distributed memory and dynamic AMR techniques to address density-based RTO problems efficiently. The proposed strategy allows us to address the computationally intensive problem of the numerical approximation of multi-dimensional integrals using sparse grid SCMs. We introduce computing buffers in the framework to adapt the evaluation of deterministic problems at the collocation points to the parallel computing resources. Such computing buffers calculate the collocation points of the SCM incrementally, requiring a memory allocation similar to the one used in the deterministic counterpart if we group the computing resources by physical nodes into computing buffers. The sequential calculation of the collocation points permits us to save heavy computations, improving the performance using modest computational resources. We can mention the coarsening and refinement only once per RTO iteration and computing buffer, which requires a relevant computational cost with intensive use of communications.

The numerical experiments show that the proposal permits us to simulate efficiently a significant number of collocation points using FEA. The Michell-type model with almost 250k finite elements requires 20700 analyses using FEA in 300 topology optimization iterations, which takes less than 2 h for the density-based RTO using the AMR strategy. The incremental parallel calculation of the collocation points of the sparse grid SCM used by the computing buffer also allows us to address the problem obviating the AMR method, requiring more than 12 h using the best preconditioning using AMG for the two-dimensional problem. The three-dimensional short-cantilever and double-hook models have 442k and 720k finite elements, respectively. We evaluate the second experiment using a different number of computing buffers, showing performance increments as exploiting the embarrassing level of parallelism of sparse grid SCMs. Both experiments require 8700 simulations, because they require solving the same number of occurrences of the state equation. Such experiments take 5 and 7 h using the AMR strategy. We can address these RTO problems obviating the AMR strategy, but the resolution can require up to a couple of days. The proposal allows us to address RTO problems with high computational requirements using modest computational resources.

The experimental results show a significant increment in computing performance for the overall computing time using AMR techniques. We also evaluate the scalability of the proposal by organizing the computing buffers in different forms, showing that grouping the computing threads of the computing buffers by physical computing nodes presents computing advantages and permits us to reduce the intensive use of inter-node communications. In this sense, we have to propose the performance evaluation of using many computing buffers per computing node as future work. We also have to mention the introduction of solution-based error estimators needed to coarse and refine solid material regions for stress-constrained RTO using AMR as future work. Finally, we have to mention as a future extension of the work evaluating the performance using some PDE filter for regularizing, which will permit us to exploit load balance in the non-conforming mesh used for solving the deterministic problems at the collocation points.