Recent Advances in Optimization and its Applications in Engineering

Copositive programming is a relatively young field in mathematical optimization. It can be seen as a generalization of semidefinite programming, since it means optimizing over the cone of so called copositive matrices. Like semidefinite programming, it has proved particularly useful in combinatorial and quadratic optimization. The purpose of this survey is to introduce the field to interested readers in the optimization community who wish to get an understanding of the basic concepts and recent developments in copositive programming, including modeling issues and applications, the connection to semidefinite programming and sum-of-squares approaches, as well as algorithmic solution approaches for copositive programs.

This work applies the quasi-LPV technique to the design of robust observers for a class of bioreactors. The system nonlinearities are modeled in terms of two time varying parameter vectors,

θ

(

t

) and

δ

(

t

). The vector

θ

(

t

) contains all nonlinear terms that are only function of the measurements, whereas the remaining terms are lumped into the vector

δ

(

t

). Then, a

θ

(

t

) parameter-dependent Luenberger-like observer is proposed, where the design conditions are given in terms of linear matrix inequality constraints. These conditions ensure regional stability w.r.t. to a set of admissible initial conditions and also minimizes an upper-bound on the

L

2

-gain of the error system. These results are applied to a high cell density bioreator.

We solve a class of convex infinite-dimensional optimization problems using a numerical approximation method that does not rely on discretization. Instead, we restrict the decision variable to a sequence of finite-dimensional linear subspaces of the original infinite-dimensional space and solve the corresponding finite-dimensional problems in a efficient way using structured convex optimization techniques.We prove that, under some reasonable assumptions, the sequence of these optimal values converges to the optimal value of the original infinite-dimensional problem and give an explicit description of the corresponding rate of convergence.

We present a new family of sums of squares (SOS) relaxations to cones of positive polynomials. The SOS relaxations employed in the literature are cones of polynomials which can be represented as ratios, with an SOS as numerator and a fixed positive polynomial as denominator. We employ nonlinear transformations of the arguments instead. A fixed cone of positive polynomials, considered as a subset in an abstract coefficient space, corresponds to an infinite, partially ordered set of concrete cones of positive polynomials of different degrees and in a different number of variables. To each such concrete cone corresponds its own SOS cone, leading to a hierarchy of increasingly tighter SOS relaxations for the abstract cone.

We consider the problem of minimizing the sum of convex functions over a network when each component function is known (with stochastic errors) to a specific network agent. We discuss a gossip based algorithm of [2], and we analyze its error bounds for a constant stepsize that is uncoordinated across the agents.

In this paper we describe several modifications to reduce the memory requirement of the total quasi-Newton method proposed by Andreas Griewank et al.

The idea is based on application of the compact representation formulae for the wellknown BFGS and SR1 update for unconstrained optimization. It is shown how these definitions can be extended to a total quasi-Newton approach for the constrained case.

A brief introduction to the limited-memory approach is described in the present paper using an updated null-space factorization for the KKT system as well as an efficient numerical implementation of the null-space method in which the null-space representation is not stored directly. It can be proven that the number of operations per iteration is bounded by a bilinear order

$$\mathcal{O}(n \cdot max(m,l))$$

instead of a cubic order

$$\mathcal{O}(m \cdot {n^2})$$

for standard SQP methods. Here

n

denotes the number of variables,

m

the maximal number of active constraints, and

l

the user-selected number of stored update vectors.

The problem of geometric ellipsoid fitting is considered. In connection with a conjugate gradient procedure a suitable approximation for the Euclidean distance of a point to an ellipsoid is used to calculate the fitting parameters. The approach we follow here ensures optimization over the set of all ellipsoids with codimension one rather than allowing for different conics as well. The distance function is analyzed in some detail and a numerical example supports our theoretical considerations.

The solution of mixed-integer nonlinear programming (MINLP) problems often suffers from a lack of robustness, reliability, and efficiency due to the combined computational challenges of the discrete nature of the decision variables and the nonlinearity or even nonconvexity of the equations. By means of a continuous reformulation, the discrete decision variables can be replaced by continuous decision variables and the MINLP can then be solved by reliable NLP solvers. In this work, we reformulate 98 representative test problems of the MINLP library MINLPLib with the help of Fischer-Burmeister (FB) NCP-functions and solve the reformulated problems in a series of NLP steps while a relaxation parameter is reduced. The solution properties are compared to the MINLP solution with branch & bound and outer approximation solvers. Since a large portion of the reformulated problems yield local optima of poor quality or cannot even be solved to a discrete solution, we propose a reinitialization and a post-processing procedure. Extended with these procedures, the reformulation achieved a comparable performance to the MINLP solvers SBB and DICOPT for the 98 test problems. Finally, we present a large-scale example from synthesis of distillation systems which we were able to solve more efficiently by continuous reformulation compared to MINLP solvers.

This paper introduces sequential convex programming (SCP), a local optimzation method for solving nonconvex optimization problems. A full-step SCP algorithm is presented. Under mild conditions the local convergence of the algorithm is proved as a main result of this paper. An application to optimal control illustrates the performance of the proposed algorithm.

H-infinity controllers are frequently used in control theory due to their robust performance and stabilization. Classical H-infinity controller synthesis methods for finite dimensional LTI MIMO plants result in high-order controllers for highorder plants whereas low-order controllers are desired in practice. We design fixedorder H-infinity controllers for a class of time-delay systems based on a non-smooth, non-convex optimization method and a recently developed numerical method for H-infinity norm computations.

Optimization problems such as the parameter design of dynamical systems are often computationally expensive. In this paper, we apply Krylov based model order reduction techniques to the parameter design problem of an acoustic cavity to accelerate the computation of both function values and derivatives, and therefore, drastically improve the performance of the optimization algorithms. Two types of model reduction techniques are explored: conventional model reduction and parameterized model reduction. The moment matching properties of derivative computation via the reduced model are discussed. Numerical results show that both methods are efficient in reducing the optimization time.

This paper provides an introduction to the topic of optimization on manifolds. The approach taken uses the language of differential geometry, however,we choose to emphasise the intuition of the concepts and the structures that are important in generating practical numerical algorithms rather than the technical details of the formulation. There are a number of algorithms that can be applied to solve such problems and we discuss the steepest descent and Newton’s method in some detail as well as referencing the more important of the other approaches.There are a wide range of potential applications that we are aware of, and we briefly discuss these applications, as well as explaining one or two in more detail.

This paper deals with the best low multilinear rank approximation of higher-order tensors. Given a tensor, we are looking for another tensor, as close as possible to the given one and with bounded multilinear rank. Higher-order tensors are used in higher-order statistics, signal processing, telecommunications and many other fields. In particular, the best low multilinear rank approximation is used as a tool for dimensionality reduction and signal subspace estimation.

Computing the best low multilinear rank approximation is a nontrivial task. Higher-order generalizations of the singular value decomposition lead to suboptimal solutions. The higher-order orthogonal iteration is a widely used linearly convergent algorithm for further refinement. We aim for conceptually faster algorithms. However, applying standard optimization algorithms directly is not a good idea since there are infinitely many equivalent solutions. Nice convergence properties are observed when the solutions are isolated. The present invariance can be removed by working on quotient manifolds. We discuss three algorithms, based on Newton’s method, the trust-region scheme and conjugate gradients. We also comment on the local minima of the problem.

In this paper, we discuss methods to refine locally optimal solutions of sparse PCA. Starting from a local solution obtained by existing algorithms, these methods take advantage of convex relaxations of the sparse PCA problem to propose a refined solution that is still locally optimal but with a higher objective value.

This work considers the problem of fitting data on a Lie group by a coset of a compact subgroup. This problem can be seen as an extension of the problem of fitting affine subspaces in ℝ

n

to data which can be solved using principal component analysis. We show how the fitting problem can be reduced for biinvariant distances to a generalized mean calculation on an homogeneous space. For biinvariant Riemannian distances we provide an algorithm based on the Karcher mean gradient algorithm. We illustrate our approach by some examples on

SO

(

n

).

We present an algorithm model, called Riemannian BFGS (RBFGS), that subsumes the classical BFGS method in ℝ

n

as well as previously proposed Riemannian extensions of that method. Of particular interest is the choice of transport used to move information between tangent spaces and the different ways of implementing the RBFGS algorithm.

This paper presents a new approach to identify time-varying ARX models by imposing a penalty on the coefficient variation. Two different coefficient normalizations are compared and a method to solve the two corresponding optimization problems is proposed.

The flow of the Kepler problem (motion of two mutually attracting bodies) is known to be geodesic after the work of Moser [21], extended by Belbruno and Osipov [2, 22]: Trajectories are reparameterizations of minimum length curves for some Riemannian metric. This is not true anymore in the case of the three-body problem, and there are topological obstructions as observed by McCord

et al.

[20]. The controlled formulations of these two problems are considered so as to model the motion of a spacecraft within the influence of one or two planets. The averaged flow of the (energy minimum) controlled Kepler problem with two controls is shown to remain geodesic. The same holds true in the case of only one control provided one allows singularities in the metric. Some numerical insight into the control of the circular restricted three-body problem is also given.

In this paper we examine the numerical efficiency and effectiveness of some algorithms proposed for the computation of the effective Hamiltonian, a classical problem arising e.g. in weak KAM theory and homogenization. In particular, we will focus our attention on the performances of an algorithm of direct constrained minimization based on the SPG (Spectral Projected Gradient) algorithm proposed in [3, 4]. We will apply this method to the minimization of a functional proposed by C. Evans in [9] and we will compare the results with other methods.

To operate crane systems in high rack warehouses, reference trajectories have to ensure that the swinging of the crane is under control during the fast movement and disappears at the final point. These trajectories can be obtained solving optimal control problems.

For security reasons the optimal control problem of a main trajectory is augmented by additional constraints depending on the optimal solution of several safety stop trajectories leading to a bilevel optimal control problem.

The concept of consistent control procedures is introduced in optimal control computations. The stock of such procedures of the MSE, a direct method of dynamic optimization, is extended to handle state-constrained and interior arcs. Thus equipped, the MSE can automatically identify optimal control structures and yield arbitrarily exact approximations of optimal solutions by adjusting a bounded number of parameters.

The consistent control procedures for state-constrained and interior arcs are implemented in the MSE, and their performance demonstrated on numerical examples. For state constrained problems with index 1, a two-phase technique is proposed which ensures the exact fulfillment of the state constraint. To enhance efficiency of the method of prototype adjoints applied to consistent representation of interior control arcs, a new ‘freezing’ technique is used.

Optimal and suboptimal protocols are given for a mathematical model for tumor anti-angiogenesis. If a linear model for the pharmacokinetics of the antiangiogenic agent is included in the modeling, optimal controls have chattering arcs, but excellent suboptimal approximations can be given.

We describe a prey-predator model by a nonlinear optimal control problem with infinite horizon. This problem is non convex. Therefore we apply a duality theory developed in [17] with quadratic statements for the dual variables

S

. The essential idea is to use weighted Sobolev spaces as spaces for the states and to formulate the dual problem in topological dual spaces. We verify second order sufficient optimality condition to prove local optimality of the steady state in [

T, ∞

).

In this paper we investigate the performance of unconstrained nonlinear model predictive control (NMPC) schemes, i.e., schemes in which no additional terminal constraints or terminal costs are added to the finite horizon problem in order to enforce stability properties. The contribution of this paper is twofold: on the one hand in Section 3 we give a concise summary of recent results from [7, 3, 4] in a simplified setting. On the other hand, in Section 4 we present a numerical case study for a control system governed by a semilinear parabolic PDE which illustrates how our theoretical results can be used in order to explain the differences in the performance of NMPC schemes for distributed and boundary control.

In this contribution we apply receding horizon constrained nonlinear optimal control to the computation of insulin administration for people with type 1 diabetes. The central features include a multiple shooting algorithm based on sequential quadratic programming (SQP) for optimization and an explicit Dormand-Prince Runge-Kutta method (DOPRI54) for numerical integration and sensitivity computation. The study is based on a physiological model describing a virtual subject with type 1 diabetes. We compute the optimal insulin administration in the cases with and without announcement of the meals (the major disturbances). These calculations provide practical upper bounds on the quality of glycemic control attainable by an artificial

β

-cell.

This article addresses the fast on-line solution of a sequence of quadratic programs underlying a linear model predictive control scheme. We introduce an algorithm which is tailored to efficiently handle small to medium sized problems with relatively small number of active constraints. Different aspects of the algorithm are examined and its computational complexity is presented. Finally, we discuss a modification of the presented algorithm that produces “good” approximate solutions faster.

We consider nonlinear control problems subject to control and state constraints and develop a model-predictive controller which aims at tracking a given reference solution. Instead of solving the nonlinear problem, we suggest solving a local linear-quadratic approximation in each step of the algorithm. Application of the virtual control concept introduced in [1, 4] ensures that the occuring control-state constrained linear-quadratic problems are solvable and accessible to fast function space methods like semi-smooth Newton methods. Numerical examples support this approach and illustrate the idea.

In this paper we consider unconstrained model predictive control (MPC) schemes and investigate known stability and performance estimates with respect to their applicability in the context of sampled–data systems. To this end, we show that these estimates become rather conservative for sampling periods tending to zero which is, however, typically required for sampled–data systems in order to inherit the stability behavior of their continuous–time counterparts. We introduce a growth condition which allows for incorporating continuity properties in the MPC performance analysis and illustrate its impact – especially for fast sampling.

We review a closely connected family of algorithmic approaches for fast and real–time capable nonlinear model predictive control (NMPC) of dynamic processes described by ordinary differential equations or index-1 differential-algebraic equations. Focusing on active–set based algorithms, we present emerging ideas on adaptive updates of the local quadratic subproblems (QPs) in a multi–level scheme. Structure exploiting approaches for the solution of these QP subproblems are the workhorses of any fast active–set NMPC method. We present linear algebra tailored to the QP block structures that act both as a preprocessing and as block structured factorization methods.

The Newton-Picard method for the computation of time-periodic solutions of Partial Differential Equations (PDE) is an established Newton-type method. We present an improvement of the contraction rate by an overrelaxation for the Picard iteration which comes with no additional cost. Theoretical convergence results are given. Further, we extend the idea of Newton-Picard to the solution of optimization problems with time-periodic Partial Differential Equations. We discuss the resulting inexact Sequential Quadratic Programming (SQP) method and present numerical results for the ModiCon variant of the Simulated Moving Bed process.

Optimal control problems of Bingham fluid flow in pipes are considered. After introducing a family of regularized problems, convergence of the regularized solutions towards the orignal one is verified. An optimality condition for the original problem is obtained as limit of the regularized optimality systems. For the solution of each regularized system a semismooth Newton algorithm is proposed.

In this paper optimal Dirichlet boundary control problems governed by the wave equation and the strongly damped wave equation with control constraints are analyzed. For treating inequality constraints semismooth Newton methods are discussed and their convergence properties are investigated. For numerical realization a space-time finite element discretization is introduced. Numerical examples illustrate the results.

Motivated by car safety applications the goal is to deternmine a thickness coefficient in the nonlinear

p

-Laplace equation. The associated optimal problem is hard to solve numerically. Therefore, the computationally expensive, nonlinear

p

-Laplace equation is replaced by a simpler, linear model. The space mapping technique is utilized to link the linear and nonlinear equations and drives the optimization iteration of the time intensive nonlinear equation using the fast linear equation.For this reason an efficient realization of the space mapping is utilized. Numerical examples are presented to illustrate the advantage of the proposed approach.

In this article, we present computational techniques for optimal control of monodomain equations which are a well established model for describing wave propagation of the action potential in the heart. The model consists of a non-linear parabolic partial differential equation of reaction-diffusion type, where the reaction term is a set of ordinary differential equations which characterize the dynamics of cardiac cells.

Specifically, an optimal control formulation is presented for the monodomain equations with an extracellular current as the control variable which must be determined in such a way that wavefronts of transmembrane voltage are smoothed in an optimal manner. Numerical results are presented based on the optimize before discretize and discretize before optimize techniques. Moreover, the derivation of the optimality system is given for both techniques and numerical results are discussed for higher order methods to solve the optimality system. Finally, numerical results are reported which show superlinear convergence when using Newton’s method.

We consider an optimal control problem from hyperthermia treatment planning and its barrier regularization. We derive basic results, which lay the groundwork for the computation of optimal solutions via an interior point path-following method in function space. Further, we report on a numerical implementation of such a method and its performance at an example problem.

The subject of this paper is an optimal control problem with ODE as well as PDE constraints. As it was inspired, on the one hand, by a recently investigated flight path optimization problem of a hypersonic aircraft and, on the other hand, by the so called ”rocket car on a rail track“-problem from the pioneering days of ODE optimal control, we would like to call it ”hypersonic rocket car problem”. While it features essentially the same ODE-PDE coupling structure as the aircraft problem, the rocket car problem’s level of complexity is significantly reduced. Due to this fact it is possible to obtain more easily interpretable results such as an insight into the structure of the active set and the regularity of the adjoints. Therefore, the rocket car problem can be seen as a prototype of an ODE-PDE optimal control problem. The main objective of this paper is the derivation of first order necessary optimality conditions.

The automotive industry represents a significant part of the economic activity, in Europe and globally. Common drivers are the improvement of customer satisfaction (performance, personalization, safety, comfort, brand values,) and the adherence to increasingly strict environmental and safety regulations, while at the same time reducing design and manufacturing costs and reducing the time to market. The product evolution is dominated by pushing the envelope on these conflicting demands.

Based on the dynamical models for machine tool manipulators in Zirn [11, 12], we compare state-of-the-art feedback controls with optimal controls that either minimize the transfer time or damp vibrations of the system. The damping performance can be improved substantially by imposing state constraints on some of the state variables. Optimal control results give hints for suitable jerk limitations in the setpoint generator of numerical control systems for machine tools.

We consider an evolutionary method applied to a topology optimization problem. We compare two material distribution formalisms (static vs. Voronoibased dynamic), and two sets of reproduction mechanisms (standard vs. topologyadapted). We test those four variants on both theoretical and practical test cases, to show that the Voronoi-based formalism combined with adapted reproduction mechanisms performs better and is less sensitive to its parameters.

A second order variational model is tested to extract texture from an image. An existence result is given. A fixed point algorithm is proposed to solve the discretized problem. Some numerical experiments are done for two images.

This paper deals with the design and optimization of a vehicle bumper subsystem, which is a key scenario for vehicle component design. More than ever before, the automotive industry operates in a highly competitive environment. Manufacturers must deal with competitive pressure and with conflicting demands from customers and regulatory bodies regarding the vehicle functional performance and the environmental and societal impact, which forces them to develop products of increasing quality in even shorter time. As a result, bumper suppliers are under pressure to increasingly limit the weight, while meeting all relevant design targets for crashworthiness and safety. In the bumper design process, the structural crashworthiness performance as the key attribute taken into account, mainly through the Allianz crash repair test, but also through alternative tests such as the impact to pole test. The structural bumper model is created, parameterizing its geometric and sectional properties. A Design of Experiments (DOE) strategy is adopted to efficiently identify the most important design parameters. Subsequently, an optimization is performed on efficient Response Surface Models (RSM), in order to minimize the vehicle bumper weight, while meeting all design targets.

We look at a stochastic online scheduling problem where exact joblenghts are unknown and jobs arrive over time. Heuristics exist which perform very well, but do not extend to multi-stage problems where all jobs must be processed by a sequence of machines.

We apply Learning Automata (LA), a Reinforcement Learning technique, successfully to such a multi-stage scheduling setting. We use a Learning Automaton at each decision point in the production chain. Each Learning Automaton has a probability distribution over the machines it can chose. The difference with simple randomization algorithms is the update rule used by the LA. Whenever a job is finished, the LA are notified and update their probability distribution: if the job was finished faster than expected the probability for selecting the same action is increased, otherwise it is decreased.

Due to this adaptation, LA can learn processing capacities of the machines, or more correctly: the entire downstream production chain.

This contribution presents some of the tools developed at Cenaero to tackle industrial multidisciplinary designs. Cenaero’s in-house optimization platform, Minamo implements mono- and multi-objective variants of Evolutionary Algorithms strongly accelerated by efficient coupling with surrogate models. The performance of Minamo will be demonstrated on a turbomachinery design application.

In this paper we discuss the problem of modeling Magnetic Resonance Spectroscopic Imaging (MRSI) signals, in the aim of estimating metabolite concentration over a region of the brain. To this end, we formulate nonconvex optimization problems and focus on appropriate constraints and starting values for the model parameters. Furthermore, we explore the applicability of spatial smoothness for the nonlinear model parameters across the MRSI grid. In order to simultaneously fit all signals in the grid and to impose spatial constraints, an adaptive alternating nonlinear least squares algorithm is proposed. This method is shown to be much more reliable than independently fitting each signal in the grid.

This work introduces a mathematical model for laser cutting which involves two coupled nonlinear partial differential equations. The model will be investigated by linear stability analysis to study the occurence of ripple formations at a cutting surface. We define a measurement for the roughness of the cutting surface and give a method for minimizing the roughness with respect to process parameters. A numerical solution of this nonlinear optimization problem will be presented and compared with the results of the linear stability analysis.