Introduction to Optimization

We believe that there is no better way to convince our readers of the interest and applicability of certain mathematical ideas or techniques than to show the type of practical problems and situations that can be tackled, and eventually solved, by using them. At the same time, this initial list of problems and examples may serve as a clear statement of the objectives and goals of this text. Some of the examples might not be completely understandable in a first reading. This should not bother our readers, since we will insist on them throughout this chapter and their significance will be more clearly grasped by the end of it. Most of the examples we will analyze are very well known and academic, in the sense that the size of real problems is not comparable, in the least, to the situations we will study. More complex versions of these problems can be found in advanced textbooks. We think, however, that the main ideas will be conveyed through them and will endow readers with the basic tools for more realistic situations.

It is likely that our readers may have already realized that solving optimization problems explicitly is not an easy task. As a matter of fact, it is typically an impossible job. Not only for those problems with a high number of variables involved is it virtually hopeless to compute by hand the optimal solutions, but even for many modest-sized problems it is almost impossible to solve and manipulate so many equations. It is therefore of primary importance to show how solutions for optimization problems can be efficiently approximated. This need is even more unavoidable from the engineering and practical point of view, since explicit, accurate approximation is as important as the understanding of the underlying problem. As usual, our aim in this chapter is to cover the basic algorithms that researchers have developed over the years to approximate solutions for NLPP without trying to exhaust all possibilities, describe the most recent trends, or even show where the algorithms come from and why they have their particular structure. We will try to motivate, however, the most popular ones so that the reader may have a feeling of their nature without entering into technical details. It is also true that this is a highly technical subject evolving very rapidly, so that the methods that seem best now will probably be abandoned in a few years and replaced either by old ideas in a new framework or by entirely novel techniques. See, for instance, [17] for a very nice survey on all this and the importance of interior point methods nowadays.

We start in this chapter the analysis of optimization problems of a different nature. Specifically, this chapter is devoted to variational problems of finding the infimum of the integrals where Ω ⊂ R ^N , the functions u: Ω → R must be differentiable, and they typically are also constrained in some other way such as having their boundary values on ∂Ω fixed by some preassigned function u ₀, i.e., u = u ₀ on ∂Ω.

Optimal control is an important part of optimization, with many applications in different areas, especially in engineering. In this last chapter, we will simply study the basic ideas for tackling such problems. In particular, we will focus on Pontryagin’s maximum principle, trying to insist on its importance through several examples.