1. INTRODUCTION

Formally, Mathematical Optimization is the process of subject to the constraints: where \(f(\mathbf{x})\), \(g_j(\mathbf{x})\) and \(h_j(\mathbf{x})\) are scalar functions of the real column vector \(\mathbf{x}\). Mathematical Optimization is often also called Nonlinear Programming, Mathematical Programming or Numerical Optimization. In more general terms Mathematical Optimization may be described as the science of determining the best solutions to mathematically defined problems, which may be models of physical reality or of manufacturing and management systems. The emphasis of this book is almost exclusively on gradient-based methods. This is for two reasons. (i) The authors believe that the introduction to the topic of mathematical optimization is best done via the classical gradient-based approach and (ii), contrary to the current popular trend of using non-gradient methods, such as genetic algorithms GA’s, simulated annealing, particle swarm optimization and other evolutionary methods, the authors are of the opinion that these search methods are, in many cases, computationally too expensive to be viable. The argument that the presence of numerical noise and multiple minima disqualify the use of gradient-based methods, and that the only way out in such cases is the use of the above mentioned non-gradient search techniques, is not necessarily true as outlined in Chapters 6 and 8.