Mathematical Preliminaries

This chapter presents some of the basic mathematical concepts that are used frequently in the later chapters. A review of matrix notation and some of the basic concepts from linear algebra are presented in the first section. Taylor series approximation, which plays an important role in developing various optimization techniques, is presented in section 2. Definitions of the gradient vector and the Hessian matrix of functions of n variables are introduced in this section also. As an application of the Taylor series, and also since nonlinear equations are encountered frequently in solving optimization problems, the Newton-Raphson method for the numerical solution of nonlinear equations is presented in the third section. A discussion of quadratic functions and convex functions and sets is included in the last two sections.