Positive Bases and Nonsmooth Optimization

Abstract

Chapter 3 introduced a first practical DFO algorithm for unconstrained optimization, the coordinate search (CS) algorithm. While it was proven to converge to the first order in some circumstance (see Theorem 3.4), it was also noted that the algorithm can fail on very simple nondifferentiable convex functions (see Example 3.3). The CS algorithm is an example of the subclass of DFO methods called direct search methods. Direct search methods are methods that work from an incumbent solution and examine a collection of trial points. If improvement is found, then the incumbent solution is updated; while if no improvement is found, then a step size parameter is decreased and a new collection of trial points is examined.