Abstract
The Nelder–Mead algorithm (1965) for unconstrained optimization has been used extensively to solve parameter estimation and other problems. Despite its age, it is still the method of choice for many practitioners in the fields of statistics, engineering, and the physical and medical sciences because it is easy to code and very easy to use. It belongs to a class of methods which do not require derivatives and which are often claimed to be robust for problems with discontinuities or where the function values are noisy. Recently (1998), it has been shown that the method can fail to converge or converge to nonsolutions on certain classes of problems. Only very limited convergence results exist for a restricted class of problems in one or two dimensions. In this paper, a provably convergent variant of the Nelder–Mead simplex method is presented and analyzed. Numerical results are included to show that the modified algorithm is effective in practice.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
NELDER, J. A., and MEAD, R., A Simplex Method for Function Minimization, Computer Journal, Vol. 7, pp. 308-313, 1965.
WRIGHT, M. H., Direct Search Methods: Once Scorned, Now Respectable, Numerical Analysis 1995, Edited by D. F. Griffiths and G. A. Watson, Pitman Research Notes in Mathematics, Addison-Wesley Longman, London, England, Vol. 344, 1996.
MCKINNON, K. I. M., Convergence of the Nelder-Mead Simplex Method to a Nonstationary Point, SIAM Journal on Optimization, Vol. 9, pp 148-158, 1998.
LAGARIUS, J. C., REEDS, J. A., WRIGHT, M. H., and WRIGHT, P. E., Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions, SIAM Journal on Optimization, Vol. 9, pp. 112-147, 1998.
KELLEY, C. T., Detection and Remediation of Stagnation in the Nelder-Mead Algorithm Using a Sufficient Decrease Condition, SIAM Journal on Optimization, Vol. 10, pp. 43-55, 2000.
RYKOV, A. S., Simplex Algorithms for Unconstrained Minimization, Problems of Control and Information Theory, Vol. 12, pp. 195-208, 1983.
DENNIS, J. E., and TORCZON, V., Direct Search Methods on Parallel Machines, SIAM Journal on Optimization, Vol. 1, pp. 448-474, 1991.
COOPE, I. D., and PRICE, C. J., Frame Based Methods for Unconstrained Optimization, Journal of Optimization Theory and Applications, Vol. 107, pp 261-274, 2000.
DAVIS, C., Theory of Positive Linear Dependence, American Journal of Mathematics, Vol. 76, pp. 733-746, 1954.
COOPE, I. D., and PRICE, C. J., On the Convergence of Grid Based Methods for Unconstrained Minimization, SIAM Journal on Optimization, Vol. 11, pp. 859-69, 2001.
BYATT, D., Convergent Variants of the Nelder-Mead Algorithm, University of Canterbury, Christchurch, New Zealand, MSc Thesis, 2000.
MORÉ, J. J., GARBOW, B. S., and HILLSTROM, K. E., Testing Unconstrained Optimization Software, ACM Transactions on Mathematical Software, Vol. 7, pp. 17-41, 1981.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Price, C., Coope, I. & Byatt, D. A Convergent Variant of the Nelder–Mead Algorithm. Journal of Optimization Theory and Applications 113, 5–19 (2002). https://doi.org/10.1023/A:1014849028575
Issue Date:
DOI: https://doi.org/10.1023/A:1014849028575