1992 | OriginalPaper | Chapter
Experimental Results for Gradient Estimation and Optimization of a Markov Chain in Steady-State
Authors : Pierre L’Ecuyer, Nataly Giroux, Peter W. Glynn
Published in: Simulation and Optimization
Publisher: Springer Berlin Heidelberg
Included in: Professional Book Archive
Activate our intelligent search to find suitable subject content or patents.
Select sections of text to find matching patents with Artificial Intelligence. powered by
Select sections of text to find additional relevant content using AI-assisted search. powered by
Infinitesimal perturbation analysis (IPA) and the likelihood ratio (LR) method have drawn tots of attention recently, as ways of estimating the gradient of a performance measure with respect to continuous parameters in dynamic stochastic systems. In this paper, we experiment with the use of these estimators in stochastic approximation algorithms, to perform so-called “single-run optimizations” of steady-state systems, as suggested in [23]. We also compare them to finite-difference estimators, with and without common random numbers. In most cases, the simulation length must be increased from iteration to iteration, otherwise the algorithm converges to the wrong value. We have performed extensive numerical experiments with a simple M/M/1 queue. We state convergence results, but do not give the proofs. The proofs are given in [14].