Likelilood ratio gradient estimation: an overview

ABSTRACT

The likelihood ratio method for gradient estimation is briefly surveyed. Two applications settings are described, namely Monte Carlo optimization and statistical analysis of complex stochastic systems. Steady-state gradient estimation is emphasized, and both regenerative and non-regenerative approaches are given. The paper also indicates how these methods apply to general discrete-event simulations; the idea is to view such systems as general state space Markov chains.

REFERENCES

• 1: Peter W. Glynn, Stochastic approximation for Monte Carlo optimization, Proceedings of the 18th conference on Winter simulation, p.356-365, December 08-10, 1986, Washington, D.C., United States [doi>10.1145/318242.318459]
• 2: Glynn, P. W. (1986b). Sensitivity analysis for stationary probabilities of a Markov chain. In: Proceedings of the Fourth Army Conference on Applied Mathematics and Computing.
• 3: Martin I. Reiman , Alan Weiss, Sensitivity analysis via likelihood ratios, Proceedings of the 18th conference on Winter simulation, p.285-289, December 08-10, 1986, Washington, D.C., United States [doi>10.1145/318242.318450]
• 4: Rubins~ein, R. Y. (1981). Sensitivity analysis and performance extrapolation for computer simulation models, Technical Report, tlaxvard University, Cambridge, MA.

INDEX TERMS

Primary Classification:
  G. Mathematics of Computing

Additional Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING
      I.6.8 Types of Simulation
          Subjects: Monte Carlo

General Terms:
Algorithms, Design, Measurement, Performance, Theory