Simulation in exponential families

ABSTRACT

An acceptance-rejection algorithm for the simulation of random variables in statistical exponential families is described. This algorithm does not require any prior knowledge of the family, except sufficient stati stics and the value of the parameter. It allows simulation from many members of the exponential family. We present some bounds on computing time, as well as the main properties of the empirical measures of samples simulated by our methods (functional Glivenko-Cantelli and central limit theorems). This algorithm is applied in order to evaluate the distribution of M-estimators under composite alternatives; we also propose its use in Bayesian statistics in order to simulate from posterior distributions.

REFERENCES

• 1: BARNDORFF-NIELSEN, O. 1978. Information and Exponential Families in Statistical Theory. John Wiley and Sons, Inc., New York, NY.
• 2: BERGER, J. O. AND PERICCHI, L. R. 1996. The intrisic Bayes factor for model selection and prediction. J. Am. Stat. Assoc. 91,433, 109-122.
• 3: BINGHAM, N., GOLDIE, C., AND TEUGELS, J. 1987. Regular Variation. Cambridge University Press, New York, NY.
• 4: BIRMAN, L. AND SOLOMJAK, S. 1967. Piecewise polynomial approximation of functions of the class Wp. Math Sbornik 73, 295-317.
• 5: L. D. Brown, Fundamentals of statistical exponential families: with applications in statistical decision theory, Institute of Mathematical Statistics, Hayworth, CA, 1986
• 6: BRONIATOWSKI, M. AND FUCHS, C. 1995. Tauberian theorems, Chernoff Inequality and the tail behaviour of finite convolutions of distribution functions. Adv. Math. 116, 1, 12-33.
• 7: CHOW, Y. S. AND TEICHER, H. 1978. Probability Theory, Independence Interchangeability Martingales. Springer-Verlag, New York, NY.
• 8: CRESSIE, N. 1991. Statistics for Spatial Data. John Wiley and Sons, Inc., New York, NY.
• 9: DELLAPORTAS, P. AND SMITH, A. F. M. 1993. Bayesian inference for generalized linear and proportional hazards models via Gibbs sampling. J. Roy. Stat. Soc., C 3, 443-459.
• 10: DEMBO, A. AND ZEITOUNI, O. 1993. Large Deviation Techniques and Applications. Jones and Bartlett Publ., Inc., Sudbury, MA.
• 11: DEVROYE, L. 1987. Non-Uniform Random Variate Generation. Springer-Verlag, New York, NY.
• 12: GELFAND, A. E. AND SMITH, A. F. M. 1990. Sampling based approaches to calculating marginal densities. J. Am. Stat. Assoc. 85, 389-409.
• 13: GILKS, W. R. AND WILD, P. 1992. Adaptive rejection sampling for Gibbs sampling. Appl. Stat. 41, 2, 337-348.
• 14: David Haussler, Sphere packing numbers for subsets of the Boolean n-cube with bounded Vapnik-Chervonenkis dimension, Journal of Combinatorial Theory Series A, v.69 n.2, p.217-232, Feb. 1995 [doi>10.1016/0097-3165(95)90052-7]
• 15: ITLIS, M. 1995. Sharp asymptotics of large deviations in Rd. J. Theor. Probab. 8, 3, 501-522.
• 16: KASS, R. E. AND RAFERTY, A. E. 1995. Bayes factors. J. Am. Stat. Assoc. 90, 430, 773-795.
• 17: KOLMOGOROV, A. N. AND TIKHOMINOV, V. M. 1961. c -entropy and c -capacity of sets in function spaces. AMS Translations 2, 17, 277-364.
• 18: LORENTZ, G. G. 1986. Approximation of Functions. Chelsea Publishing Co., Inc., New York, NY.
• 19: POLLARD, D. 1984. Convergence of Stochastic Processes. Springer-Verlag, New York, NY.
• 20: POLLARD, D. 1987. Empirical Processes:Theory and Applications. American Statistical Association, Washington, DC.
• 21: ROBERT, C. P. 1992. L'Analyse Statistique Bayesienne. Economica.
• 22: ROBERT, C. P. AND TITTERINGTON, M. 1996. Resampling schemes for hidden Markov chains and their application for maximum likelihood estimation. Tech. Rep. Dept. of Statistics. University of Glasgow, Glasgow, Scotland, UK.
• 23: ROUSSAS, G. G. 1972. Contiguity of Probability Measures: Some Applications in Statistics. Cambridge University Press, New York, NY.
• 24: SERFLING, R.J. 1980. Approximation Theorems of Mathematical Statistics. John Wiley and Sons, Inc., New York, NY.
• 25: J. S. Sadowsky , J. A. Bucklew, Large deviations theory techniques in Monte Carlo simulation, Proceedings of the 21st conference on Winter simulation, p.505-513, December 04-06, 1989, Washington, D.C., United States [doi>10.1145/76738.76804]
• 26: TALAGRAND, M. 1995. Concentration of measure and isoperimetric inequalities in product spaces. Pub. Math. IHES 81, 73-205.

INDEX TERMS

Primary Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Random number generation

General Terms:
Algorithms, Theory

Keywords:
Bayesian statistics, M-estimators, computing time, empirical measure, exponential families, posterior distributions, simulation