On batch means in the simulation and statistics communities

ABSTRACT

Batching is a well known technique for estimating the variance of point estimators computed from simulation experiments. The batch statistic variance estimator is simply the (appropriately scaled) sample variance of the estimator computed on subsets of data. The simulation and statistics communities seem to be largely unaware of each other's results in this area. Some empirical and theoretical results from the simulation and statistics literature will be discussed and compared. In particular, we discuss the important issue of selecting batch size and present a new data based method for determining it. The basic idea is to empirically estimate the optimal batch size for a smaller simulation length, and then extrapolate using knowledge of the optimal order of magnitude of batch length for the original simulation length. We provide a small simulation showing the effectiveness of the proposed method.

REFERENCES

• 1: Paul Bratley , Bennett L. Fox , Linus E. Schrage, A guide to simulation (2nd ed.), Springer-Verlag New York, Inc., New York, NY, 1987
• 2: Carlstein, E. 1986a. The use of subseries values for estimating the variance of a general statistic from a stationary sequence. Annals of Statistics 14:1171- 1179.
• 3: Carlstein, E. 1986b. Asymptotic normality for a general statistic from a stationary sequence. Annals of Probability 14:1371-1379.
• 4: Cox, D.R. and Lewis, P. 1961. The statistical analysis of series of events. London: Methuen.
• 5: Damerji, H. 1994. Mean-square consistency of the variance estimator in steady-state simulation output analysis. Operations Research 43:282-291.
• 6: George S. Fishman, Principles of Discrete Event Simulation, John Wiley & Sons, Inc., New York, NY, 1978
• 7: P. W. Glynn , D. L. Iglehart, Simulation output analysis using standardized time series, Mathematics of Operations Research, v.15 n.1, p.1-16, Feb. 1990 [doi>10.1287/moor.15.1.1]
• 8: Goldsman, D. and Melamed, B. 1992. A large-sample result for the method of batch means. Technical Report. School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia.
• 9: Goldsman, D. and Schruben, L. 1984. Asymptotic properties of some confidence interval estimators for simulation output. Management Science 30:1217-25.
• 10: Hall, P. and Jing, B.Y. 1994. On sample re-use methods for dependent data. Research Report ~SR8- 94. Centre for Math. and its Appl., Australian National Univ., Canberra.
• 11: Kiinsch, H. 1989. The jackknife and bootstrap for general stationary observations. Annals of Statistics 17:1217-1241.
• 12: Marc S. Meketon , Bruce Schmeiser, Overlapping batch means: something for nothing?, Proceedings of the 16th conference on Winter simulation, p.226-230, December 28-30, 1984, Dallas, TX
• 13: Antonio C. Pedrosa , Bruce W. Schmeiser, Asymptotic and finite-sample correlations between OBM estimators, Proceedings of the 25th conference on Winter simulation, p.481-488, December 12-15, 1993, Los Angeles, California, United States [doi>10.1145/256563.256692]
• 14: Politis, D.N. and Romano, J.P. 1993. On the sample variance of linear statistics derived from mixing sequences. Stochastic Processes and Appl. 45:155- 167.
• 15: Possolo, A. 1991. Subsampling a random field. In Spatial Statistics and Imaging, Ed. by A. Possolo, IMS Lecture Notes. 20: 286-294.
• 16: Robert G. Sargent , Keebom Kang , David Goldsman, An investigation of finite-sample behavior of confidence interval estimators, Operations Research, v.40 n.5, p.898-913, Sept.–Oct. 1992 [doi>10.1287/opre.40.5.898]
• 17: Schmeiser, B. 1982. Batch size effects in the analysis of simulation output. Operations Research. 30:556-568.
• 18: Bruce W. Schmeiser , Thanos N. Avramidis , Sherif Hashem, Overlapping batch statistics, Proceedings of the 22nd conference on Winter simulation, p.395-398, December 09-12, 1990, New Orleans, Louisiana, United States
• 19: Schruben, L. 1983. Confidence interval estimation using standardized time series. Operations Research 31:1090-1108.
• 20: Sherman, M. 1994. Asymptotic normality for a general statistic computed from a random field. Mathematical Methods of Statistics 3:326-345.
• 21: Sherman, M. 1995. Variance estimation for statistics computed from spatial lattice data. To appear in Journal of the Royal Star. Soc. B.
• 22: Song, W.M.T. and Schmeiser, B. 1988. Estimating standard errors' empirical behavior of asymptotic MSE-optimal batch sizes. In Comp. Science and Star.: Proc. of the 20th Symposimum on the In terrace, 575-580.

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.7 Organization and Design
          Subjects: Batch processing systems**

Additional Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING

General Terms:
Experimentation, Theory, Verification