Selection of input models using bootstrap goodness-of-fit

ABSTRACT

Bootstrap methods are a natural adjunct of computer simulation experiments; both use resampling techniques to construct the statistical distributions of quantities of interest. In this paper we consider how bootstrap methods can be used in selecting appropriate input models for use in a computer simulation experiment. The proposed method uses a goodness of-fit statistic to decide on which of several competing input models should be used. We use bootstrapping to find the distribution of the test statistic under different assumptions as to which model is the correct fit. This allows the quality of fit of the different models to be compared. The bootstrapping process can be extended to the simulation experiment itself, allowing the effect of variability of estimated parameters on the simulation output to be assessed. The methodology is described and illustrated by application to a queueing example investigating the delays experienced by motorists caused by toll booths at a bridge river crossing.

REFERENCES

• 1: Banks, J., Carson ii, J.S. and Nelson, B.L. (1984). Discrete-Event Simulation, (2nd Edn). Upper Saddle River, NJ: Prentice Hall.
• 2: Russell R. Barton , Lee W. Schruben, Uniform and bootstrap resampling of empirical distributions, Proceedings of the 25th conference on Winter simulation, p.503-508, December 12-15, 1993, Los Angeles, California, United States [doi>10.1145/256563.256699]
• 3: Cheng, R.C.H. and Holland, W. (1995). The Effect of Input Parameters on the Variability of Simulation Output. Proceedings of the Second United Kingdom Simulation Society Conference (Eds R.C.H. Cheng and R.J. Pooley), North Berwick, April 1995, Edinburgh University, pp 29-36.
• 4: Cheng, R.C.H. and Holland, W. (1996). Sensitivity of Computer Simulation Experiments to Errors in Input Data. To appear in J. of Statis~ical Computation and Simulation.
• 5: Cheng, R.C.H. and Iles, T.C. (1990). }Embedded Models in Three-Parameter Distributions and their Estimation. J. R. Statist. Soc. B, 52, 135-149.
• 6: Ralph B D'Agostino , Michael A Stephens, Goodness-of-fit techniques, Marcel Dekker, Inc., New York, NY, 1986
• 7: Efron B. (1979). Bootstrap Methods : Another Look at the Jackknife. The Annals of Statistics, 7, pp 1-26.
• 8: Efron, B. (1982). The Jackknife, the Bootstrap and Other Resampling Plans. Vol. 38 of CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM.
• 9: Efron, B. and Tibshirani, R.J. (1993). An Introduction to the Bootstrap. New York and London: Chapman and Hall.
• 10: Griffiths, J.D. and Williams J.E. (1984). Traffic Studies on the Severn Bridge. Trai~c Engineering and Control, 25, pp 268-71,274
• 11: Helton, J.C. (1993). Uncertainty and Sensitivity Analysis Techniques for Use in Performance Assessment for Radioactive Waste Disposal. Reliability Engineering and System Safety, 42,327-367.
• 12: Helton, J.C. (1994). Treatment of Uncertainty in Performance Assessments for Complex Systems. Risk Analysis, 14, 483-511.
• 13: M. G. Kendall , A. Stuart , J. K. Ord, Kendall's advanced theory of statistics, Oxford University Press, Inc., New York, NY, 1987
• 14: Kleijnen J.P.C. (1995). Sensitivity Analysis and Related Analyses : a Survey of Statistical Techniques (submitted for publication).
• 15: Averill M. Law , W. David Kelton, Simulation Modeling and Analysis, McGraw-Hill Higher Education, 1997
• 16: Shanker, A. and Kelton, W. D. (1994). Measuring Output Error due to Input Error in Simulation: Analysis of Fitted vs. Mixed Empirical Distributions for Queues. To appear.
• 17: Swain, j.J., Venkatraman, S. and Wilson, J.R. (1988). Distribution Selection and Validation. J. of Statist. Comput. and Simul., 29, 271-297.