A Bayesian approach to analysis of limit standards

ABSTRACT

Limit standards are probabilistic requirements or benchmarks regarding the proportion of replications conforming or not conforming to a desired threshold. Sample proportions resulting from the analysis of replications are known to be beta distributed. As a result, standard constructs for defining a confidence interval on such a proportion, based on critical points from the normal or Student's t distribution, are increasingly inaccurate as the mean sample proportion approaches the limits of 0 or 1. We consider the Bayesian relationship between the beta and binomial distributions as the foundation for a sequential methodology in the analysis of limit standards. The benefits of using the beta distribution methodology are variance reduction, and smaller sample size (when compared to other analysis methodologies).

REFERENCES

• 1: E. Jack Chen , W. David Kelton, Simulation-based estimation of quantiles, Proceedings of the 31st conference on Winter simulation: Simulation---a bridge to the future, p.428-434, December 05-08, 1999, Phoenix, Arizona, United States [doi>10.1145/324138.324272]
• 2: Chen, E. and W. Kelton (2004). "Quantile and tolerance-interval estimation in simulation," accepted for European Journal of Operational Research.
• 3: E. Jack Chen , W. David Kelton, Experimental performance evaluation of histogram approximation for simulation output analysis, Proceedings of the 36th conference on Winter simulation, December 05-08, 2004, Washington, D.C.
• 4: Stephen E. Chick, Selecting the best system: a decision-theoretic approach, Proceedings of the 29th conference on Winter simulation, p.326-333, December 07-10, 1997, Atlanta, Georgia, United States [doi>10.1145/268437.268499]
• 5: Stephen E. Chick, Input Distribution Selection for Simulation Experiments: Accounting for Input Uncertainty, Operations Research, v.49 n.5, p.744-758, September 2001 [doi>10.1287/opre.49.5.744.10606]
• 6: Stephen E. Chick, Bayesian ideas and discrete event simulation: why, what and how, Proceedings of the 38th conference on Winter simulation, December 03-06, 2006, Monterey, California
• 7: Stephen E. Chick , Koichiro Inoue, Sequential allocations that reduce risk for multiple comparisons, Proceedings of the 30th conference on Winter simulation, p.669-676, December 13-16, 1998, Washington, D.C., United States
• 8: Cochran, W. (1963). Sampling Techniques, 2^nd ed. New York: Wiley.
• 9: Roy R. Creasey, Jr. , K. Preston White, Jr. , Melanie B. Marks , Bennie D. Waller, Determination of the "best" system that meets a limit standard, Proceedings of the 37th conference on Winter simulation, December 04-07, 2005, Orlando, Florida
• 10: Roy R. Creasey, Jr. , K. Preston White, Jr., Comparison of limit standards using a sequential probability ratio test, Proceedings of the 38th conference on Winter simulation, December 03-06, 2006, Monterey, California
• 11: Umesh Gavasakar, A comparison of two elicitation methods for a prior distribution for a binomial parameter, Management Science, v.34 n.6, p.784-790, June 1988 [doi>10.1287/mnsc.34.6.784]
• 12: Shane G. Henderson, Input modeling: input model uncertainty: why do we care and what should we do about it?, Proceedings of the 35th conference on Winter simulation: driving innovation, December 07-10, 2003, New Orleans, Louisiana
• 13: Johnson, N., S. Kotz, and N. Balakrishnan (1994). Continuous Univariate Distributions, 2^nd ed., New York: John Wiley & Sons, Inc.
• 14: Johnson, N., A. Kemp, S. Kotz (2005). Univariate Discrete Distributions, 3^rd. ed. Hoboken, NJ: John Wiley & Sons, Inc.
• 15: Seong-Hee Kim , Barry L. Nelson, On the Asymptotic Validity of Fully Sequential Selection Procedures for Steady-State Simulation, Operations Research, v.54 n.3, p.475-488, May-June 2006 [doi>10.1287/opre.1060.0281]
• 16: Averill M. Law , W. David Kelton, Simulation Modeling and Analysis, McGraw-Hill Higher Education, 1997
• 17: Lee, P. (1997). Bayesian Statistics: An Introduction, 2^nd ed., New York: John Wiley & Sons, Inc.
• 18: Kimmo E. E. Raatikainen, Simulation-based estimation of proportions, Management Science, v.41 n.7, p.1202-1223, July 1995 [doi>10.1287/mnsc.41.7.1202]
• 19: White, K., K. Johnson, T. Adams, C. Everline, and R. Creasey (2007). "Best practices and recommended statistical techniques for developing Monte Carlo sampling plans." NASA Technical Paper.