Selecting the best system in steady-state simulations using batch means

ABSTRACT

Suppose that we want to compare k different systems, where /spl mu//sub i/ denotes the steady state mean performance of system i. Our goal is to use simulation to pick the "best" system (i.e., the one with the largest or smallest steady state mean). To do this, we present some two stage procedures based on the method of batch means. Our procedures also construct multiple comparisons with the best (MCB) confidence intervals for /spl mu//sub i/-max/sub j/spl ne/i//spl mu//sub j/, i=1,...,k. Under the assumption of an indifference zone of (absolute or relative) width /spl delta/, we can show that asymptotically (as /spl delta//spl rarr/0 with the size of the batches proportional to 1//spl delta//sup 2/), the joint probability of correctly selecting the best system and of the MCB confidence intervals simultaneously containing /spl mu//sub i/-max/sub j/spl ne/i//spl mu//sub j/, i=1,...,k, is at least 1-/spl alpha/, where /spl alpha/ is prespecified by the user.

REFERENCES

• 1: Bechhofer, R. E. 1954. A single-sample multiple decision procedure for ranking means of normal populations with known variances. Annals of Mathematical Statistics 25:16-39.
• 2: Billingsley, P. 1968. Convergence o} Probability Measures. New York: John Wiley.
• 3: Damerdji, H., P. W. Glynn, M. K. Nakayama, and J. R.. Wilson. 1995. Selecting the Best System in Steady-State Simulations. Forthcoming technical report.
• 4: Ethier, S. N. and T. C. Kurtz. 1986. Markov Processes: Characterization and Convergence. New York: John Wiley.
• 5: Glynn, P. W. 1990. Diffusion approximations. Chapter 4 of Handbooks in Operations Research and Management Science, Vol. 2, Stochastic Models, ed. D. Heyman and M. Sobel. Elsevier Science Publishers B. V. (North-Holland).
• 6: P. W. Glynn , W. Whitt, Sufficient conditions for functional-limit-theorem versions of L=&lgr;W, Queueing Systems: Theory and Applications, v.1 n.3, p.279-287, March 1987 [doi>10.1007/BF01149539]
• 7: David Goldsman , Barry L. Nelson, Ranking, selection and multiple comparisons in computer simulations, Proceedings of the 26th conference on Winter simulation, p.192-199, December 11-14, 1994, Orlando, Florida, United States
• 8: Hsu, J. C. 1984. Constrained simultaneous confidence intervals for multiple comparisons with the best. Annals of Statistics 12:1136-1144.
• 9: Matejcik, F. J. and B. L. Nelson. 1992. Two-stage multiple comparisons with the best for computer simulation. Operations Research, forthcoming.
• 10: Marvin K. Nakayama, Two-stage stopping procedures based on standardized time series, Management Science, v.40 n.9, p.1189-1206, Sept. 1994 [doi>10.1287/mnsc.40.9.1189]
• 11: Nakayama, M. K. 1995. Procedures for simultaneous selection and multiple comparisons in steady-state simulations. Unpublished manuscript.
• 12: Newman, C. M. and A. L. Wright. 1981. An invariance priciple for certain dependent sequences. Annals of Probability 9:671-675.
• 13: Rinott, Y. 1978. On two-stage selection procedures and related probability-inequalities. Communications in Statistics--Theory and Methods A7:799- 811.
• 14: W. Whitt, Planning queueing simulations, Management Science, v.35 n.11, p.1341-1366, Nov. 1989 [doi>10.1287/mnsc.35.11.1341]
• 15: W. Whitt, Simulation run length planning, Proceedings of the 21st conference on Winter simulation, p.106-112, December 04-06, 1989, Washington, D.C., United States [doi>10.1145/76738.76751]
• 16: Wilcox, R. R. 1984. A table for Rinott's selection procedure. Journal o} Quality Technology 16:97- 100.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING

Additional Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.7 Organization and Design
          Subjects: Batch processing systems**

  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Selection process

General Terms:
Performance, Theory