Statistical selection of the best system

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

Statistical selection of the best system

Full text

Pdf (127 KB)

Source

Winter Simulation Conference archive
Proceedings of the 33nd conference on Winter simulation table of contents

Arlington, Virginia

TUTORIAL SESSION: Advanced tutorials table of contents

Pages: 139 - 146

Year of Publication: 2001

ISBN:0-7803-7309-X

Authors

David Goldsman	Georgia Institute of Technology, Atlanta, GA
Barry L. Nelson	Northwestern University, Evanston, IL

Sponsors

INFORMS/CS : Institute for Operations Research and the Management Sciences/College on Simulation
IEEE/SMCS : Institute of Electrical and Electronics Engineers/Systems, Man, and Cybernetics Society
NIST : National Institute of Standards and Technology
ACM: Association for Computing Machinery
SCS : The Society for Computer Simulation International
SIGSIM: ACM Special Interest Group on Simulation and Modeling
IIE : Institute of Industrial Engineers
IEEE/CS : Institute of Electrical and Electronics Engineers/Computer Society
ASA : American Statistical Association

Publisher

IEEE Computer Society Washington, DC, USA

Bibliometrics

Downloads (6 Weeks): 1, Downloads (12 Months): 6, Citation Count: 9

Additional Information:

abstract references cited by index terms collaborative colleagues

Tools and Actions:	Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

ABSTRACT

This tutorial discusses some statistical procedures for selecting the best of a number of competing systems. The term "best" may refer to that simulated system having, say, the largest expected value or the greatest likelihood of yielding a large observation. We describe six procedures for finding the best, three of which assume that the underlying observations arise from competing normal distributions, and three of which are essentially nonparametric in nature. In each case, we comment on how to apply the above procedures for use in simulations.

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	Bechhofer, R. E., S. Elmaghraby, and N. Morse. 1959. A single-sample multiple decision procedure for selecting the multinomial event which has the highest probability. Ann. Math. Stat. 30:102-119.
	2	Bechhofer, R. E., and D. M. Goldsman. 1986. Truncation of the Bechhofer-Kiefer-Sobel sequential procedure for selecting the multinomial event which has the largest probability (II): Extended tables and an improved procedure. Comm. Stat.-Simul. and Comp. B15:829-851.
	3	Bechhofer, R. E., T. J. Santner, and D. Goldsman. 1995. Design and Analysis of Experiments for Statistical Selection, Screening and Multiple Comparisons. New York: John Wiley and Sons.
	4	Boesel, J., and B. L. Nelson. 1998. Accounting for randomness in heuristic simulation optimization. IIE Transactions, forthcoming.
	5	Hsiao-Chang Chen , Liyi Dai , Chun-Hung Chen , Enver Yücesan, New development of optimal computing budget allocation for discrete event simulation, Proceedings of the 29th conference on Winter simulation, p.334-341, December 07-10, 1997, Atlanta, Georgia, United States [doi>10.1145/268437.268501]
	6	Stephen E. Chick, Selecting the best system: a decision-theoretic approach, Proceedings of the 29th conference on Winter simulation, p.326-333, December 07-10, 1997, Atlanta, Georgia, United States [doi>10.1145/268437.268499]
	7	Damerdji, H., P. W. Glynn, M. K. Nakayama, and J. R. Wilson. 1997a. Selection of the best system in steady-state simulations. Technical Report 97-5, Dept. of Industrial Engineering, North Carolina State Univ., Raleigh, North Carolina.
	8	Damerdji, H., P. W. Glynn, M. K. Nakayama, and J. R. Wilson. 1997b. Selection of the best system in transient simulations. Technical Report 97-6, Dept. of Industrial Engineering, North Carolina State Univ., Raleigh, North Carolina.
	9	Damerdji, H. and M. K. Nakayama. 1996. Two-stage multiple-comparison procedures for steady-state simulations. Technical Report, Dept. of Industrial Engineering, North Carolina State Univ., Raleigh, North Carolina.
	10	Dunnett, C. W. 1989. Multivariate normal probability integrals with product correlation structure. Applied Statistics 38:564-579. Correction: 42:709.
	11	Jean Dickinson Gibbons , Ingram Olkin , Milton Sobel, Selecting and ordering populations: a new statistical methodology, Society for Industrial and Applied Mathematics, Philadelphia, PA, 1999
	12	Goldsman, D., S.-H. Kim, W. S. Marshall, and B. L. Nelson. 2001. Ranking and selection for steady-state simulation: Procedures and perspectives. Working Paper, Department of Industrial Engineering & Management Sciences, Northwestern University, Evanston, Illinois.
	13	David Goldsman , Barry L. Nelson, Ranking, selection and multiple comparisons in computer simulations, Proceedings of the 26th conference on Winter simulation, p.192-199, December 11-14, 1994, Orlando, Florida, United States
	14	David Goldsman , Barry L. Nelson, Statistical screening, selection, and multiple comparison procedures in computer simulation, Proceedings of the 30th conference on Winter simulation, p.159-166, December 13-16, 1998, Washington, D.C., United States
	15	Goldsman, D., and B. L. Nelson. 1998b. Comparing systems via simulation. Handbook of Simulation, ed. J. Banks, Chapter 8. New York: John Wiley and Sons.
	16	Gupta, S. S. 1956. On a Decision Rule for a Problem in Ranking Means. Ph.D. Dissertation (Mimeo. Ser. No. 150). Inst. of Statist., Univ. of North Carolina, Chapel Hill, North Carolina.
	17	Gupta, S. S. 1965. On some multiple decision (selection and ranking) rules. Technometrics 7:225-245.
	18	Y. Hochberg , A. C. Tamhane, Multiple comparison procedures, John Wiley & Sons, Inc., New York, NY, 1987
	19	Hsu, J. C. 1984. Constrained simultaneous confidence intervals for multiple comparisons with the best. Annals of Statistics 12:1136-1144.
	20	Seong-Hee Kim , Barry L. Nelson, A fully sequential procedure for indifference-zone selection in simulation, ACM Transactions on Modeling and Computer Simulation (TOMACS), v.11 n.3, p.251-273, July 2001 [doi>10.1145/502109.502111]
	21	Kim, S-H. and B. L. Nelson. 2001b. On the asymptotic validity of fully sequential selection procedures for steady-state simulation. Working Paper, Department of Industrial Engineering & Management Sciences, Northwestern University, Evanston, Illinois.
	22	Averill M. Law , David M. Kelton, Simulation Modeling and Analysis, McGraw-Hill Higher Education, 1999
	23	Matejcik, F. J., and B. L. Nelson. 1995. Two-stage multiple comparisons with the best for computer simulation. Operations Research 43:633-640.
	24	Miller, J. O., B. L. Nelson, and C. H. Reilly. Efficient multinomial selection in simulation. 1998. Naval Research Logistics 45:459-482.
	25	Nakayama, M. K. 1997. Multiple-comparison procedures for steady-state simulations. Annals of Statistics 25:2433-2450.
	26	Barry L. Nelson , Frank J. Matejcik, Using common random numbers for indifference-zone selection and multiple comparisons in simulation, Management Science, v.41 n.12, p.1935-1945, Dec. 1995 [doi>10.1287/mnsc.41.12.1935]
	27	Barry L. Nelson , Julie Swann , David Goldsman , Wheyming Song, Simple Procedures for Selecting the Best Simulated System When the Number of Alternatives is Large, Operations Research, v.49 n.6, p.950-963, November 2001 [doi>10.1287/opre.49.6.950.10019]
	28	Rinott, Y. 1978. On two-stage selection procedures and related probability-inequalities. Comm. Stat.-Thy. and Meth. A7:799-811.
	29	Wilcox, R. R. 1984. A table for Rinott's selection procedure. J. Quality Technology 16:97-100.

CITED BY 9

	Marvin K. Nakayama, Output analysis: analysis of simulation output, Proceedings of the 35th conference on Winter simulation: driving innovation, December 07-10, 2003, New Orleans, Louisiana
	Mahmoud H. Alrefaei , Ameen J. Alawneh, Selecting the best stochastic system for large scale problems in DEDS, Mathematics and Computers in Simulation, v.64 n.2, p.237-245, 27 January
	Marvin K. Nakayama, Output analysis: simulation output analysis, Proceedings of the 34th conference on Winter simulation: exploring new frontiers, December 08-11, 2002, San Diego, California
	E. Jack Chen, Recent advances in simulation optimization: a conservative adjustment to the ETSS procedure, Proceedings of the 34th conference on Winter simulation: exploring new frontiers, December 08-11, 2002, San Diego, California
	David Goldsman , Seong-Hee Kim , Barry L. Nelson, Statistical selection of the best system, Proceedings of the 37th conference on Winter simulation, December 04-07, 2005, Orlando, Florida
	Matthew H. Jones , K. Preston White, Jr., Stochastic approximation with simulated annealing as an approach to global discrete-event simulation optimization, Proceedings of the 36th conference on Winter simulation, December 05-08, 2004, Washington, D.C.
	Peter Glynn , Sandeep Juneja, A large deviations perspective on ordinal optimization, Proceedings of the 36th conference on Winter simulation, December 05-08, 2004, Washington, D.C.
	Mark Broadie , Minsup Han , Assaf Zeevi, Implications of heavy tails on simulation-based ordinal optimization, Proceedings of the 39th conference on Winter simulation: 40 years! The best is yet to come, December 09-12, 2007, Washington D.C.
	Jose Blanchet , Jingchen Liu , Bert Zwart, Large deviations perspective on ordinal optimization of heavy-tailed systems, Proceedings of the 40th Conference on Winter Simulation, December 07-10, 2008, Miami, Florida