|
 ABSTRACT
We present procedures for selecting the best or near-best of a finite number of simulated systems when best is defined by maximum or minimum expected performance. The procedures are appropriate when it is possible to repeatedly obtain small, incremental samples from each simulated system. The goal of such a sequential procedure is to eliminate, at an early stage of experimentation, those simulated systems that are apparently inferior, and thereby reduce the overall computational effort required to find the best. The procedures we present accommodate unequal variances across systems and the use of common random numbers. However, they are based on the assumption of normally distributed data, so we analyze the impact of batching (to achieve approximate normality or independence) on the performance of the procedures. Comparisons with some existing indifference-zone procedures are also provided.
REFERENCES
Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.
| |
1
|
BECHHOFER, R. E., DUNNETT,C.W.,GOLDSMAN,D.M.,AND HARTMANN, M. 1990. A comparison of the performances of procedures for selecting the normal population having the largest mean when the populations have a common unknown variance. Commun. Stat. B19, 971-1006.
|
| |
2
|
BECHHOFER, R. E., SANTNER,T.J.,AND GOLDSMAN, D. M. 1995. Design and Analysis for Statistical Selection, Screening and Multiple Comparisons. Wiley, New York.
|
| |
3
|
BOESEL, J., NELSON,B.L.,AND KIM, S. 2001. Using ranking and selection to clean up after a simulation search. Tech. Rep. Department of Industrial Engineering and Management Sciences, Northwestern Univ., Evanston, Ill.
|
| |
4
|
CHEN, C.-H. 1996. A lower bound for the correct-selection probability and its application to discrete event simulations. IEEE Trans. Autom. Contr. 41, 1227-1231.
|
| |
5
|
|
| |
6
|
|
 |
7
|
|
| |
8
|
FABIAN, V. 1974. Note on Anderson's sequential procedures with triangular boundary. Ann. Statis. 2, 170-176.
|
| |
9
|
|
| |
10
|
GOLDSMAN,D.M.,AND NELSON, B. L. 1998a. Comparing systems via simulation. In Handbook of Simulation, J. Banks, Ed., Chap. 8. Wiley, New York.
|
| |
11
|
David Goldsman , Barry L. Nelson, Statistical screening, selection, and multiple comparison procedures in computer simulation, Proceedings of the 30th conference on Winter simulation, p.159-166, December 13-16, 1998, Washington, D.C., United States
|
| |
12
|
HARTMANN, M. 1988. An improvement on Paulson's sequential ranking procedure. Sequen. Analysis 7, 363-372.
|
| |
13
|
HARTMANN, M. 1991. An improvement on Paulson's procedure for selecting the population with the largest mean from k normal populations with a common unknown variance. Sequent. Analysis 10, 1-16.
|
| |
14
|
|
| |
15
|
HSU, J. C. 1996. Multiple Comparisons: Theory and Methods. Chapman and Hall, New York.
|
| |
16
|
JENNISON, C., JOHNSTONE,I.M.,AND TURNBULL, B. W. 1982. Asymptotically optimal procedures for sequential adaptive selection of the best of several normal means. In Statistical Decision Theory and Related Topics III, Vol 2. Academic Press, New York.
|
| |
17
|
LEWIS, P. A. W. 1980. Simple models for positive-valued and discrete-valued time series with ARMA correlation structure. In Multivariate Analysis V, P. R. Krishnaiah, Ed. North-Holland, New York, pp. 151-156.
|
| |
18
|
J. O. Miller , Barry L. Nelson , Charles H. Reilly, Getting more from the data in a multinomial selection problem, Proceedings of the 28th conference on Winter simulation, p.287-294, December 08-11, 1996, Coronado, California, United States
[doi> 10.1145/256562.256631]
|
| |
19
|
MILLER,J.O.,NELSON,B.L.,AND REILLY, C. H. 1998a. Efficient multinomial selection in simulation. Naval Res. Log. 45, 459-482.
|
| |
20
|
MILLER, J. O., NELSON,B.L.,AND REILLY, C. H. 1998b. Comparing simulated systems based on the probability of being the best. Tech. Rep., Dept. Industrial Engineering and Management Sciences, Northwestern Univ., Evanston, Ill.
|
| |
21
|
|
| |
22
|
NELSON, B. L., AND BANERJEE, S. 2001. Selecting a good system: Procedures and inference. IIE Trans. 33, 149-166.
|
| |
23
|
|
| |
24
|
|
| |
25
|
PAULSON, E. 1964. A sequential procedure for selecting the population with the largest mean from k normal populations. Ann. Math. Stat. 35, 174--180.
|
| |
26
|
RINOTT, Y. 1978. On two-stage selection procedures and related probability-inequalities. Commun. Stat. A7, 799-811.
|
| |
27
|
ROBBINS, H. 1970. Statistical methods related to the law of the iterated logarithm. Ann. Math. Stat. 41, 1397-1409.
|
| |
28
|
SWANEPOEL,J.W.H.,AND GEERTSEMA, J. C. 1976. Sequential procedures with elimination for selecting the best of k normal populations. S. Afr. Statist. J. 10, 9-36.
|
| |
29
|
TAMHANE, A. C. 1977. Multiple comparisons in model I: One-way anova with unequal variances. Commun. Stat. A6, 15-32.
|
CITED BY 40
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Mary Court , Jennifer Pittman , Christos Alexopoulos , David Goldsman , Seong-Hee Kim , Margaret Loper , Amy Pritchett , Jorge Haddock, A framework for simulating human cognitive behavior and movement when predicting impacts of catastrophic events, Proceedings of the 36th conference on Winter simulation, December 05-08, 2004, Washington, D.C.
|
|
|
Sigrün Andradóttir , David Goldsman , Lee W. Schruben , Bruce W. Schmeiser , Enver Yücesan, Analysis methodology: are we done?, Proceedings of the 37th conference on Winter simulation, December 04-07, 2005, Orlando, Florida
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Adobe Acrobat
QuickTime
Windows Media Player
Real Player
Pdf
G.3