The use of variance reduction techniques in the estimation of simulation metamodels

ABSTRACT

Variance reduction techniques can be useful strategies for improving the estimates of simulation metamodel coefficients. Depending upon the goals of the experimenter, the type of metamodel being estimated, and the characteristics of the system being simulated, an appropriate variance reduction technique can be applied. This paper provides a review of recent research that investigates the application of variance reduction techniques in the simulation metamodeling context. One strategy, Schruben and Margolin's (1978) assignment rule, which utilizes a combination of antithetic and common random number streams, is found to be a particularly useful variance reduction technique for the estimation of simulation metamodels.

REFERENCES

• 1: Russell R. Barton, Metamodeling: a state of the art review, Proceedings of the 26th conference on Winter simulation, p.237-244, December 11-14, 1994, Orlando, Florida, United States
• 2: Bauer, K. W., and J. R. Wilson. 1992. Control variate selection criteria. Naval Research Logistics 39:307-321.
• 3: George E P Box , Norman R Draper, Empirical model-building and response surface, John Wiley & Sons, Inc., New York, NY, 1986
• 4: Joan M. Donohue , Ernest C. Houck , Raymond H. Myers, Simulation designs for quadratic response surface models in the presence of model misspecification, Management Science, v.38 n.12, p.1765-1791, Dec. 1992 [doi>10.1287/mnsc.38.12.1765]
• 5: Joan M. Donohue , Ernest C. Houck , Raymond H. Myers, Simulation designs and correlation induction for reducing second-order bias in first-order response surfaces, Operations Research, v.41 n.5, p.880-902, Sept.–Oct 1993 [doi>10.1287/opre.41.5.880]
• 6: Joan M. Donohue , Ernest C. Houck , Raymond H. Myers, A sequential experimental design procedure for the estimation of first- and second-order simulation metamodels, ACM Transactions on Modeling and Computer Simulation (TOMACS), v.3 n.3, p.190-224, July 1993 [doi>10.1145/174153.174156]
• 7: Joan M. Donohue , Ernest C. Houck , Raymond H. Myers, Simulation designs for the estimation of quadratic response surface gradients in the presence of model misspecification, Management Science, v.41 n.2, p.244-262, Feb. 1995 [doi>10.1287/mnsc.41.2.244]
• 8: Fishman, G. S. 1974. Correlated simulation experiments. Simulation 23:177-180.
• 9: Friedman, L. W., and I. Pressman. 1988. The metamodel in simulation analysis: Can it be trusted? European Journal of Operational Research 39:939- 948.
• 10: Galbraith, L., and C. R. Standridge. 1994. Analysis in manufacturing systems simulation: A case study. Simulation 63:369-376.
• 11: Gordon, S. C., J. A. Ausink, and R. J. Berdine. 1994. Using experimental design techniques for spacecraft control simulation. Simulation 62:303-309.
• 12: Hesterberg, T. 1995. Weighted average importance sampling and defensive mixture distributions. Technometrics 37:185-194.
• 13: Hussey, J. R., R. H. Myers, and E. C. Houck. 1987a. Pseudorandom number assignments in quadratic response surface designs, lie Transactions 19:395- 403.
• 14: James R. Hussey , Raymond H. Myers , Ernest C. Houck, Correlated simulation experiments in first-order response surface design, Operations Research, v.35 n.5, p.744-758, September-October 1987 [doi>10.1287/opre.35.5.744]
• 15: Joshi, S., and J. D. Tew. 1995. Validation and statistical analysis procedures Under the common random number correlation-induction strategy for multipopulation simulation experiments. To appear in European Journal of Operational Research.
• 16: Kleijnen, J. P. C. 1977. Design and analysis of simulations: Practical statistical techniques. Szmulation 29:81-90.
• 17: Jack P. C. Kleijnen, Regression metamodels for simulation with common random numbers: comparison of validation tests and confidence intervals, Management Science, v.38 n.8, p.1164-1185, Aug. 1992 [doi>10.1287/mnsc.38.8.1164]
• 18: Kuei, C. H., and C. N. Madu. 1994. Polynomial metamodelling and Taguchi designs in simulation with application to the maintenance float policy. European Journal of Operatzonal Research 72:364- 375.
• 19: Chimyung Kwon , Jeffrey D. Tew, Strategies for combining antithetic variates and control variates in designed simulation experiments, Management Science, v.40 n.8, p.1021-1034, Aug. 1994 [doi>10.1287/mnsc.40.8.1021]
• 20: Lavenberg, S. S., T. L. Moeller, and P. D. Welch. 1982. Statistical results on control variables with application to queueing network simulation. Operatzons Research 30:182-202.
• 21: Madu, C. N., and C. Kuei. 1992. Simulation metamodels of system availability and optimum spare and repair units, lIE Transactions 24:99-104.
• 22: Barry L. Nelson, Designing efficient simulation experiments, Proceedings of the 24th conference on Winter simulation, p.126-132, December 13-16, 1992, Arlington, Virginia, United States [doi>10.1145/167293.167317]
• 23: A. Nozari , S. F. Arnold , C. D. Pegden, Statistical analysis for use with the Schruben and Margolin correlation induction strategy, Operations Research, v.35 n.1, p.127-139, Jan.-Feb. 1987 [doi>10.1051/ro:2001107]
• 24: Schruben, L. W., and B. H. Margolin. 1978. Pseudorandom number assignment in statistically designed simulation and distribution sampling experiments. Journal of the American Statistical Association 73:504-520.
• 25: Lee W. Schruben , Susan M. Sanchez , Paul J. Sanchez , Veronica A. Czitrom, Variance reallocation in Taguchi's robust design framework, Proceedings of the 24th conference on Winter simulation, p.548-556, December 13-16, 1992, Arlington, Virginia, United States [doi>10.1145/167293.167636]
• 26: Tew, J. D. 1991. Correlated replicates for first-order metamodel estimation in simulation experiments. Transactions of The Soczety for Computer Simulation 8:218-244.
• 27: Jeffrey D. Tew, Using central composite designs in simulation experiments, Proceedings of the 24th conference on Winter simulation, p.529-538, December 13-16, 1992, Arlington, Virginia, United States [doi>10.1145/167293.167634]
• 28: Jeffrey D. Tew , Marnita D. Crenshaw, Heuristic diagnostics for the presence of pure error in computer simulation models, Proceedings of the 22nd conference on Winter simulation, p.347-356, December 09-12, 1990, New Orleans, Louisiana, United States
• 29: Jeffrey D. Tew , James R. Wilson, Validation of simulation analysis methods for the Schruben-Margolin correlation-induction strategy, Operations Research, v.40 n.1, p.87-103, Jan.–Feb. 1992 [doi>10.1287/opre.40.1.87]
• 30: Tew, J. D., and J. R. Wilson. 1994. Estimating simulation metamodels using combined correlationbased variance reduction techniques. IIE Transactions 26:2-16.
• 31: Zeimer, M. A., and J. D. Tew. 1994. Selection of a pure error generator for simulation experiments. Transactions of The Society for Computer Simulation 11:132-158.

INDEX TERMS

Primary Classification:
  F. Theory of Computation
  F.1 COMPUTATION BY ABSTRACT DEVICES
      F.1.3 Complexity Measures and Classes
          Subjects: Reducibility and completeness

Additional Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING
      I.6.1 Simulation Theory
          Subjects: Systems theory

General Terms:
Experimentation, Measurement