A tutorial on design and analysis of simulation experiments

ABSTRACT

This paper describes, in general terms, methods to help design the runs for simulation models and interpret their output. Statistical methods are described for several different purposes, and related problems like comparison, variance reduction, sensitivity estimation, metamodeling and optimization are mentioned. The main point is to call attention to the challenges and opportunities in using simulation models carefully and effectively.

REFERENCES

• 1: Christos Alexopoulos, Advanced simulation output analysis for a single system, Proceedings of the 25th conference on Winter simulation, p.89-96, December 12-15, 1993, Los Angeles, California, United States [doi>10.1145/256563.256593]
• 2: Christos Alexopoulos, A review of advanced methods for simulation output analysis, Proceedings of the 26th conference on Winter simulation, p.133-140, December 11-14, 1994, Orlando, Florida, United States
• 3: Farhad Azadivar, A tutorial on simulation optimization, Proceedings of the 24th conference on Winter simulation, p.198-204, December 13-16, 1992, Arlington, Virginia, United States [doi>10.1145/167293.167332]
• 4: Banks, J. and J.S. Carson, II. 1984. Discrete-event system simulation. Englewood Cliffs, New Jersey: Prentice- Hall.
• 5: Russell R. Barton, Metamodels for simulation input-output relations, Proceedings of the 24th conference on Winter simulation, p.289-299, December 13-16, 1992, Arlington, Virginia, United States [doi>10.1145/167293.167352]
• 6: Paul Bratley , Bennett L. Fox , Linus E. Schrage, A guide to simulation (2nd ed.), Springer-Verlag New York, Inc., New York, NY, 1987
• 7: John M. Charnes, Multivariate simulation output analysis, Proceedings of the 23rd conference on Winter simulation, p.187-193, December 08-11, 1991, Phoenix, Arizona, United States
• 8: John M. Charnes, Statistical analysis of output processes, Proceedings of the 25th conference on Winter simulation, p.41-49, December 12-15, 1993, Los Angeles, California, United States [doi>10.1145/256563.256579]
• 9: Lori S. Cook, Factor screening of multiple responses, Proceedings of the 24th conference on Winter simulation, p.174-180, December 13-16, 1992, Arlington, Virginia, United States [doi>10.1145/167293.167327]
• 10: George S. Fishman, Principles of Discrete Event Simulation, John Wiley & Sons, Inc., New York, NY, 1978
• 11: Michael C. Fu, A tutorial review of techniques for simulation optimization, Proceedings of the 26th conference on Winter simulation, p.149-156, December 11-14, 1994, Orlando, Florida, United States
• 12: Paul Glasserman , Peter W. Glynn, Gradient estimation for regenerative processes, Proceedings of the 24th conference on Winter simulation, p.280-288, December 13-16, 1992, Arlington, Virginia, United States [doi>10.1145/167293.167350]
• 13: David Goldsman, Simulation output analysis, Proceedings of the 24th conference on Winter simulation, p.97-103, December 13-16, 1992, Arlington, Virginia, United States [doi>10.1145/167293.167310]
• 14: David Goldsman , Barry L. Nelson, Ranking, selection and multiple comparisons in computer simulations, Proceedings of the 26th conference on Winter simulation, p.192-199, December 11-14, 1994, Orlando, Florida, United States
• 15: David Goldsman , Barry L. Nelson , Bruce Schmeiser, Methods for selecting the best system, Proceedings of the 23rd conference on Winter simulation, p.177-186, December 08-11, 1991, Phoenix, Arizona, United States
• 16: David Alan Grier, Graphical techniques for output analysis, Proceedings of the 24th conference on Winter simulation, p.314-319, December 13-16, 1992, Arlington, Virginia, United States [doi>10.1145/167293.167356]
• 17: Yu-Chi Ho, Perturbation analysis: concepts and algorithms, Proceedings of the 24th conference on Winter simulation, p.231-240, December 13-16, 1992, Arlington, Virginia, United States [doi>10.1145/167293.167340]
• 18: Sarah J. Hood , Peter D. Welch, Experimental design issues in simulation with examples from semiconductor manufacturing, Proceedings of the 24th conference on Winter simulation, p.255-263, December 13-16, 1992, Arlington, Virginia, United States [doi>10.1145/167293.167346]
• 19: Sarah J. Hood , Peter D. Welch, Response surface methodology and its application in simulation, Proceedings of the 25th conference on Winter simulation, p.115-122, December 12-15, 1993, Los Angeles, California, United States [doi>10.1145/256563.256598]
• 20: W. David Kelton, Analysis of output data, Proceedings of the 26th conference on Winter simulation, p.62-68, December 11-14, 1994, Orlando, Florida, United States
• 21: Khoshnevis, B. 1994. Discrete systems simulation. New York: McGraw-Hill.
• 22: Jack P C Kleijnen, Statistical tools for simulation practitioners, Marcel Dekker, Inc., New York, NY, 1986
• 23: Averill M. Law , W. David Kelton, Simulation Modeling and Analysis, McGraw-Hill Higher Education, 1997
• 24: Pierre L'Ecuyer, An overview of derivative estimation, Proceedings of the 23rd conference on Winter simulation, p.207-217, December 08-11, 1991, Phoenix, Arizona, United States
• 25: Pierre L'Ecuyer, Efficiency improvement and variance reduction, Proceedings of the 26th conference on Winter simulation, p.122-132, December 11-14, 1994, Orlando, Florida, United States
• 26: P. A. W. Lewis , E. J. Orav, Simulation methodology for statisticians, operations analysts, and engineers: vol. 1, Wadsworth Publ. Co., Belmont, CA, 1989
• 27: Byron J.T. Morgan, Elements of simulation, Chapman & Hall, Ltd., London, UK, 1984
• 28: Barry L. Nelson, Designing efficient simulation experiments, Proceedings of the 24th conference on Winter simulation, p.126-132, December 13-16, 1992, Arlington, Virginia, United States [doi>10.1145/167293.167317]
• 29: John S. Ramberg , Susan M. Sanchez , Paul J. Sanchez , Ludwig J. Hollick, Designing simulation experiments: Taguchi methods and response surface metamodels, Proceedings of the 23rd conference on Winter simulation, p.167-176, December 08-11, 1991, Phoenix, Arizona, United States
• 30: Brian D. Ripley, Stochastic simulation, John Wiley & Sons, Inc., New York, NY, 1987
• 31: Andrew F. Seila, Output analysis for simulation, Proceedings of the 23rd conference on Winter simulation, p.28-36, December 08-11, 1991, Phoenix, Arizona, United States
• 32: Andrew F. Seila, Advanced output analysis for simulation, Proceedings of the 24th conference on Winter simulation, p.190-197, December 13-16, 1992, Arlington, Virginia, United States [doi>10.1145/167293.167330]
• 33: Stephen G. Strickland, Gradient/sensitivity estimation in discrete-event simulation, Proceedings of the 25th conference on Winter simulation, p.97-105, December 12-15, 1993, Los Angeles, California, United States [doi>10.1145/256563.256594]
• 34: James J. Swain , Phillip A. Farrington, Designing simulation experiments for evaluating manufacturing systems, Proceedings of the 26th conference on Winter simulation, p.69-76, December 11-14, 1994, Orlando, Florida, United States
• 35: Susan M. Sanchez, A robust design tutorial, Proceedings of the 26th conference on Winter simulation, p.106-113, December 11-14, 1994, Orlando, Florida, United States
• 36: Arne Thesen , Laurel E. Travis , Robert J. Gordon, Simulation for Decision Making, West Publishing Co., St. Paul, MN, 1992

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING

Additional Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING
      I.6.1 Simulation Theory
          Subjects: Systems theory
      I.6.5 Model Development
          Subjects: Modeling methodologies

General Terms:
Design, Experimentation