Sensitivity analysis and optimization in simulation: design of experiments and case studies

ABSTRACT

This paper is an advanced tutorial on the use of statistical techniques in sensitivity analysis, including the application of these techniques to optimization and validation of simulation models. Sensitivity analysis is divided into two phases. The first phase is a pilot stage, which consists of screening or searching for the important factors; a simple technique is sequential bifurcation. In the second phase, regression analysis is used to approximate the input/output behavior of the simulation model. This regression analysis gives better results when the simulation experiment is well designed, using classical statistical designs such as fractional factorials. To optimize the simulated system, Response Surface Methodology (RSM) is applied; RSM combines regression analysis, design of experiments, and steepest ascent. To validate a simulation model that lacks input/output data, again regression analysis and design of experiments are applied. Several case studies are summarized; they illustrate how in practice statistical techniques can make simulation studies give more general results, in less time.

REFERENCES

• 1: Bettonvil, B. and J.P.C. Kleijnen. 1994. Identifying the important factors in simulation models with many factors. Tilburg University.
• 2: P W. Glynn , D. L. Iglehart, Importance sampling for stochastic simulations, Management Science, v.35 n.11, p.1367-1392, Nov. 1989 [doi>10.1287/mnsc.35.11.1367]
• 3: Ho,Y. and X. Cao. 1991. Perturbation analysis of discrete event systems. Dordrecht: Kluwer.
• 4: Sarah J. Hood , Peter D. Welch, Response surface methodology and its application in simulation, Proceedings of the 25th conference on Winter simulation, p.115-122, December 12-15, 1993, Los Angeles, California, United States [doi>10.1145/256563.256598]
• 5: Jack P C Kleijnen, Statistical tools for simulation practitioners, Marcel Dekker, Inc., New York, NY, 1986
• 6: Jack P. C. Kleijnen, Simulation and optimization in production planning: a case study, Decision Support Systems, v.9 n.3, p.269-280, April 1993 [doi>10.1016/0167-9236(93)90058-B]
• 7: Kleijnen, J.P.C. 1995a. Verification and validation of simulation models. European Journal of Operational Research 82:145-162.
• 8: Kleijnen, J.P.C. 1995b. Statistical validation of simulation models: a case-study. European Journal of Operational Research (in press).
• 9: Kleijnen, J.P.C. 1996. Simulation: sensitivity analysis and optimization through regression analysis and experimental design. In Proceedings of NATO Advanced Study Institute on Current Issues and Challenges in the Reliability and Maintenance of Complex Systems, Heidelberg: Springer-Verlag.
• 10: Kleijnen, J.P.C, B. Bettonvil, and W. Van Groenendaal. 1995. Validation of simulation models: regression analysis revisited. Tilburg University.
• 11: Kleijnen, J.P.C., G. Van Ham, and J. Rotmans. 1992. Techniques for sensitivity analysis of simulation models: a case study of the CO2 greenhouse effect. Simulation 58: 410-417.
• 12: Jack P. C. Kleijnen , Willen van Groenendaal, Simulation: a statistical perspective, John Wiley & Sons, Inc., New York, NY, 1992
• 13: Oren, T.I. 1993. Three simulation experimentation environments: SIMAD, SIMGEST, and E/SLAM. In Proceedings of the 1993 European Simulation Symposium. La Jolla: Society for Computer Simulation.
• 14: Rubinstein, R.Y. and A. Shapiro. 1993. Discrete event systems: sensitivity analysis and stochastic optimization via the score function method, New York: Wiley.
• 15: Van Groenendaal, W. 1994. Investment analysis and DSS for gas transmission on Java. Tilburg (Netherlands): Tilburg University.
• 16: Van Meel, J. 1994. The dynamics of business engineering. Delft (Netherlands): Delft University.
• 17: Bernard P. Zeigler, Theory of Modelling and Simulation, Krieger Publishing Co., Inc., Melbourne, FL, 1984

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING

Additional Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Statistical computing

  I. Computing Methodologies
  I.6 SIMULATION AND MODELING

General Terms:
Design, Experimentation