Building valid models: how to build valid and credible simulation models

ABSTRACT

In this tutorial we present techniques for building valid and credible simulation models. Ideas to be discussed include the importance of a definitive problem formulation, discussions with subject-matter experts, interacting with the decision-maker on a regular basis, development of a written conceptual model, structured walk-through of the conceptual model, use of sensitivity analysis to determine important model factors, and comparison of model and system performance measures for an existing system (if any). Each idea will be illustrated by one or more real-world examples. We will also discuss the difficulty in using formal statistical techniques (e.g., confidence intervals) to validate simulation models.

REFERENCES

• 1: Balci, O. 1998. "Verification, Validation and Testing," in The Handbook of Simulation, J. Banks, ed., Chapter 10, John Wiley, New York.
• 2: Banks, J., J. S. Carson, B. L. Nelson, and D. M. Nicol. 2001. Discrete-Event System Simulation, Third Edition, Prentice-Hall, Upper Saddle River, N. J.
• 3: Carson J. S. 1986. "Convincing Users of Model's Validity Is Challenging Aspect of Modeler's Job," Industrial Engineering, 18: 74-85.
• 4: Law, A. M. 1991. "Simulation Study Puts the Right Heat On at Kaiser Aluminum," Industrial Engineering, 23: 16-17.
• 5: Averill M. Law , David M. Kelton, Simulation Modeling and Analysis, McGraw-Hill Higher Education, 1999
• 6: Montgomery, D. C. 2000. Design and Analysis of Experiments, 5th Edition, John Wiley, New York.
• 7: Robert G. Sargent, Verifying and validating simulation models, Proceedings of the 28th conference on Winter simulation, p.55-64, December 08-11, 1996, Coronado, California, United States [doi>10.1145/256562.256572]
• 8: Shannon, R. E. 1975. Systems Simulation: The Art and Science, Prentice-Hall, Englewood Cliffs, N.J.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING
      I.6.4 Model Validation and Analysis

Additional Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING
      I.6.1 Simulation Theory
          Subjects: Model classification
      I.6.5 Model Development
          Subjects: Modeling methodologies

General Terms:
Algorithms, Design, Theory