Verification and validation: some approaches and paradigms for verifying and validating simulation models

ABSTRACT

In this paper we discuss verification and validation of simulation models. The different approaches to deciding model validity are described, two different paradigms that relate verification and validation to the model development process are presented, the use of graphical data statistical references for operational validity is discussed, and a recommended procedure for model validation is given.

REFERENCES

• 1: Harold Albert Anderson, Jr., An empirical investigation into foreground-background scheduling for an interactive computing system, 1972
• 2: Anderson, H. A. and R. G. Sargent. 1974. An Investigation into Scheduling for an Interactive Computer System. IBM Journal of Research and Development, 18 (2): 125-137.
• 3: Banks, J., D. Gerstein, and S. P. Searles. 1988. Modeling Processes, Validation, and Verification of Complex Simulations: A Survey. In Methodology and Validation, Simulation Series, Vol. 19, No. 1, The Society for Computer Simulation, 13-18. San Diego, CA: Society for Modeling and Simulation International.
• 4: O. Balci, How to assess the acceptability and credibility of simulation results, Proceedings of the 21st conference on Winter simulation, p.62-71, December 04-06, 1989, Washington, D.C., United States [doi>10.1145/76738.76746]
• 5: Gass, S. I. 1993. Model Accreditation: A Rationale and Process for Determining a Numerical Rating. European Journal of Operational Research, 66(2): 250-258.
• 6: Gass, S. I. and L. Joel. 1987. Concepts of Model Confidence. Computers and Operations Research, 8 (4): 341-346.
• 7: Johnson, R. A. 1994. Miller and Freund's Probability and Statistics for Engineers, 5th edition. Englewood Cliffs, NJ: Prentice-Hall.
• 8: Julie C. Lowery, Design of hospital admissions scheduling system using simulation, Proceedings of the 28th conference on Winter simulation, p.1199-1204, December 08-11, 1996, Coronado, California, United States [doi>10.1145/256562.256932]
• 9: Pace, D. K. 2001a. Simulation Conceptual Model Role in Determining Compatibility of Candidate Simulations for a HLA Federation. In Proceedings of 2001 Spring Simulation Interoperability Workshop, Paper 01S-SIW-024. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers. Available online via <http://www.sisostds.org/> {accessed July 16, 2001}.
• 10: Pace, D. K. 2001b. Impact of Federate Conceptual Model Quality and Documentation on Assessing HLA Federation Validity. In Proceedings of 2001 European Simulation Interoperability Workshop, Paper 01E-SIW-014. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers. Available online via <http://www.sisostds.org/> {accessed July 16, 2001}.
• 11: Sargent, R. G. 1982. Verification and Validation of Simulation Models. Chapter IX in Progress in Modelling and Simulation, ed. F. E. Cellier, 159-169. London: Academic Press.
• 12: Sargent, R. G. 1984. Simulation Model Validation. Chapter 19 in Simulation and Model-Based Methodologies: An Integrative View, ed. T. I. Oren, B. P. Zeigler, and M. S. Elzas, 537-555. Heidelberg, Germany: Springer-Verlag.
• 13: Robert G. Sargent, Some subjective validation methods using graphical displays of data, Proceedings of the 28th conference on Winter simulation, p.345-351, December 08-11, 1996, Coronado, California, United States [doi>10.1145/256562.256648]
• 14: Robert G. Sargent, Verification, validation, and accreditation: verification, validation, and accreditation of simulation models, Proceedings of the 32nd conference on Winter simulation, December 10-13, 2000, Orlando, Florida
• 15: Sargent, R. G. 2001. Graphical Displays of Simulation Model Data as Statistical References. In Simulation 2001 (Proceedings of the 4th St. Petersburg Workshop on Simulation), ed. S. M. Ermakor, Yu. N. Kashtanov, and V. B. Melas, 109-118. Publisher: Chemistry Research Institute of St. Petersburg University.
• 16: Schlesinger, S. et al. 1979. Terminology for Model Credibility. Simulation, 32,3, 103-104.
• 17: Shannon, R. E. 1975. System Simulation: The Art and the Science. Englewood Cliffs, NJ: Prentice-Hall.
• 18: Walpole, R. and R. Myers. 1993. Probability and Statistics for Engineers and Scientists, 5th edition. New York: Macmillan Publishing Company.
• 19: David O. Wood, MIT model analysis program: what we have learned about policy model review, Proceedings of the 18th conference on Winter simulation, p.248-252, December 08-10, 1986, Washington, D.C., United States [doi>10.1145/318242.318314]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING
      I.6.4 Model Validation and Analysis

Additional Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING
      I.6.1 Simulation Theory
          Subjects: Model classification
      I.6.7 Simulation Support Systems
          Subjects: Environments

General Terms:
Algorithms, Theory, Verification