An overview of verification and validation of simulation models

ABSTRACT

We give a general introduction to verification and validation of simulation models, define the various validation techniques, and present a recommended model validation procedure in this overview paper.

REFERENCES

• 1: Anderson, H.A. and R.G. Sargent (1974). An Investigation into Schedu2ing for an Interactive Cornputer System, 1BM Journal of Research and Development, 18, 2, pp. 125-137.
• 2: Balci, O. (1987). Credibility Assessment of Simulation Results: The State of the Art, Methodology and Validation, Simulation Series, Vol. 19, No. l, The Society for Computer Simulation, pp. 19-25.
• 3: Osman Balci , Robert G. Sargent, A methodology for cost-risk analysis in the statistical validation of simulation models, Communications of the ACM, v.24 n.4, p.190-197, April 1981 [doi>10.1145/358598.358609]
• 4: Balci, O. and R.G. Sargent (1982a). Validation of Multivariate Respon~'~e Simulation Models by Using {-lotelling's Two-Sample T2 Test, Simulation, Vol. 39, No. 6, pp. 185-192.
• 5: Osman Balci , Robert G. Sargent, Some examples of simulation model validation using hypothesis testing, Proceedings of the 14th conference on Winter Simulation, December 06-08, 1982, San Diego, California
• 6: Osman Balci , Robert G. Sargent, Validation fo Multivariate Response Trace-driven Simulation Models, Proceedings of the 9th International Symposium on Computer Performance Modelling, Measurement and Evaluation, p.309-323, May 25-27, 1983
• 7: Balci, O. and R.G. Sargent (I984a). A Bibliography on the Credibility, Assessment and Validation of Simulation and Mathematical Models, Simuletter, Vol. 15, No. 3, pp. 15-27,
• 8: Balci, O. and R.G. Sargent (1984b). Validation of Simulation Models via Simultaneous Confidence Intervals, American Journal of Mathematical and Management Science, Vol. 4, No. 3 and 4, pp. 375- 406.
• 9: Banks, J. and J.S. Carson II (1984). Discrete-Event System Simulation, Prentice-Hall, New Jersey.
• 10: Banks, 5., D. Gerstein, and S.P. Searles (1987). Modeling Processes, Validation, and Verification of Complex Simulations: A Survey, Methodology and Validation, Simulation Series, Vol. 19, No. 1. The Society for Computer Simulation, pp. 13-18.
• 11: Chattergy, R. and V.W. Pooch (I977). Integrated Design and Verification of Simulation Programs, Computer, Vol. 10, No. 4, pp. 40-46.
• 12: B. C. Chen , R. G. Sargent, Using standardized time series to estimate confidence intervals for the difference between two stationary stochastic processes, Operations Research, v.35 n.3, p.428-436, May-June 1987 [doi>10.1287/opre.35.3.428]
• 13: Edward A. Davis, Use of seminar gaming to specify and validate simulation models, Proceedings of the 18th conference on Winter simulation, p.242-247, December 08-10, 1986, Washington, D.C., United States [doi>10.1145/318242.318313]
• 14: Fairley, R.E, (1976). Dynamic Testing of Simulation Software, Proceedings of the 1976 Summer Computer Simulation Conference, Washington, D.C., pp. 40-46.
• 15: Gass, S.I. and B.W. Thompson (1980). Guidelines for Model Evaluation: An Abridged Version of the U.S. General Accounting Office Exposure Draft, Operations Research, 28, 2, pp. 431-479.
• 16: Gass, S.I. (1983). Decision-Aiding Models: Validation, Assessment, and Related Issues for Policy Analysis, Operations Research, 31, 4, pp. 601-663.
• 17: Jack P C Kleijnen, Statistical tools for simulation practitioners, Marcel Dekker, Inc., New York, NY, 1986
• 18: Kleijnen, J.P.C. (1982). Experimentation with Models: Statistical Design and Analysis Techniques, Progress in Modelling and Simulation, F.E. Cellier, Editor, Academic Press, London.
• 19: Law, A.M. (1983). Statistical Analysis of Simulation Output Data, Operations Research, Vol. 31, No. 6, pp. 983-1029.
• 20: Averill M. Law , W. David Kelton, Simulation Modeling and Analysis, McGraw-Hill Higher Education, 1997
• 21: Miller, I. and J.E. Feund (1985). Probability and Statistics for Engineers, Prentice-Hall, Englewood Cliffs, New Jersey.
• 22: Naylor, T.H. and J.M. Finger (1967). Verification of Computer Simulation Models, Management Science, 14, 2, pp. B92-B 101.
• 23: Tuncer I. Ören, Concepts and criteria to assess acceptability of simulation studies: a frame of reference, Communications of the ACM, v.24 n.4, p.180-189, April 1981 [doi>10.1145/358598.358605]
• 24: Robert G. Sargent, Validation of simulation models, Proceedings of the 11th conference on Winter simulation, p.497-503, December 03-05, 1979, San Diego, CA, United States
• 25: Sargent, R.G. (1981). An Assessment Procedure and a Set of Criteria for Use in the Evaluation of Computerized Models and Computer-Based Modelling Tools, Final Technical Report RADC-TR-80-409.
• 26: Sargent, R.G. (1982). Verification and Validation of Simulation Models, Chapter IX in Progress in Modelling and Simulation, edited by F.E. Cellier, Academic Press, London, pp. 159-169.
• 27: Sargent, R.G. (1984). Simulation Model Validation, Simulat'.t.on and Model-Based Methodologies: A n integrative View, edited by Oven, et al., Springer-Verlag.
• 28: Robert G. Sargent, An expository on verification and validation of simulation models, Proceedings of the 17th conference on Winter simulation, p.15-22, December 11-13, 1985, San Francisco, California, United States [doi>10.1145/21850.253058]
• 29: Robert G. Sargent, The use of graphical models in model validation, Proceedings of the 18th conference on Winter simulation, p.237-241, December 08-10, 1986, Washington, D.C., United States [doi>10.1145/318242.318312]
• 30: Schlesinger, et al. (1979). Terminology for Model Credibility, Simulation, 32, 3, pp. 103-104.
• 31: Schruben, L.W. (1980). Establishing the Credibility of Simulations, Simulation, 34, 3, pp. 101-105.
• 32: Shannon, R.B. (1975). Systems Simulation: The Art and the Science, Prentice-Hall.
• 33: Robert E. Shannon, Tests for the verification and validation of computer simulation models, Proceedings of the 13th conference on Winter simulation, p.573-577, December 09-11, 1981, Atlanta, Georgia, United States
• 34: Smith, D.E. and C.A. Mauro (1982). Factor Screening in Computer Simulation, Simulan'on, Vol. 38, No. 2, pp. 49-54.
• 35: David O. Wood, MIT model analysis program: what we have learned about policy model review, Proceedings of the 18th conference on Winter simulation, p.248-252, December 08-10, 1986, Washington, D.C., United States [doi>10.1145/318242.318314]
• 36: Bernard P. Zeigler, Theory of Modelling and Simulation, Krieger Publishing Co., Inc., Melbourne, FL, 1984

INDEX TERMS

Primary Classification:
  A. General Literature

Additional Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING

General Terms:
Design, Measurement, Performance, Reliability, Theory, Verification