An expository on verification and validation of simulation models

ABSTRACT

In this expository paper we give a general introduction to verification and validation of simulation models, define the various validation techniques, and present a recommended model validation procedure.

REFERENCES

• 1: Anderson, H.A. and R.G. Sargent (1974), "An Investigation into Scheduling for an Interactive Computer System," IBM Journal of Research and Development, 18, 2, pp. 125-137.
• 2: Osman Balci , Robert G. Sargent, A methodology for cost-risk analysis in the statistical validation of simulation models, Communications of the ACM, v.24 n.4, p.190-197, April 1981 [doi>10.1145/358598.358609]
• 3: Balci, O. and R.G. Sargent (1982a), "Validation of Multivariate Response Simulation Models by Using Hotelling's Two-Sample T2 Test," Simulation, Vol. 39, No. 6, pp. 195-192.
• 4: Osman Balci , Robert G. Sargent, Some examples of simulation model validation using hypothesis testing, Proceedings of the 14th conference on Winter Simulation, December 06-08, 1982, San Diego, California
• 5: Osman Balci , Robert G. Sargent, Validation fo Multivariate Response Trace-driven Simulation Models, Proceedings of the 9th International Symposium on Computer Performance Modelling, Measurement and Evaluation, p.309-323, May 25-27, 1983
• 6: Balcl, O. and R.G. Sargent (1984a), "A Bibliography on the Credibility, Assessment and Validation of Simulation and Mathematical Models," Simuletter~ Vol. 15, No. 3, pp. 15-27.
• 7: Balci, O. and R.G. Sargent (1984b), "Validation of Simulation Models via Simultaneous Confidence Intervais,'~ American Journal of Mathematical and Management Science, Vol. 4, No. 3 and 4, pp. 375- 406.
• 8: Banks, J. and J.S. Carson ii (i984), Discrete-Event System Simulation, Prentice-Hail, N.J.
• 9: Chattergy,R. and V.W. Pooch (1977), "Integrated Design and Verification of Simulation Programs," Computer, Vol. i0, No. 4, pp. 40-46.
• 10: Chen, B. and R.G. Sargent (1984), "Confidence Interval Estimation for the Difference Between Two Stochastic Processes Using Standardized Time Series," Working Paper #84-004, Department of Industrial Engineering and Operations Research, Syracuse University.
• 11: Fairley, R.E. (i976), "Dynamic Testing of Simulation Software," Proceedinss of 1976 Summer Computer Simulation Conference, Washington, D.C., pp. 40-46.
• 12: Gass, S.I. and B.W. Thompson (1980), "Guidelines for Model Evaluation: An Abridged Version of the UoS. General Accounting Office Exposure Draft," Operations Research, Z$, Z, 431-479.
• 13: Gass, S.I. (1983), "Decision-Aiding Models: Validation, Assessment, and Related Issues for Policy Analysis," O peraions Research, 31, 4, pp. 601-63.
• 14: Kleijnen, J.P.C. (1975), Statistical Techniques in Simulation, Part II, Marcel Dekker, New York.
• 15: Kleijnen, J.P.C. (1982), "Experimentation with Models: Statistical Design and Analysis Techniques," Progress in Modelling & Simulation, F.E. Celller, Editor, Academic Press, London.
• 16: Law, A.M. (i9831 "Statistical Analysis of Simulation Output Data," Operations Research, Voi. 31, No. 6, pp. 983-1029.
• 17: Averill M. Law , W. David Kelton, Simulation Modeling and Analysis, McGraw-Hill Higher Education, 1997
• 18: Miller, I. and J.E. Feund (1985), Probability and Statistics for Engineers, Prentice-Hail, Englewood Cliffs, N.J.
• 19: Naylor, T.H. and J.M. Finger (1967), "Verification of Computer Simulation Models," Management Science, 14, 2, pp. B92-B101.
• 20: Tuncer I. Ören, Concepts and criteria to assess acceptability of simulation studies: a frame of reference, Communications of the ACM, v.24 n.4, p.180-189, April 1981 [doi>10.1145/358598.358605]
• 21: Robert G. Sargent, Validation of simulation models, Proceedings of the 11th conference on Winter simulation, p.497-503, December 03-05, 1979, San Diego, CA, United States
• 22: Sargent, R.G. (}_981), "An Assessment Procedure and a Set of Criteria for Use in the Evaluation of Computerized Models and Computer-Based Modelling Tools," Final Technical Report RADC-TR-80-409.
• 23: Sargent, R.G. (1.982), "Verification and Validation of Simulation Models," Chapter IX in Progress in Modelling and Simulation, edited by F.E. Celiier, Academic Press (London), pp. 159-i69.
• 24: Sargent, R.G. (J984a), "Simulation Model Validation," Simulation and Model-Based Methodologies: an Integrative View, edited by Oren et ai., Springer-Ve rlag.
• 25: Robert G. Sargent, A tutorial on verification and validation of simulation models, Proceedings of the 16th conference on Winter simulation, p.114-121, December 28-30, 1984, Dallas, TX
• 26: Schlesinger, et al. (1979), "Terminology for Model Credibility," Simulation, 32, 3, pp. i03-I04.
• 27: Shannon, R.E. (1975), Systems Simulation: The Art and the Science, Prentice-Hall.
• 28: Robert E. Shannon, Tests for the verification and validation of computer simulation models, Proceedings of the 13th conference on Winter simulation, p.573-577, December 09-11, 1981, Atlanta, Georgia, United States
• 29: Schruben, L.W. (1980), "Establishing the CredibiiiEy of Simulations," Simulation, 34, 3, pp. I0i-i05.
• 30: Smith, D.E. and CoA. Mauro (1982), "Factor Screening in Computer Simulation," Simulation, Voi. 38, No. 2, pp. 49-54.
• 31: Bernard P. Zeigler, Theory of Modelling and Simulation, Krieger Publishing Co., Inc., Melbourne, FL, 1984