Automated analysis of simulation output data

ABSTRACT

Appropriate analysis of simulation output is important to the success of a simulation study. Many users, however, do not have the skills required to perform such analyses. One way of overcoming this problem is to provide automated tools for analyzing simulation output. An Excel based automated "Analyser" is described that performs an analysis of a single scenario. The Analyser links to a commercial simulation software package, SIMUL8, and provides recommendations on warm-up, number of replications and run-length. Various standard procedures are used in the Analyser with some adaptations to make them suitable for automation. This research demonstrates the potential of the approach. A requirement for further development is more thorough testing of the analysis procedures, particularly for their robustness and generality in use. Further adaptation of the procedures for automation may also be required.

REFERENCES

• 1: Banks, J., J. S. Carson, B. L. Nelson and D. M. Nicol. 2001. Discrete-Event System Simulation, 3^rd ed. Prentice Hall, Upper Saddle River, NJ.
• 2: Diane P. Bischak , W. David Kelton , Stephen M. Pollock, Weighted batch means for confidence intervals in steady-state simulations, Management Science, v.39 n.8, p.1002-1019, Aug. 1993 [doi>10.1287/mnsc.39.8.1002]
• 3: Fishman, G. 1978. Grouping Observations in Digital Simulation. Management Science, 24, 510--521.
• 4: Fishman, G. and L. Yarberry. 1997. An Implementation of the Batch Means Method. INFORMS Journal of Computing, 9, 296--310.
• 5: Fox, F., D. Goldsman and J. Swain. 1991. Spaced Batch Means. Operations Research Letters, 10, 255--263.
• 6: Gafarian, A. V., C. J. Ancker and T. Morisaku. 1978. Evaluation of Commonly Used Rules for Detecting "Steady State" in Computer Simulation. Naval Research Logistics Quarterly, 25, 511--529.
• 7: Goldsman, D., L. W. Schruben and J. J. Swain. 1994. Tests for Transient Means in Simulated Time Series. Naval Research Logistics, 41, 171--187.
• 8: Hollocks, B. W. 2001. Discrete-Event Simulation: An Inquiry into User Practice. Simulation Practice and Theory, 8, 451--471.
• 9: Ingalls, RG., M. D. Rossetti, J. S. Smith and B. A. Peters. 2004. Proceeding of the 2004 Winter Simulation Conference. IEEE, Piscataway, NJ.
• 10: Law, A. and J. Carson. 1979. A Sequential Procedure for Determining the Length of a Steady-State Simulation. Operations Research, 27, 1011--1025.
• 11: Averill M. Law , W. David Kelton, Simulation Modeling and Analysis, McGraw-Hill Higher Education, 1997
• 12: Averill M. Law , Michael G. McComas, Simulation optimization: simulation-based optimization, Proceedings of the 34th conference on Winter simulation: exploring new frontiers, December 08-11, 2002, San Diego, California
• 13: Marc S. Meketon , Bruce Schmeiser, Overlapping batch means: something for nothing?, Proceedings of the 16th conference on Winter simulation, p.226-230, December 28-30, 1984, Dallas, TX
• 14: Krzysztof Pawlikowski, Steady-state simulation of queueing processes: survey of problems and solutions, ACM Computing Surveys (CSUR), v.22 n.2, p.123-170, June 1990 [doi>10.1145/78919.78921]
• 15: Stewart Robinson, New simulation output analysis techniques: a statistical process control approach for estimating the warm-up period, Proceedings of the 34th conference on Winter simulation: exploring new frontiers, December 08-11, 2002, San Diego, California
• 16: Stewart Robinson, Simulation: The Practice of Model Development and Use, John Wiley & Sons, 2004
• 17: Roth, E. 1994. The Relaxation Time Heuristic for the Initial Transient Problem in M/M/k Queueing Systems. European Journal of Operational Research, 72, 376--386.
• 18: Welch, P. D. 1983. The Statistical Analysis of Simulation Results. The Computer Performance Modeling Handbook (Lavenberg, S., ed.). Academic Press, New York, 268--328.
• 19: K. Preston White, Jr. , Michael J. Cobb , Stephen C. Spratt, A comparison of five steady-state truncation heuristics for simulation, Proceedings of the 32nd conference on Winter simulation, December 10-13, 2000, Orlando, Florida
• 20: Wilson, J. R. and Pritsker, A. B. 1978. A Survey of Research on the Simulation Startup Problem. Simulation, 31, 55--59.