Achieving specific accuracy in simulation output analysis

ABSTRACT

This paper extends the use of the regenerative property of queueing systems in the analysis of simulation output. In particular, it describes a sequential estimation method which when used with the regenerative property allows results to be obtained with specified statistical accuracy. This method includes a test to check the normality assumption on which the sequential procedure relies. The paper illustrates the method using the empty and idle state as the regenerative state. A second example then describes how using the most frequently entered state as the regenerative state reduces the chance of making a costly error in a preliminary simulation run. The paper also described how a variance reduction method due to Page [9] can be used to obtain a specified accuracy with considerably fewer job completions than are required when no variance reduction technique is applied.

REFERENCES

• 1: Chow, Y.S., and Robbins, H. On the asymptotic theory of fixed width sequential confidence intervals for the mean. Ann. Math. Stat. 36 (1965), 457-462.
• 2: Cox, D.R. Estimation by double sampling. Biometrika 39 (1952), 217-227.
• 3: Michael A. Crane , Donald L. Iglehart, Simulating Stable Stochastic Systems, I: General Multiserver Queues, Journal of the ACM (JACM), v.21 n.1, p.103-113, Jan. 1974 [doi>10.1145/321796.321805]
• 4: Michael A. Crane , Donald L. Iglehart, Simulating Stable Stochastic Systems, II: Markov Chains, Journal of the ACM (JACM), v.21 n.1, p.114-123, Jan. 1974 [doi>10.1145/321796.321806]
• 5: Fishman, G.S. Concepts and Methods in Discrete Event Digital Simulation. Wiley, New York, 1973.
• 6: Fishman, G.S. Statistical analysis for queueing simulations. Manage Sci. 20, 3 (Nov. 1973), 363-369.
• 7: Fishman, G.S. Estimation in multiserver queueing simulations. Oper. Res. 22, 1 (Jan.-Feb., 1974), 72-78.
• 8: Fishman, G.S. Simulation data analysis using continuous time markov processes. Tech. Rep. No. 74-1, Oper. Res. and Syst. Analysis, U. of North Carolina, Chapel Hill, N.C., Nov. 1974.
• 9: Page, E.S. On Monte Carlo methods in congestion problems II: Simulation of queueing systems. Oper. Res. 13, 2 (1965), 300-305.
• 10: Shapiro, S.S., and Wilk, M.B. An analysis of variance test for normality. Biometrika 52, 3 and 4 (1965), 591-611.
• 11: Shapiro, S.S., Wilk, M.B., and Chen, H.J. A comparative study of various tests of normality. J. ASA 63, (1968), 1343-1372.
• 12: Starr, N. The performance of a sequential procedure for the fixed-width interval estimation of the mean. Ann. Math. Stat. 37, 1 (Feb. 1966), 36-50.
• 13: Tin, M. Comparison of some ratio measures. J. ASA 60 (1965), 294--307.

INDEX TERMS

Keywords:
confidence interval, ratio estimator, regenerative property, sequential estimation, simulation, stopping rule, variance reduction