Using nonparametric statistics in simulation analysis: a review

ABSTRACT

Techniques that make the minimum of assumptions about the underlying characteristics of the simulation output series are particularly useful for simulation analysis. This tutorial discusses robust non-parametric techniques with immediate applicability to such crucial steps in simulation analysis as sampling, experimental design, and output analysis. Algorithms are provided for various tasks.

REFERENCES

• 1: Russell R. Barton , Lee W. Schruben, Uniform and bootstrap resampling of empirical distributions, Proceedings of the 25th conference on Winter simulation, p.503-508, December 12-15, 1993, Los Angeles, California, United States [doi>10.1145/256563.256699]
• 2: Berry, K.J. and P.W. Jr. Mielke. 1985. Computation of Exact and Approxinmte Probability Values for a Matched Pairs Permutation Test. Communications in Statistics - Part B: Simulation and Computation, 14: 229-248.
• 3: Box, G.E.P. and S.L. Aixterselt 1955. Permutation Theory in the Derivation of Robust Criteria and the Study of Delmrtures from Assumptions. Journal of the Royal Statistical Society, Series B, 17: 1-34.
• 4: Eugene S Edgington, Randomization tests, Marcel Dekker, Inc., New York, NY, 1986
• 5: Efron, B. 1979. Bootstrap Methods: Another Look at Jackknife. Annals of Statistics, 7: 1-26.
• 6: Efron, B. and R. Tibshirani. 1984. Bootstrap Methods for Standard Errors, Confidence Intervals and Other Measures of Statistical Accuracy. Statistical Science, 1: 54-77.
• 7: Fisher, R.A. 1925. Statistical Methods for Research Workers. Edinburgh: Oliver & Boyd.
• 8: Fisher, R.A. 1935. The Design of Experiments. Edinburgh: Oliver & Boyd.
• 9: Foutz, R.V. 1980. A Method for Constructing Exact Tests from Test Statistics that Have Unknown Null Distributions. J. Statist. Comput. Simul., 10: 187-193.
• 10: Good, P. 1994. Permutation Tests. New York: Springer- Verlag.
• 11: Hoeffding, W. 1952. The Large Smnple Power of Tests Based on Permutations of Observations. Annals of Mathematical Statistics, 23: 169-192.
• 12: Y. B. Kim , T. R. Willemain , J. Haddock , G. C. Runger, The threshold bootstrap: a new approach to simulation output analysis, Proceedings of the 25th conference on Winter simulation, p.498-502, December 12-15, 1993, Los Angeles, California, United States [doi>10.1145/256563.256697]
• 13: Pierre L'Ecuyer, Random numbers for simulation, Communications of the ACM, v.33 n.10, p.85-97, Oct. 1990 [doi>10.1145/84537.84555]
• 14: Noreen, E. 1989. Computer Intensive Methods for Testing Hypotheses: An Introduction. New York: Wiley.
• 15: Pagano, M. and D. Tritchler. 1983. On Obtaining Permutation Distributions in Polynomial Time. Journal of the American Statistical Association, 78: 435-440.
• 16: Scheff6, H. 1959. The Analysis of Variance. New York: Wiley.
• 17: Spino, C. and M. Pagano. 1991. Efficient Calculation of the Permutation Distribution of Trimmed Means. Journal of the American Statistical Association, 86: 729-737.
• 18: Welch, B.L. 1938. The Significance of the Difference between two Means when the Population Variances are Unequal. Biometrica, 25: 350-362.
• 19: Yiicesan, E. 1992. Evaluating Random Number Generators Using Permutation Tests. In Proceedings of the 1992 European Simulation Multiconference, ed. Stephenson, 94-98.
• 20: Yiicesan, E. 1993. Randomization Tests for Initial Bias in Simulation Output. Naval Research Logistics, 40: 643-663.
• 21: Yticesan, E. 1994. Comparing Alternative System Configurations: A Nonparametric Approach. Annals of Operations Research, 53:471-484.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING

Additional Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Modeling techniques

  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Statistical computing

General Terms:
Algorithms, Experimentation, Theory, Verification