Using Covariance Analysis as an aid to interpret the results of a performance measurement

ABSTRACT

This paper reports on a measurement study of the Scheduler in the PDP-10 TENEX Operating System under normal loads in a Computer Science research environment at Case Western Reserve University. A complete description of the methodology and the results of a Covariance Analysis is presented here. The analysis was made in parallel with the analysis of the collected data, therefore a brief description of the experiments as well as some comments of the most interesting results are also discussed in this report. The complete report of the study is in González [5] which shall serve as a reference document for this paper.

REFERENCES

• 1: Anderson, H.A., and Sargent, R.G. "A Statistical Evaluation of the Scheduler of an Experimental Interactive Computing System", Statistical Computer Performance Evaluation, W. Freiberger, Editor, Academic Press, 1972, pp. 73-98.
• 2: BBN-TENEX Memo 12, "Scheduling and Storage Management",# Cambridge, Mass., 1971.
• 3: Daniel G. Bobrow , Jerry D. Burchfiel , Daniel L. Murphy , Raymond S. Tomlinson, TENEX, a paged time sharing system for the PDP-10, Proceedings of the third ACM symposium on Operating systems principles, p.1, October 18-20, 1971, Palo Alto, California, United States [doi>10.1145/800212.806492]
• 4: Peter J. Denning, The working set model for program behavior, Communications of the ACM, v.11 n.5, p.323-333, May 1968 [doi>10.1145/363095.363141]
• 5: Carlos Manuel Gonzalez, Performance measurements of the scheduler in the pdp-10 tenex operating system., 1975
• 6: Goodman, A.F., "Data Modeling and analysis for Users - A guide for the Perplexed", Proc. AFIPS Vol. 41, FJCC 1972, pp. 1163-1169.
• 7: Harman, H., "Factor Analysis", Mathematical Methods for Digital Computers, Edited by A. Ralston and H.S. Wilf. John Wiley and Sons, 1965.
• 8: Gerald Waldbaum, Evaluating computing system changes by means of regression models, Proceeding of the 1973 ACM SIGME symposium, p.127-135, January 1973 [doi>10.1145/800268.809344]
• 9: Bard Y. "Performance Criteria and Measurements for a Time-Sharing System" IBM Systems Journal, Vol. 10, No. 3, pp. 193-216, 1971.
• 10: Michael F. Morris, Kiviat graphs: conventions and "figures of merit", ACM SIGMETRICS Performance Evaluation Review, v.3 n.3, p.2-8, October 1974 [doi>10.1145/1041691.1041692]
• 11: H. E. Barry Merrill, A technique for comparative analysis of Kiviat graphs, ACM SIGMETRICS Performance Evaluation Review, v.3 n.1, p.34-39, March 1974 [doi>10.1145/1041619.1041622]
• 12: B. W. Boehm , T. E. Bell, Issues in computer performance evaluation: some consensus, some divergence, ACM SIGMETRICS Performance Evaluation Review, v.4 n.3, p.4-39, July 1975 [doi>10.1145/1041707.1041709]

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.8 Performance

Additional Classification:
  G. Mathematics of Computing
  G.1 NUMERICAL ANALYSIS
      G.1.3 Numerical Linear Algebra

General Terms:
Measurement, Performance