Measurements of program similarity in identical task environments

ABSTRACT

This paper summarizes the results of a study which compared the efficiency of two methods of measuring program similarity in the context of novice programmers trying to reach identical objectives. Both methods look for similarity by comparing 'program profiles'. Such profiles are created by feature extraction routines which map each program onto a tuple <f1, f2,..., fn> where each fi is a count of an occurrence of a particular feature. A comparison routine is then invoked which detects similarities between tuples. The results showed that in this environment the comparison routine based on the Halstead metric failed to perform as well as a conceptually simpler alternative.

REFERENCES

• 1: Berghel, H. and C. Daly: "A Comparison of Three Approaches Toward Teaching Computer Literacy", <u>Proceedings of the IEEE ED COMPCON-83</u>, IEEE Computer Society, Spring, 1984 (forthcoming).
• 2: Berghel, H. and D. Sallach: "Identifying Program Similarity: the Limits of the Halstead Metric" (forthcoming).
• 3: Necdet Bulut, Invariant properties of algorithms., 1973
• 4: Neodet Bulut , Maurice H. Halstead, Impurities found in algorithm implementations, ACM SIGPLAN Notices, v.9 n.3, March 1974 [doi>10.1145/987419.987421]
• 5: Bulut N., M. Halstead, and R. Bayer: "Experimental Validation of a Structured Property of FORTRAN Algorithms", Computer Science Department Internal Report CSD-TR 115, Purdue University, (April 1974).
• 6: Cornell, L. and M. Halstead, "Predicting the Number of Bugs Expected in a Program Module", Computer Science Department Internal Report CSD-TR 205, Purdue University, (October, 1976).
• 7: John L. Donaldson , Ann-Marie Lancaster , Paula H. Sposato, A plagiarism detection system, ACM SIGCSE Bulletin, v.13 n.1, p.21-25, February 1981
• 8: James L. Elshoff, Measuring commercial PL/I programs using Halstead's criteria, ACM SIGPLAN Notices, v.11 n.5, p.38-46, May 1976 [doi>10.1145/956003.956006]
• 9: Ann Fitzsimmons , Tom Love, A Review and Evaluation of Software Science, ACM Computing Surveys (CSUR), v.10 n.1, p.3-18, March 1978 [doi>10.1145/356715.356717]
• 10: Gordon, R. and M. Halstead, "An Experiment Comparing FORTRAN Programming Time with the Software Physics Hypothesis", <u>Proceedings of the AFIPS National Computer Conference</u>, Vol. 45, AFIPS Press, Montvale, NJ, 1976, pp. 935--937.
• 11: Sam Grier, A tool that detects plagiarism in Pascal programs, ACM SIGCSE Bulletin, v.13 n.1, p.15-20, February 1981
• 12: Halstead, M.: "A Theoretical Relationship Between Mental Work and Machine Language Programming", Computer Science Department Internal Report CSD-TR 67, Purdue University, (February 1972).
• 13: M. H. Halstead, Natural laws controlling algorithm structure?, ACM SIGPLAN Notices, v.7 n.2, p.19-26, February 1972 [doi>10.1145/953363.953366]
• 14: M. H. Halstead, An experimental determination of the "purity" of a trivial algorithm, ACM SIGMETRICS Performance Evaluation Review, v.2 n.1, p.10-15, March 1973 [doi>10.1145/1041606.1041608]
• 15: Halstead, M.: "Software Physics: Basic Principles", RJ 1582, IBM, Yorktown Heights, N. Y., (1975).
• 16: Halstead, M.: "Using the Methodology of Natural Science to Understand Software", Computer Science Department Internal Report CSD-TR 190, Purdue University, (1976).
• 17: Maurice H. Halstead, Elements of Software Science (Operating and programming systems series), Elsevier Science Inc., New York, NY, 1977
• 18: Hanushek, Eric A. and John E. Jackson: <u>Statistical Methods for Social Scientists</u>, Academic Press, New York, (1977).
• 19: L. T. Love , A. B. Bowman, An independent test of the theory of software physics, ACM SIGPLAN Notices, v.11 n.11, November 1976 [doi>10.1145/987335.987340]
• 20: Ottenstein, K.: "A Program to Count Operators and Operands for ANSI-FORTRAN Modules", Computer Science Department Internal Report CSD-TR 196, Purdue University, (June 1976).
• 21: K. J. Ottenstein, An algorithmic approach to the detection and prevention of plagiarism, ACM SIGCSE Bulletin, v.8 n.4, p.30-41, Dec. 1976 [doi>10.1145/382222.382462]
• 22: Shaw, M., A. Jones, P. Kneuven, J. McDermott, P. Miller and D. Notkin: "Cheating Policy in a Computer Science Department", <u>SIGSCE Bulletin</u>, Vol. 12, No. 2, (July 1980), pp. 72--76.