Towards a mutation-based automatic framework for evaluating code clone detection tools

ABSTRACT

In the last decade, a great many code clone detection tools have been proposed. Such a large number of tools calls for a quantitative comparison, and there have been several attempts to empirically evaluate and compare many of the state-of-the-art tools. However, a recent study shows that there are several factors that could influence the the validity of the results of such comparisons. In order to overcome the effects of such factors (at least in part), in this student poster paper we outline a mutation-based controlled frame-work for evaluating clone detection tools using edit-based mutation operators that model cloning actions. While the framework is not yet completely implemented and as yet we do not have experimental data, we anticipate that such a framework will provide a useful contribution to the community by providing a more solid objective foundation for tool evaluation.

REFERENCES

• 1: J. H. Andrews , L. C. Briand , Y. Labiche, Is mutation an appropriate tool for testing experiments?, Proceedings of the 27th international conference on Software engineering, May 15-21, 2005, St. Louis, MO, USA [doi>10.1145/1062455.1062530]
• 2: Brenda S. Baker, Finding Clones with Dup: Analysis of an Experiment, IEEE Transactions on Software Engineering, v.33 n.9, p.608-621, September 2007 [doi>10.1109/TSE.2007.70720]
• 3: B. S. Baker, On finding duplication and near-duplication in large software systems, Proceedings of the Second Working Conference on Reverse Engineering, p.86, July 14-16, 1995
• 4: Stefan Bellon , Rainer Koschke , Giulio Antoniol , Jens Krinke , Ettore Merlo, Comparison and Evaluation of Clone Detection Tools, IEEE Transactions on Software Engineering, v.33 n.9, p.577-591, September 2007 [doi>10.1109/TSE.2007.70725]
• 5: Magiel Bruntink , Arie van Deursen , Remco van Engelen , Tom Tourwe, On the Use of Clone Detection for Identifying Crosscutting Concern Code, IEEE Transactions on Software Engineering, v.31 n.10, p.804-818, October 2005 [doi>10.1109/TSE.2005.114]
• 6: Elizabeth Burd , John Bailey, Evaluating Clone Detection Tools for Use during Preventative Maintenance, Proceedings of the Second IEEE International Workshop on Source Code Analysis and Manipulation (SCAM'02), p.36, October 01-01, 2002
• 7: James R. Cordy, The TXL source transformation language, Science of Computer Programming, v.61 n.3, p.190-210, August 2006 [doi>10.1016/j.scico.2006.04.002]
• 8: J. Howard Johnson, Visualizing textual redundancy in legacy source, Proceedings of the 1994 conference of the Centre for Advanced Studies on Collaborative research, p.32, October 31-November 03, 1994, Toronto, Ontario, Canada
• 9: Cory Kapser , Michael W. Godfrey, "Cloning Considered Harmful" Considered Harmful, Proceedings of the 13th Working Conference on Reverse Engineering, p.19-28, October 23-27, 2006 [doi>10.1109/WCRE.2006.1]
• 10: Miryung Kim , Vibha Sazawal , David Notkin , Gail Murphy, An empirical study of code clone genealogies, Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, September 05-09, 2005, Lisbon, Portugal
• 11: Rainer Koschke , Raimar Falke , Pierre Frenzel, Clone Detection Using Abstract Syntax Suffix Trees, Proceedings of the 13th Working Conference on Reverse Engineering, p.253-262, October 23-27, 2006 [doi>10.1109/WCRE.2006.18]
• 12: Zhenmin Li , Shan Lu , Suvda Myagmar , Yuanyuan Zhou, CP-Miner: Finding Copy-Paste and Related Bugs in Large-Scale Software Code, IEEE Transactions on Software Engineering, v.32 n.3, p.176-192, March 2006 [doi>10.1109/TSE.2006.28]
• 13: C. K. Roy and J. R. Cordy. A Survey on Software Clone Detection Research. School of Computing TR 2007-541, Queen's University, 115 pp., 2007.
• 14: C. K. Roy and J. R. Cordy. Scenario-Based Comparison of Clone Detection Techniques. In ICPC, 10 pp., 2008 (to appear).
• 15: C. K. Roy and J. R. Cordy. NICAD: Accurate Detection of Near-Miss Intentional Clones Using Flexible Pretty-Printing and Code Normalization. In ICPC, 10 pp., 2008 (to appear).
• 16: F. V. Rysselberghe and S. Demeyer. Evaluating Clone Detection Techniques. In ELISA, 12pp., 2003.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.7 Distribution, Maintenance, and Enhancement

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging

General Terms:
Experimentation, Measurement

Keywords:
clone detection techniques, evaluation, framework, maintenance, mutation analysis, software engineering