Evaluation of source code copy detection methods on freebsd

ABSTRACT

Studies have shown that substantial code reuse is common in open source and in commercial projects. However, the precise extent of reuse and its impact on productivity and quality are not well investigated in the open source context. Previously, we have introduced a simple-to-use method that needs only a set of file pathnames to identifies directories that share filenames and partially validated its performance on a set of closed-source projects. To evaluate this method and to improve reuse detection at the file level, we apply it and four additional file copy detection methods that utilize the underlying content of multiple versions of the source code on the FreeBSD project. The evaluation quantified unique advantages of each method and showed that the filename method detected roughly half of all reuse cases. We are still faced with a challenge to scale the content based methods to large repositories containing all versions of open source files.

REFERENCES

• 1: B. S. Baker, On finding duplication and near-duplication in large software systems, Proceedings of the Second Working Conference on Reverse Engineering, p.86, July 14-16, 1995
• 2: Bruno Lague , Daniel Proulx , Jean Mayrand , Ettore M. Merlo , John Hudepohl, Assessing the Benefits of Incorporating Function Clone Detection in a Development Process, Proceedings of the International Conference on Software Maintenance, p.314, October 01-03, 1997
• 3: Akito Monden , Daikai Nakae , Toshihiro Kamiya , Shin-ichi Sato , Ken-ichi Matsumoto, Software Quality Analysis by Code Clones in Industrial Legacy Software, Proceedings of the 8th International Symposium on Software Metrics, p.87, June 04-07, 2002
• 4: Stefan Haefliger, Georg von Krogh and Sebastian Spaeth. Code reuse in open source software. Management Science, Articles in Advance, pp. 1--14.
• 5: Hung-Fu Chang , Audris Mockus, Constructing universal version history, Proceedings of the 2006 international workshop on Mining software repositories, May 22-23, 2006, Shanghai, China [doi>10.1145/1137983.1138002]
• 6: E. Damiani, S. De Capitani di Vimercati, S. Paraboschi, P. Samarati. An Open Digest-based Technique for Spam Detection. ACM, vol. 41, no. 8, pp. 74--83. The 2004 International Workshop on Security in Parallel and Distributed Systems.
• 7: Michael W. Berry , Murray Browne, Understanding search engines: mathematical modeling and text retrieval, Society for Industrial and Applied Mathematics, Philadelphia, PA, 1999
• 8: Ira D. Baxter , Andrew Yahin , Leonardo Moura , Marcelo Sant'Anna , Lorraine Bier, Clone Detection Using Abstract Syntax Trees, Proceedings of the International Conference on Software Maintenance, p.368, March 16-19, 1998
• 9: Stéphane Ducasse , Matthias Rieger , Serge Demeyer, A Language Independent Approach for Detecting Duplicated Code, Proceedings of the IEEE International Conference on Software Maintenance, p.109, August 30-September 03, 1999
• 10: Toshihiro Kamiya , Shinji Kusumoto , Katsuro Inoue, CCFinder: a multilinguistic token-based code clone detection system for large scale source code, IEEE Transactions on Software Engineering, v.28 n.7, p.654-670, July 2002 [doi>10.1109/TSE.2002.1019480]
• 11: Audris Mockus, Large-Scale Code Reuse in Open Source Software, Proceedings of the First International Workshop on Emerging Trends in FLOSS Research and Development, p.7, May 20-26, 2007 [doi>10.1109/FLOSS.2007.10]
• 12: Daniel M. German, Using software distributions to understand the relationship among free and open source software projects, Proceedings of the 29th International Conference on Software Engineering Workshops, p.24, May 20-26, 2007 [doi>10.1109/ICSEW.2007.229]
• 13: Cory Kapser , Michael W. Godfrey, Improved Tool Support for the Investigation of Duplication in Software, Proceedings of the 21st IEEE International Conference on Software Maintenance, p.305-314, September 25-30, 2005 [doi>10.1109/ICSM.2005.52]
• 14: Stefan Bellon , Rainer Koschke , Giulio Antoniol , Jens Krinke , Ettore Merlo, Comparison and Evaluation of Clone Detection Tools, IEEE Transactions on Software Engineering, v.33 n.9, p.577-591, September 2007 [doi>10.1109/TSE.2007.70725]
• 15: Michael W. Godfrey , Lijie Zou, Using Origin Analysis to Detect Merging and Splitting of Source Code Entities, IEEE Transactions on Software Engineering, v.31 n.2, p.166-181, February 2005 [doi>10.1109/TSE.2005.28]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.7 Distribution, Maintenance, and Enhancement
          Subjects: Restructuring, reverse engineering, and reengineering

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.7 Distribution, Maintenance, and Enhancement
          Subjects: Version control

General Terms:
Algorithms, Measurement

Keywords:
clone detection, cloning, code copying, open source, version control