Building a scalable and accurate copy detection mechanism

ABSTRACT

Often, publishers are reluctant to offer valuable digital documents on the Internet for fear that they will be re-transmitted or copied widely. A Copy Detection Mechanism can help identify such copying. For example, publishers may register their documents with a copy detection server, and the server can then automatically check public sources such as UseNet articles and Web sites for potential illegal copies. The server can search for exact copies, and also for cases where significant portions of documents have been copied. In this paper we study, for the first time, the performance of various copy detection mechanisms, including the disk storage requirements, main memory requirements, response times for registration, and response time for querying. We also contrast performance to the accuracy of the mechanisms (how well they detect partial copies). The results are obtained using SCAM, an experimental server we have implemented, and a collection of 50,000 netnews articles.

REFERENCES

• 1: J. Brassil, S. Low, N. Maxemchuk, and L.O'Gorman. Document marking and identification using both line and word shifting. Technical report, AT&T Bell Labratories, 1994. May be obtained from ftp://ftp.research.att.com/dist/brassil/docmarkZ.ps.
• 2: J. Brassil, S. Low, N. Maxemchuk, and L.O'Gorman. Electronic marking and identification techniques to discourage document copying. Technical report, AT~T Bell Labratories, 1994.
• 3: Sergey Brin , James Davis , Héctor García-Molina, Copy detection mechanisms for digital documents, Proceedings of the 1995 ACM SIGMOD international conference on Management of data, p.398-409, May 22-25, 1995, San Jose, California, United States
• 4: A. Choudhury, N. Maxemchuk, S. Paul, and H. Schulzrinne. Copyright protection for electronic publishing over computer networks. Technical report, AT&T Bell Labratories, 1994. Submitted to IEEE Network Magazine June 1994.
• 5: Peter J. Denning, Plagiarism in the web, Communications of the ACM, v.38 n.12, p.29, Dec. 1995 [doi>10.1145/219663.219675]
• 6: Gaston H. Gonnet , Ricardo A. Baeza-Yates , Tim Snider, New indices for text: PAT Trees and PAT arrays, Information retrieval: data structures and algorithms, Prentice-Hall, Inc., Upper Saddle River, NJ, 1992
• 7: G. N. Griswold. A method for protecting copyright on networks. In Joint Harvard MIT Workshop on Technology Strategies .for Protecting Intellectual Property in the Networked Multimedia Environment, April 1993.
• 8: Udi Manber , Gene Myers, Suffix arrays: a new method for on-line string searches, Proceedings of the first annual ACM-SIAM symposium on Discrete algorithms, p.319-327, January 22-24, 1990, San Francisco, California, United States
• 9: U. Manber and S. Wu. Glimpse: A tool to search through entire file systems. In Proceedings of the winter USENIX Conference, January 1994.
• 10: David D. Palmer , Marti A. Hearst, Adaptive sentence boundary disambiguation, Proceedings of the fourth conference on Applied natural language processing, October 13-15, 1994, Stuttgart, Germany [doi>10.3115/974358.974376]
• 11: Gerald J. Popek , Charles S. Kline, Encryption and Secure Computer Networks, ACM Computing Surveys (CSUR), v.11 n.4, p.331-356, Dec. 1979 [doi>10.1145/356789.356794]
• 12: Gerard Salton , Christopher Buckley, Term-weighting approaches in automatic text retrieval, Information Processing and Management: an International Journal, v.24 n.5, p.513-523, 1988 [doi>10.1016/0306-4573(88)90021-0]
• 13: Gerard Salton, The state of retrieval system evaluation, Information Processing and Management: an International Journal, v.28 n.4, p.441-449, July-Aug. 1992 [doi>10.1016/0306-4573(92)90002-H]
• 14: N. Shivakumar and H. Garcia-Molina. SCAM: A copy detection mechanism for digital documents. In Proceedings of 2rid International Conference in Theory and Practice o} Digital Libraries (DL'95), Austin, Texas, June 1995.
• 15: D. Wheeler. Computer networks are said to offer new opportunities for plagiarists. The Chronicle of Higher Education, pages t7, 19, June 1993.
• 16: T. Yah and H. Garcia-Molina. Duplicate detection in information dissemination. In Proceedings of Very Large Databases (VLDB'95) Conference, Zurich, Switzerland, September 1995.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.0 General
          Subjects: Security and protection (e.g., firewalls)

  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

  J. Computer Applications
  J.7 COMPUTERS IN OTHER SYSTEMS
      Subjects: Publishing

General Terms:
Algorithms, Design, Performance, Security