Rapper: a wrapper generator with linguistic knowledge

ABSTRACT

Database management systems are becoming available for semistructured data, however, these tools cannot be used on many real-world data sources (e.g., most web sites) in their native form. Often, wrappers are needed to extract information and organize it into a graph structure that makes explicit the concepts users want to query and update. This paper presents a new approach to wrapper generation that exploits linguistic knowledge. The approach produces a more fine-grained parse of sources with natural language text than previous efforts. The resulting graph structured databases answer queries that could not be formulated in database produced by prior generated wrappers. In addition, our approach may be more robust in the face of slight variations in word choice and order. We discuss a prototype implementation, lessons learned to date, evaluation issues, and future research directions.

REFERENCES

• 1: Brad Adelberg, NoDoSE—a tool for semi-automatically extracting structured and semistructured data from text documents, Proceedings of the 1998 ACM SIGMOD international conference on Management of data, p.283-294, June 01-04, 1998, Seattle, Washington, United States
• 2: Naveen Ashish , Craig A. Knoblock, Wrapper generation for semi-structured Internet sources, ACM SIGMOD Record, v.26 n.4, p.8-15, Dec. 1997 [doi>10.1145/271074.271078]
• 3: Peter Buneman , Susan Davidson , Gerd Hillebrand , Dan Suciu, A query language and optimization techniques for unstructured data, Proceedings of the 1996 ACM SIGMOD international conference on Management of data, p.505-516, June 04-06, 1996, Montreal, Quebec, Canada
• 4: CIA World Factbook, http://www.odci, gov/ci~publications/factbook/i ndex. html
• 5: Mary Fernández , Daniela Florescu , Jaewoo Kang , Alon Levy , Dan Suciu, Catching the boat with Strudel: experiences with a Web-site management system, Proceedings of the 1998 ACM SIGMOD international conference on Management of data, p.414-425, June 01-04, 1998, Seattle, Washington, United States
• 6: J. Hammer , H. Gaxcia-Molina , I. Cho , R. Axanha, A. Crespo, "Extracting Semistructured Information from the Web," SIGMOD Record, 26(4), December 1997
• 7: Ling Liu , Wei Han , David Buttler , Calton Pu , Wei Tang, An XJML-based wrapper generator for Web information extraction, Proceedings of the 1999 ACM SIGMOD international conference on Management of data, p.540-543, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States
• 8: Jason McHugh , Serge Abiteboul , Roy Goldman , Dallas Quass , Jennifer Widom, Lore: a database management system for semistructured data, ACM SIGMOD Record, v.26 n.3, p.54-66, Sept. 1997 [doi>10.1145/262762.262770]
• 9: Yannis Papakonstantinou , Hector Garcia-Molina , Jennifer Widom, Object Exchange Across Heterogeneous Information Sources, Proceedings of the Eleventh International Conference on Data Engineering, p.251-260, March 06-10, 1995
• 10: Dan Suciu, Management of semistructured data, ACM SIGMOD Record, v.26 n.4, p.4-7, Dec. 1997 [doi>10.1145/271074.271077]
• 11: USNI Military Database, .http:/(www.periscope. usni.com/demo/demoinfo.html
• 12: Marc Vilain , David Day, Finite-state phrase parsing by rule sequences, Proceedings of the 16th conference on Computational linguistics, August 05-09, 1996, Copenhagen, Denmark [doi>10.3115/992628.992677]
• 13: World Wide Web Consortium, XML Home Page, http:www, w3.org/XML

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE

General Terms:
Experimentation, Management, Theory

Keywords:
discovering structure, information extraction, semistructured data, web databases, wrapper generation