Learning query languages of Web interfaces

ABSTRACT

This paper studies the problem of automatic acquisition of the query languages supported by a Web information resource. We describe a system that automatically probes the search interface of a resource with a set of test queries and analyses the returned pages to recognize supported query operators. The automatic acquisition assumes the availability of the number of matches the resource returns for a submitted query. The match numbers are used to train a learning system and to generate classification rules that recognize the query operators supported by a provider and their syntactic encodings. These classification rules are employed during the automatic probing of new providers to determine query operators they support. We report on results of experiments with a set of real Web resources.

REFERENCES

• 1: The Invisible Web, http://www.invisibleweb.com/.
• 2: BrightPlanet, http://www.brightplanet.com/.
• 3: CompletePlanet, http://www.completeplanet.com/.
• 4: G. Alonso. Myths around web services. IEEE Bulletin on Data Engineering, 25(4):3--9, 2002.
• 5: Dana Angluin, Queries and Concept Learning, Machine Learning, v.2 n.4, p.319-342, April 1988 [doi>10.1023/A:1022821128753]
• 6: M. K. Bergman. The Deep Web: Surfacing hidden value. Journal of Electronic Publishing, 7(1), 2001.
• 7: D. Bredelet and B. Roustant. Java IWrap: Wrapper induction by grammar learning. Master's thesis, ENSIMAG Grenoble, 2000.
• 8: S. Byers, J. Freire, and C. T. Silva. Efficient acquisition of web data through restricted query interfaces. In Proc. WWW Conf., China, May 2001.
• 9: Jamie Callan , Margaret Connell , Aiqun Du, Automatic discovery of language models for text databases, Proceedings of the 1999 ACM SIGMOD international conference on Management of data, p.479-490, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States
• 10: Kevin Chen-Chuan Chang , Hector Garcia-Molina, Approximate Query Translation Across Heterogeneous Information Sources, Proceedings of the 26th International Conference on Very Large Data Bases, p.566-577, September 10-14, 2000
• 11: Kevin Chen-Chuan Chang , Hector Garcia-Molina , Andreas Paepcke, Boolean Query Mapping Across Heterogeneous Information Sources, IEEE Transactions on Knowledge and Data Engineering, v.8 n.4, p.515-521, August 1996 [doi>10.1109/69.536244]
• 12: Boris Chidlovskii, Automatic Repairing of Web Wrappers by Combining Redundant Views, Proceedings of the 14th IEEE International Conference on Tools with Artificial Intelligence (ICTAI'02), p.399, November 04-06, 2002
• 13: Luis Gravano , Héctor García-Molina , Anthony Tomasic, GlOSS: text-source discovery over the Internet, ACM Transactions on Database Systems (TODS), v.24 n.2, p.229-264, June 1999 [doi>10.1145/320248.320252]
• 14: P. G. Ipeirotis and L. Gravano. Distributed search over the hidden web: Hierarchical database sampling and selection. In Proc. VLDB Conf., pp. 394--405, Hong Kong, China, August 2002.
• 15: Panagiotis G. Ipeirotis , Luis Gravano , Mehran Sahami, Probe, count, and classify: categorizing hidden web databases, Proceedings of the 2001 ACM SIGMOD international conference on Management of data, p.67-78, May 21-24, 2001, Santa Barbara, California, United States
• 16: Mike Perkowitz , Robert B. Doorenbos , Oren Etzioni , Daniel S. Weld, Learning to Understand Information on the Internet: AnExample-Based Approach, Journal of Intelligent Information Systems, v.8 n.2, p.133-153, March/April 1997 [doi>10.1023/A:1008672508721]
• 17: Sriram Raghavan , Hector Garcia-Molina, Crawling the Hidden Web, Proceedings of the 27th International Conference on Very Large Data Bases, p.129-138, September 11-14, 2001
• 18: Shalom Tsur , Serge Abiteboul , Rakesh Agrawal , Umeshwar Dayal , Johannes Klein , Gerhard Weikum, Are Web Services the Next Revolution in e-Commerce? (Panel), Proceedings of the 27th International Conference on Very Large Data Bases, p.614-617, September 11-14, 2001
• 19: Wenxian Wang , Weiyi Meng , Clement Yu, Concept Hierarchy Based Text Database Categorization in a Metasearch Engine Environment, Proceedings of the First International Conference on Web Information Systems Engineering (WISE'00)-Volume 1, p.283, June 19-20, 2000
• 20: Ramana Yerneni , Chen Li , Hector Garcia-Molina , Jeffrey Ullman, Computing capabilities of mediators, Proceedings of the 1999 ACM SIGMOD international conference on Management of data, p.443-454, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States

INDEX TERMS

Keywords:
hidden Web, learning, query operators, search interface