HDSampler: revealing data behind web form interfaces

ABSTRACT

A large number of online databases are hidden behind the web. Users to these systems can form queries through web forms to retrieve a small sample of the database. Sampling such hidden databases is widely desired for understanding the nature and quality of data stored in them. We have developed HDSampler, which to the best of our knowledge is the first practical system for sampling structured hidden web databases. It enables efficient sampling of the databases and accurate answering of aggregate queries, to provide analysts with valuable information for data analytics, as well as help power a multitude of third-party applications such as web-mashups and meta-search engines. For the purpose of this demo, we present an instance of HDSampler on Google Base - a content-rich hidden web database maintained by Google. By using HDSampler, the demo reveals a snapshot of the marginal distribution of various attributes of Google Base in a matter of minutes.

REFERENCES

• 1: Arjun Dasgupta , Gautam Das , Heikki Mannila, A random walk approach to sampling hidden databases, Proceedings of the 2007 ACM SIGMOD international conference on Management of data, June 11-14, 2007, Beijing, China [doi>10.1145/1247480.1247550]
• 2: Arjun Dasgupta , Nan Zhang , Gautam Das, Leveraging COUNT Information in Sampling Hidden Databases, Proceedings of the 2009 IEEE International Conference on Data Engineering, p.329-340, March 29-April 02, 2009 [doi>10.1109/ICDE.2009.112]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications

General Terms:
Algorithms, Design, Measurement, Performance

Keywords:
hidden databases, sampling, top-k interfaces