On designing a market monitoring web agent system

ABSTRACT

World-Wide-Web is a huge pool of valuable information for companies to know what their competitors are doing and what products and services they offer up-to-date. Companies can gather business intelligence from the Web for planning countermeasures strategies. Hence it is crucial to have the right tool to effectively gather such information from the Web. Many information retrieval and monitoring technologies have been developed. But they are more for generally tracking changes and downloading the whole websites for offline browsing. This paper is to shed some light on specifically the design of a Web monitoring system for gathering business information relevant to a company. The Watcher Agent is a server-based system that is built with two main parts, namely Price Watcher and Market Watcher. The system will assist company users in price information collection, news information filtering, and product ranking estimation, thus saving time and effort for them.

REFERENCES

• 1: Ling Liu , Calton Pu , Wei Tang, Continual Queries for Internet Scale Event-Driven Information Delivery, IEEE Transactions on Knowledge and Data Engineering, v.11 n.4, p.610-628, July 1999 [doi>10.1109/69.790816]
• 2: Ling Liu , Calton Pu , Wei Tang, WebCQ-detecting and delivering information changes on the web, Proceedings of the ninth international conference on Information and knowledge management, p.512-519, November 06-11, 2000, McLean, Virginia, United States [doi>10.1145/354756.354860]
• 3: L. Liu, C. Pu, W. Tang, and W. Han, 1999. "CONQUER: A continual query system for update monitoring in the WWW", International Journal of Computer Systems, Science and Engineering.
• 4: J. Naughton, D. DeWitt, D. Maier, A. Aboulnaga, J. Chen, L. Galanis, J. Kang, R. Krishnamurthy, Q. Luo, N. Prakash, R. Ramamurthy, J. Shanmugasundaram, F. Tian, K. Tufte, E. Viglas, Y. Wang, C. Zhang, B. Jackson, A. Gupta, and R. Chen, 2001. "The Niagara internet query system", IEEE Data Engineering Bulletin, 24(2):27--33.
• 5: Sandeep Pandey , Kedar Dhamdhere , Christopher Olston, WIC: a general-purpose algorithm for monitoring web information sources, Proceedings of the Thirtieth international conference on Very large data bases, p.360-371, August 31-September 03, 2004, Toronto, Canada
• 6: S. Fong, A. Sun, and K.-K. Wong, 2001. "Price Watcher Agent for E-Commerce", in Proc. of the 2nd Asia-Pacific Conf. on Intelligent Agent Technology (IAT-2001), pp. 294--299, Maebashi City, Japan, Oct.
• 7: Seung-Jin Lim , Yiu-Kai Ng, An automated approach for retrieving hierarchical data from HTML tables, Proceedings of the eighth international conference on Information and knowledge management, p.466-474, November 02-06, 1999, Kansas City, Missouri, United States [doi>10.1145/319950.320052]
• 8: Ricardo A. Baeza-Yates , Berthier Ribeiro-Neto, Modern Information Retrieval, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 9: Sudarshan S. Chawathe , Anand Rajaraman , Hector Garcia-Molina , Jennifer Widom, Change detection in hierarchically structured information, Proceedings of the 1996 ACM SIGMOD international conference on Management of data, p.493-504, June 04-06, 1996, Montreal, Quebec, Canada
• 10: Sudarshan S. Chawathe , Serge Abiteboul , Jennifer Widom, Representing and Querying Changes in Semistructured Data, Proceedings of the Fourteenth International Conference on Data Engineering, p.4-13, February 23-27, 1998
• 11: Fred Douglis , Thomas Ball , Yih-farn Chen , Eleftherios Koutsofios, The AT&T Internet Difference Engine: Tracking and viewing changes on the web, World Wide Web, v.1 n.1, p.27-44, 1998 [doi>10.1023/A:1019243126596]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.5 On-line Information Services

General Terms:
Algorithms, Design, Economics, Experimentation, Management

Keywords:
competitor-intelligence, information collecting system, web extraction, web-intelligence