Hourly analysis of a very large topically categorized web query log

ABSTRACT

We review a query log of hundreds of millions of queries that constitute the total query traffic for an entire week of a general-purpose commercial web search service. Previously, query logs have been studied from a single, cumulative view. In contrast, our analysis shows changes in popularity and uniqueness of topically categorized queries across the hours of the day. We examine query traffic on an hourly basis by matching it against lists of queries that have been topically pre-categorized by human editors. This represents 13% of the query traffic. We show that query traffic from particular topical categories differs both from the query stream as a whole and from other categories. This analysis provides valuable insight for improving retrieval effectiveness and efficiency. It is also relevant to the development of enhanced query disambiguation, routing, and caching algorithms.

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	Steven M. Beitzel , Eric C. Jensen , Abdur Chowdhury , David Grossman, Using titles and category names from editor-driven taxonomies for automatic evaluation, Proceedings of the twelfth international conference on Information and knowledge management, November 03-08, 2003, New Orleans, LA, USA [doi>10.1145/956863.956868]
	2	Andrei Broder, A taxonomy of web search, ACM SIGIR Forum, v.36 n.2, Fall 2002 [doi>10.1145/792550.792552]
	3	Abdur Chowdhury , Greg Pass, Operational requirements for scalable search systems, Proceedings of the twelfth international conference on Information and knowledge management, November 03-08, 2003, New Orleans, LA, USA [doi>10.1145/956863.956945]
	4	Caroline M. Eastman , Bernard J. Jansen, Coverage, relevance, and ranking: The impact of query operators on Web search engine results, ACM Transactions on Information Systems (TOIS), v.21 n.4, p.383-411, October 2003 [doi>10.1145/944012.944015]
	5	Nadav Eiron , Kevin S. McCurley, Analysis of anchor text for web search, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860550]
	6	Hawking, D., Craswell, N., and Griffiths, K. Which Search Engine is Best at Finding Online Services? In Proceedings of WWW10 (Hong Kong, May 2001), Posters. Actual poster available as http://pigfish.vic.cmis.csiro.au/ nickc/pubs/www10actualposter.pdf
	7	Bernard J. Jansen , Udo Pooch, A review of web searching studies and a framework for future research, Journal of the American Society for Information Science and Technology, v.52 n.3, p.235-246, Feb. 1, 2001 [doi>10.1002/1097-4571(2000)9999:9999<::AID-ASI1607>3.3.CO;2-6]
	8	Bernard J. Jansen , Amanda Spink , Tefko Saracevic, Real life, real users, and real needs: a study and analysis of user queries on the web, Information Processing and Management: an International Journal, v.36 n.2, p.207-227, Jan.1.2000 [doi>10.1016/S0306-4573(99)00056-4]
	9	Bernard J. Jansen , Abby Goodrum , Amanda Spink, Searching for multimedia: analysis of audio, video and image Web queries, World Wide Web, v.3 n.4, p.249-254, December 2000 [doi>10.1023/A:1018743024123]
	10	Lawrence, S. and Giles, C.L. Searching the World Wide Web. Science 280(5360), 98--100, 1998.
	11	Ronny Lempel , Shlomo Moran, Predictive caching and prefetching of query results in search engines, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775156]
	12	Markatos, E.P. On Caching Search Engine Query Results. In the Proceedings of the 5th International Web Caching and Content Delivery Workshop, May 2000.
	13	Vijay V. Raghavan , Hayri Sever, On the reuse of past optimal queries, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.344-350, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215381]
	14	Nancy C. M. Ross, End user searching on the Internet: an analysis of term pair topics submitted to the excite search engine, Journal of the American Society for Information Science, v.51 n.10, p.949-958, Aug. 2000 [doi>10.1002/1097-4571(2000)51:10<949::AID-ASI70>3.0.CO;2-5]
	15	Paricia Correia Saraiva , Edleno Silva de Moura , Novio Ziviani , Wagner Meira , Rodrigo Fonseca , Berthier Riberio-Neto, Rank-preserving two-level caching for scalable search engines, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.51-58, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383959]
	16	Craig Silverstein , Hannes Marais , Monika Henzinger , Michael Moricz, Analysis of a very large web search engine query log, ACM SIGIR Forum, v.33 n.1, p.6-12, Fall 1999 [doi>10.1145/331403.331405]
	17	Amanda Spink , Seda Ozmutlu , Huseyin C. Ozmutlu , Bernard J Jansen, U.S. versus European web searching trends, ACM SIGIR Forum, v.36 n.2, Fall 2002 [doi>10.1145/792550.792555]
	18	Amanda Spink , Bernard J. Jansen , Dietmar Wolfram , Tefko Saracevic, From E-Sex to E-Commerce: Web Search Changes, Computer, v.35 n.3, p.107-109, March 2002 [doi>10.1109/2.989940]
	19	Amanda Spink , Dietmar Wolfram , Major B. J. Jansen , Tefko Saracevic, Searching the Web: the public and their queries, Journal of the American Society for Information Science and Technology, v.52 n.3, p.226-234, Feb. 1, 2001 [doi>10.1002/1097-4571(2000)9999:9999<::AID-ASI1591>3.3.CO;2-I]
	20	Dietmar Wolfram , Amanda Spink , Bernard J. Jansen , Tefko Saracevic, Vox populi: the public searching of the Web, Journal of the American Society for Information Science and Technology, v.52 n.12, p.1073-1074, October 2001 [doi>10.1002/asi.1157]
	21	Spink, A., Jansen, B.J., and Ozmultu, H.C. Use of query reformulation and relevance feedback by Excite users. Internet Research: Electronic Networking Applications and Policy 10 (4), 2000.
	22	Sullivan, D. Searches Per Day. Search Engine Watch, February, 2003. http://searchenginewatch.com/reports/article.php/2156461
	23	Peiling Wang , Michael W. Berry , Yiheng Yang, Mining longitudinal web queries: trends and patterns, Journal of the American Society for Information Science and Technology, v.54 n.8, p.743-758, June 2003 [doi>10.1002/asi.10262]
	24	Query clustering using user logs, ACM Transactions on Information Systems (TOIS), v.20 n.1, p.59-81, January 2002 [doi>10.1145/503104.503108]
	25	Hsiao-Tieh Pu , Shui-Lung Chuang , Chyan Yang, Subject categorization of query terms for exploring Web users' search interests, Journal of the American Society for Information Science and Technology, v.53 n.8, p.617-630, August 2002 [doi>10.1002/asi.10071]
	26	Xie, Y., O'Hallaron, D. Locality in Search Engine Queries and Its Implications for Caching. Infocom 2002.

Collaborative Colleagues:

Steven M. Beitzel: colleagues

Eric C. Jensen: colleagues

Abdur Chowdhury: colleagues

David Grossman: colleagues

Ophir Frieder: colleagues