Implementation and evaluation of a quality-based search engine

ABSTRACT

In this paper, an approach for the implementation of a quality-based Web search engine is proposed. Quality retrieval is introduced and an overview on previous efforts to implement such a service is given. Machine learning approaches are identified as the most promising methods to determine the quality of Web pages. Features for the most appropriate characterization of Web pages are determined. A quality model is developed based on human judgments. This model is integrated into a meta search engine which assesses the quality of all results at run time. The evaluation results show that quality based ranking does lead to better results concerning the perceived quality of Web pages presented in the result set. The quality models are exploited to identify potentially important features and characteristics for the quality of Web pages.

REFERENCES

• 1: Lada A. Adamic , Bernardo A. Huberman, The Web's hidden order, Communications of the ACM, v.44 n.9, p.55-60, Sept. 2001 [doi>10.1145/383694.383707]
• 2: Brian Amento , Loren Terveen , Will Hill, Does “authority” mean quality? predicting expert quality ratings of Web documents, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.296-303, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345603]
• 3: Barabási, A.-L. Linked: The New Science of Networks. Perseus, 2002
• 4: Beck, S. Evaluation Criteria: The Good, The Bad & The Ugly: or, Why It's a Good Idea to Evaluate Web Sources. (1997) http://lib.nmsu.edu/instruction/evalcrit.html
• 5: Allan Borodin , Gareth O. Roberts , Jeffrey S. Rosenthal , Panayiotis Tsaparas, Link analysis ranking: algorithms, theory, and experiments, ACM Transactions on Internet Technology (TOIT), v.5 n.1, p.231-297, February 2005 [doi>10.1145/1052934.1052942]
• 6: Rodrigo A. Botafogo , Ehud Rivlin , Ben Shneiderman, Structural analysis of hypertexts: identifying hierarchies and useful metrics, ACM Transactions on Information Systems (TOIS), v.10 n.2, p.142-180, April 1992 [doi>10.1145/146802.146826]
• 7: Anna Bouch , Allan Kuchinsky , Nina Bhatti, Quality is in the eye of the beholder: meeting users' requirements for Internet quality of service, Proceedings of the SIGCHI conference on Human factors in computing systems, p.297-304, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332447]
• 8: Neil Bowers, Weblint: quality assurance for the World Wide Web, Proceedings of the fifth international World Wide Web conference on Computer networks and ISDN systems, p.1283-1290, May 1996, Paris, France
• 9: Sergey Brin , Lawrence Page, The anatomy of a large-scale hypertextual Web search engine, Proceedings of the seventh international conference on World Wide Web 7, p.107-117, April 1998, Brisbane, Australia
• 10: Erik P. Bucy , Annie Lang , Robert F. Potter , Maria Elizabeth Grabe, Formal features of cyberspace: relationships between Web page complexity and site traffic, Journal of the American Society for Information Science, v.50 n.13, p.1246-1256, Nov. 1999 [doi>10.1002/(SICI)1097-4571(1999)50:13<1246::AID-ASI10>3.0.CO;2-E]
• 11: Soumen Chakrabarti , Mukul M. Joshi , Kunal Punera , David M. Pennock, The structure of broad topics on the web, Proceedings of the 11th international conference on World Wide Web, May 07-11, 2002, Honolulu, Hawaii, USA [doi>10.1145/511446.511480]
• 12: Liang Chen , Shaozhi Ye , Xing Li, Template detection for large scale search engines, Proceedings of the 2006 ACM symposium on Applied computing, April 23-27, 2006, Dijon, France [doi>10.1145/1141277.1141534]
• 13: Ed H. Chi , Adam Rosien , Gesara Supattanasiri , Amanda Williams , Christiaan Royer , Celia Chow , Erica Robles , Brinda Dalal , Julie Chen , Steve Cousins, The bloodhound project: automating discovery of web usability issues using the InfoScentπ simulator, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642699]
• 14: Chun Wei Choo , Brian Detlor , Don Turnbull, Web Work: Information Seeking and Knowledge Work on the World Wide Web, Kluwer Academic Publishers, Norwell, MA, 2000
• 15: De la Cruz, T., Mandl, T. and Womser-Hacker, C. Cultural Dependency of Quality Perception and Web Page Evaluation Guidelines: Results from a Survey. In Designing for Global Markets 7: Proc. Seventh International Workshop on Internationalization of Products and Systems (IWIPS 2005) Amsterdam, The Netherlands. 15--27
• 16: Devanshu Dhyani , Wee Keong Ng , Sourav S. Bhowmick, A survey of Web metrics, ACM Computing Surveys (CSUR), v.34 n.4, p.469-503, December 2002 [doi>10.1145/592642.592645]
• 17: Stephen Dill , Ravi Kumar , Kevin S. McCurley , Sridhar Rajagopalan , D. Sivakumar , Andrew Tomkins, Self-similarity in the Web, Proceedings of the 27th International Conference on Very Large Data Bases, p.69-78, September 11-14, 2001
• 18: Dennis Fetterly , Mark Manasse , Marc Najork , Janet Wiener, A large-scale study of the evolution of web pages, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775246]
• 19: B. J. Fogg , Jonathan Marshall , Othman Laraki , Alex Osipovich , Chris Varma , Nicholas Fang , Jyoti Paul , Akshay Rangnekar , John Shon , Preeti Swani , Marissa Treinen, What makes Web sites credible?: a report on a large quantitative study, Proceedings of the SIGCHI conference on Human factors in computing systems, p.61-68, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365037]
• 20: Foltz, P.W., Klintsch, W. and Landauer, T.K. The Measurement of Textual Coherence with Latent Semantic Analysis. In Discourse Processes, vol. 25 (2&3) (1998) 285--307.
• 21: Henzinger, M. Link Analysis in Web Information Retrieval. In Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, vol. 23 (3) (2000) 3--8.
• 22: Monika R. Henzinger , Rajeev Motwani , Craig Silverstein, Challenges in web search engines, ACM SIGIR Forum, v.36 n.2, Fall 2002 [doi>10.1145/792550.792553]
• 23: Kuan-Tse Huang , Yang W. Lee , Richard Y. Wang, Quality information and knowledge, Prentice Hall PTR, Upper Saddle River, NJ, 1998
• 24: Melody Y. Ivory , Marti A. Hearst, Statistical profiles of highly-rated web sites, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503442]
• 25: Jensen, N., Hackl, R., Mandl, T. and Strötgen, R. Web Retrieval Experiments with the EuroGOV Corpus at the University of Hildesheim. In Accessing Multilingual Information Repositories: 6th Workshop of the Cross-Language Evaluation Forum, CLEF 2005, Vienna, Austria, Revised Selected Papers. Springer {LNCS 4022} (2006)
• 26: Klas, C.-P. Fuhr, N. A new effective approach for categorizing web documents. In Proceedings of the 22nd BCS-IRSG Colloquium on IR Research (2000)
• 27: Langville, A. and Meyer, C. Deeper Inside PageRank. In Internet Mathematics 1 (3) (2003) 335--380.
• 28: Mandl, T. Web Link Behavior and Consequences for Connectivity Based Authority Measures. In The Twelfth International World Wide Web Conference. 20-24 May 2003, Budapest. http://www2003.org/cdrom/papers/poster/p204/p204-mandl.html
• 29: Mandl, T. Automatische Bewertung der Qualität von Web-Seiten im Information Retrieval. Konstanz: Universitätsverlag (2006) to appear.
• 30: E. Miltsakaki , K. Kukich, Evaluation of text coherence for electronic essay scoring systems, Natural Language Engineering, v.10 n.1, p.25-55, March 2004 [doi>10.1017/S1351324903003206]
• 31: B. Uygar Oztekin , George Karypis , Vipin Kumar, Expert agreement and content based reranking in a meta search environment using Mearf, Proceedings of the 11th international conference on World Wide Web, May 07-11, 2002, Honolulu, Hawaii, USA [doi>10.1145/511446.511490]
• 32: Pennock, D., Flake, G., Lawrence, S., Glover, E. and Giles, L. Winners don't take all: Characterizing the competition for links on the web. In Proc. National Academy of Sciences 99 (8) 2002. 5207--5211
• 33: Thelwall, M. The top 100 linked-to pages on UK university web sites: high in-link counts are not usually associated with quality scholarly content. In Journal of Information Science 28 (6) 2002. 483--491.
• 34: Noam Tractinsky, Aesthetics and apparent usability: empirically assessing cultural and methodological issues, Proceedings of the SIGCHI conference on Human factors in computing systems, p.115-122, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258626]
• 35: Ellen M. Voorhees , Donna K. Harman, TREC: Experiment and Evaluation in Information Retrieval (Digital Libraries and Electronic Publishing), The MIT Press, 2005
• 36: Baoning Wu , Brian D. Davison, Undue influence: eliminating the impact of link plagiarism on web search rankings, Proceedings of the 2006 ACM symposium on Applied computing, April 23-27, 2006, Dijon, France [doi>10.1145/1141277.1141535]
• 37: Gui-Rong Xue , Hua-Jun Zeng , Zheng Chen , Wei-Ying Ma , Hong-Jiang Zhang , Chao-Jun Lu, Implicit link analysis for small web search, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860448]
• 38: Yang, K. Combining text- and link-based retrieval methods for Web IR. In Proc. the Ninth Text REtrieval Conf (TREC 9) (2000)
• 39: Xiaolan Zhu , Susan Gauch, Incorporating quality metrics in centralized/distributed information retrieval on the World Wide Web, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.288-295, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345602]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software
          Subjects: Performance evaluation (efficiency and effectiveness)
      H.3.5 On-line Information Services
          Subjects: Web-based services

General Terms:
Experimentation, Human Factors

Keywords:
quality models, quality search, web design, web metrics