User-oriented document summarization through vision-based eye-tracking

ABSTRACT

We propose a new document summarization algorithm which is personalized. The key idea is to rely on the attention (reading) time of individual users spent on single words in a document as the essential clue. The prediction of user attention over every word in a document is based on the user's attention during his previous reads, which is acquired via a vision-based commodity eye-tracking mechanism. Once the user's attentions over a small collection of words are known, our algorithm can predict the user's attention over every word in the document through word semantics analysis. Our algorithm then summarizes the document according to user attention on every individual word in the document. With our algorithm, we have developed a document summarization prototype system. Experiment results produced by our algorithm are compared with the ones manually summarized by users as well as by commercial summarization software, which clearly demonstrates the advantages of our new algorithm for user-oriented document summarization.

REFERENCES

• 1: Bibliography summarization papers. http://www.summarization.com/summ.pdf, last updated on October 20, 2008. last visited on December 11, 2008.
• 2: Ricardo A. Baeza-Yates , Berthier Ribeiro-Neto, Modern Information Retrieval, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 3: A. Bulling, J. A.Ward, H. Gellersen, and G. Tröster. Robust recognition of reading activity in transit using wearable electrooculography. In Pervasive '08: Proceedings of the 6th International Conference on Pervasive Computing, pages 19--37, 2008.
• 4: Andreas Bulling , Daniel Roggen , Gerhard Tröster, It's in your eyes: towards context-awareness and mobile HCI using wearable EOG goggles, Proceedings of the 10th international conference on Ubiquitous computing, September 21-24, 2008, Seoul, Korea [doi>10.1145/1409635.1409647]
• 5: E. H. Chi, M. Gumbrecht, and L. Hong. Visual foraging of highlighted text: An eye-tracking study. In HCII '07: Proceedings of HCI International Conference, pages 589--598, 2007.
• 6: The New York Times Company. The New York Times, http://www.nytimes.com/, last visited on December 11, 2008.
• 7: T. Darrell, N. Checka, A. Oh, and L. Morency. Exploring vision-based interfaces: How to use your head in dual pointing tasks. MIT AI Memo 2002-001, 2002.
• 8: Zhicheng Dou , Ruihua Song , Ji-Rong Wen, A large-scale evaluation and analysis of personalized search strategies, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242651]
• 9: G. Dupret, V. Murdock, and B. Piwowarski. Web search engine evaluation using clickthrough data and a user model. In WWW '07: Proceedings of International Conference on World Wide Web, Banff, Canada, 2007.
• 10: G. Erkan and D. Radev. Lexrank: Graph-based lexical centrality as salience in text summarization. Journal of Artificial Intelligence Research (JAIR), 22:457--479, 2004.
• 11: American Association for the Advancement of Science. Science magazine, http://www.sciencemag.com/, last visited on December 11, 2008.
• 12: Steve Fox , Kuldeep Karnawat , Mark Mydland , Susan Dumais , Thomas White, Evaluating implicit measures to improve web search, ACM Transactions on Information Systems (TOIS), v.23 n.2, p.147-168, April 2005 [doi>10.1145/1059981.1059982]
• 13: Xin Fu, Evaluating sources of implicit feedback in web searches, Proceedings of the 2007 ACM conference on Recommender systems, October 19-20, 2007, Minneapolis, MN, USA [doi>10.1145/1297231.1297272]
• 14: Yihong Gong , Xin Liu, Generic text summarization using relevance measure and latent semantic analysis, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.19-25, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383955]
• 15: D. Gorodnichy. Perceptual cursor-a solution to the broken loop problem in vision-based hands-free computer control devices. National Research Council Canada Publication, NRC-48472:1--23, 2006.
• 16: Laura A. Granka , Thorsten Joachims , Geri Gay, Eye-tracking analysis of user behavior in WWW search, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009079]
• 17: Zhiwei Guan , Edward Cutrell, An eye tracking study of the effect of target rank on web search, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240691]
• 18: Wadee S. Al halabi , Miroslav Kubat , Moiez Tapia, Time spent on a web page is sufficient to infer a user's interest, IASTED European Conference on Proceedings of the IASTED European Conference: internet and multimedia systems and applications, p.41-46, March 14-16, 2007, Chamonix, France
• 19: Chien-Kang Huang , Yen-Jen Oyang , Lee-Feng Chien, A Contextual Term Suggestion Mechanism for Interactive Web Search, Proceedings of the First Asia-Pacific Conference on Web Intelligence: Research and Development, p.272-281, October 23-26, 2001
• 20: Thorsten Joachims, A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text Categorization, Proceedings of the Fourteenth International Conference on Machine Learning, p.143-151, July 08-12, 1997
• 21: Thorsten Joachims, Optimizing search engines using clickthrough data, Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, July 23-26, 2002, Edmonton, Alberta, Canada [doi>10.1145/775047.775067]
• 22: Thorsten Joachims , Laura Granka , Bing Pan , Helene Hembrooke , Geri Gay, Accurately interpreting clickthrough data as implicit feedback, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076063]
• 23: K. S. Jones. What might be in a summary. In Information Retrieval 93: Von der Modellierung zur Anwendung, pages 9--26, 1993.
• 24: Diane Kelly , Nicholas J. Belkin, Reading time, scrolling and interaction: exploring implicit sources of user preferences for relevance feedback, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.408-409, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.384045]
• 25: Diane Kelly , Nicholas J. Belkin, Display time as implicit feedback: understanding task effects, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009057]
• 26: K.-N. Kim and R. Ramakrishna. Vision-based eye-gaze tracking for human computer interface. SMC '99: Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, 2:324--329, 1999.
• 27: Yuhua Li , Zuhair A. Bandar , David McLean, An Approach for Measuring Semantic Similarity between Words Using Multiple Information Sources, IEEE Transactions on Knowledge and Data Engineering, v.15 n.4, p.871-882, July 2003 [doi>10.1109/TKDE.2003.1209005]
• 28: Chin-Yew Lin, Training a selection function for extraction, Proceedings of the eighth international conference on Information and knowledge management, p.55-62, November 02-06, 1999, Kansas City, Missouri, United States [doi>10.1145/319950.319957]
• 29: Y.-P. Lin, Y.-P. Chao, C.-C. Lin, and J.-H. Chen. Webcam mouse using face and eye tracking in various illumination environments. EMBS '05: Proceedings of 27th IEEE Annual International Conference of Engineering in Medicine and Biology Society, pages 3738--3741, 2005.
• 30: Fang Liu , Clement Yu , Weiyi Meng, Personalized web search by mapping user queries to categories, Proceedings of the eleventh international conference on Information and knowledge management, November 04-09, 2002, McLean, Virginia, USA [doi>10.1145/584792.584884]
• 31: Yuanhua Lv , Le Sun , Junlin Zhang , Jian-Yun Nie , Wan Chen , Wei Zhang, An iterative implicit feedback approach to personalized search, Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, p.585-592, July 17-18, 2006, Sydney, Australia [doi>10.3115/1220175.1220249]
• 32: Mark T. Maybury , Inderjeet Mani, Advances in Automatic Text Summarization, MIT Press, Cambridge, MA, 1999
• 33: Microsoft. Word (software), http://office.microsoft.com/word/, Microsoft Corporation, last visited on December 11, 2008.
• 34: James Pitkow , Hinrich Schütze , Todd Cass , Rob Cooley , Don Turnbull , Andy Edmonds , Eytan Adar , Thomas Breuel, Personalized search, Communications of the ACM, v.45 n.9, September 2002 [doi>10.1145/567498.567526]
• 35: D. Radev, T. Allison, S. Blair-Goldensohn, J. Blitzer, A. C¸ elebi, S. Dimitrov, E. Drabek, A. Hakim, W. Lam, D. Liu, J. Otterbacher, H. Qi, H. Saggion, S. Teufel, M. Topper, A. Winkel, and Z. Zhang. MEAD--A platform for multidocument multilingual text summarization. In LREC '04: The 2nd International Conference on Language Resources and Evaluation, Lisbon, Portugal, May 2004.
• 36: Filip Radlinski , Thorsten Joachims, Query chains: learning to rank from implicit feedback, Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, August 21-24, 2005, Chicago, Illinois, USA [doi>10.1145/1081870.1081899]
• 37: Robert W. Reeder , Peter Pirolli , Stuart K. Card, WebEyeMapper and WebLogger: tools for analyzing eye tracking data collected in web-use studies, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634082]
• 38: Ravikrishna Ruddarraju , Antonio Haro , Kris Nagel , Quan T. Tran , Irfan A. Essa , Gregory Abowd , Elizabeth D. Mynatt, Perceptual user interfaces using vision-based eye tracking, Proceedings of the 5th international conference on Multimodal interfaces, November 05-07, 2003, Vancouver, British Columbia, Canada [doi>10.1145/958432.958475]
• 39: G. Salton and C. Buckley. Improving retrieval performance by relevance feedback. Journal of the American Society for Information Science, 41(4):288--297, 1990.
• 40: Micro Speretta , Susan Gauch, Personalized Search Based on User Search Histories, Proceedings of the 2005 IEEE/WIC/ACM International Conference on Web Intelligence, p.622-628, September 19-22, 2005 [doi>10.1109/WI.2005.114]
• 41: M.-C. Su, S.-Y. Su, and G.-D. Chen. A low-cost vision-based human-computer interface for people with severe disabilities. Biomedical Engineering Applications, Basis, and Communications, 17:284--292, 2005.
• 42: R. White, J. M. Jose, and I. Ruthven. Comparing explicit and implicit feedback techniques for web retrieval: Trec-10 interactive track report. In TREC 2001.
• 43: Ryen White , Ian Ruthven , Joemon M. Jose, The Use of Implicit Evidence for Relevance Feedback in Web Retrieval, Proceedings of the 24th BCS-IRSG European Colloquium on IR Research: Advances in Information Retrieval, p.93-109, March 25-27, 2002
• 44: Songhua Xu , Hao Jiang , Francis C.M. Lau, Personalized online document, image and video recommendation via commodity eye-tracking, Proceedings of the 2008 ACM conference on Recommender systems, October 23-25, 2008, Lausanne, Switzerland [doi>10.1145/1454008.1454023]
• 45: S. Xu, Y. Zhu, H. Jiang, and F. C. M. Lau. A user-oriented webpage ranking algorithm based on user attention time. In AAAI '08: Proceedings of the 23rd AAAI Conference on Artificial Intelligence, pages 1255--1260, 2008, AAAI Press.
• 46: Jen-Yuan Yeh , Hao-Ren Ke , Wei-Pang Yang , I-Heng Meng, Text summarization using a trainable summarizer and latent semantic analysis, Information Processing and Management: an International Journal, v.41 n.1, p.75-95, January 2005 [doi>10.1016/j.ipm.2004.04.003]
• 47: P. Zielinski. Opengazer: open-source gaze tracker for ordinary webcams (software), Samsung and The Gatsby Charitable Foundation. http://www.inference.phy.cam.ac.uk/opengazer/, last visited on December 11 2008.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human information processing
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Abstracting methods
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation); Input devices and strategies (e.g., mouse, touchscreen)

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Discourse; Text analysis
  I.7 DOCUMENT AND TEXT PROCESSING
      I.7.5 Document Capture
          Subjects: Document analysis

General Terms:
Algorithms, Design, Experimentation, Human Factors, Languages, Measurement, Performance

Keywords:
commodity eye-tracking, implicit user feedback, personalized discourse abstract, user attention, user-oriented document summarization