Generating summary keywords for emails using topics

ABSTRACT

Email summary keywords, used to concisely represent the gist of an email, can help users manage and prioritize large numbers of messages. We develop an unsupervised learning framework for selecting summary keywords from emails using latent representations of the underlying topics in a user's mailbox. This approach selects words that describe each message in the context of existing topics rather than simply selecting keywords based on a single message in isolation. We present and compare four methods for selecting summary keywords based on two well-known models for inferring latent topics: latent semantic analysis and latent Dirichlet allocation. The quality of the summary keywords is assessed by generating summaries for emails from twelve users in the Enron corpus. The summary keywords are then used in place of entire messages in two proxy tasks: automated foldering and recipient prediction. We also evaluate the extent to which summary keywords enhance the information already available in a typical email user interface by repeating the same tasks using email subject lines.

REFERENCES

• 1: Ron Bekkerman, Andrew McCallum, and Gary Huang. Automatic categorization of email into folders: Benchmark experiments on Enron and SRI corpora. Technical Report IR-418, University of Massachusetts Amherst, 2004.
• 2: David M. Blei , Andrew Y. Ng , Michael I. Jordan, Latent dirichlet allocation, The Journal of Machine Learning Research, 3, p.993-1022, 3/1/2003
• 3: Wray Buntine , Jaakko Lofstrom , Jukka Perkio , Sami Perttu , Vladimir Poroshin , Tomi Silander , Henry Tirri , Antti Tuominen , Ville Tuulos, A Scalable Topic-Based Open Source Search Engine, Proceedings of the 2004 IEEE/WIC/ACM International Conference on Web Intelligence, p.228-234, September 20-24, 2004 [doi>10.1109/WI.2004.12]
• 4: Giuseppe Carenini , Raymond T. Ng , Xiaodong Zhou, Summarizing email conversations with clue words, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242586]
• 5: Vitor R. Carvalho and William Cohen. Recommending recipients in the Enron email corpus. Technical Report CMU-LTI-07-005, Carnegie Mellon University, 2007.
• 6: Koby Crammer , Ofer Dekel , Joseph Keshet , Shai Shalev-Shwartz , Yoram Singer, Online Passive-Aggressive Algorithms, The Journal of Machine Learning Research, 7, p.551-585, 12/1/2006
• 7: Angelo Dalli , Yunqing Xia , Yorick Wilks, FASIL email summarisation system, Proceedings of the 20th international conference on Computational Linguistics, p.994-es, August 23-27, 2004, Geneva, Switzerland [doi>10.3115/1220355.1220498]
• 8: S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman. Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6):391--407, 1990.
• 9: Mark Dredze , Tessa Lau , Nicholas Kushmerick, Automatically classifying emails into activities, Proceedings of the 11th international conference on Intelligent user interfaces, January 29-February 01, 2006, Sydney, Australia [doi>10.1145/1111449.1111471]
• 10: Susan T. Dumais. LSI meets TREC: A status report. In Text REtrieval Conference, pages 137--152, 1992.
• 11: Michael Fink , Shai Shalev-Shwartz , Yoram Singer , Shimon Ullman, Online multiclass learning by interclass hypothesis sharing, Proceedings of the 23rd international conference on Machine learning, p.313-320, June 25-29, 2006, Pittsburgh, Pennsylvania [doi>10.1145/1143844.1143884]
• 12: Joshua Goodman and Vitor R. Carvalho. Implicit queries for email. In CEAS, 2005.
• 13: T. L. Griffiths and M. Steyvers. A probabilistic approach to semantic representation. In Proceedings of the 24th Annual Conference of the Cognitive Society, 2002.
• 14: T. Hoffman. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, 1999.
• 15: B. Klimt and Y. Yang. The Enron corpus: A new dataset for email classification research. In ECML, 2004.
• 16: Andrew McCallum, Andres Corrada-Emmanuel, and Xuerui Wang. Topic and role discovery in social networks. In IJCAI, 2005.
• 17: Andrew McCallum, Xuerui Wang, and Andres Corrada-Emmanuel. Topic and role discovery in social networks with experiments on Enron and academic email. In Journal of Artificial Intelligence Research, 2007.
• 18: Andrew Kachites McCallum. MALLET: A machine learning for language toolkit. http://mallet.cs.umass.edu, 2002.
• 19: Ryan McDonald, Koby Crammer, Kuzman Ganchev, Surya Prakash Bachoti, and Mark Dredze. Penn StructLearn. http://www.seas.upenn.edu/strctlrn/StructLearn/StructLearn.html, 2006.
• 20: Smaranda Muresan , Evelyne Tzoukermann , Judith L. Klavans, Combining linguistic and machine learning techniques for email summarization, Proceedings of the 2001 workshop on Computational Natural Language Learning, p.1-8, July 06-07, 2001, Toulouse, France [doi>10.3115/1117822.1117837]
• 21: Carman Neustaedter, A. J. Bernheim Brush, Marc A. Smith, and Danyel Fisher. The social network and relationship finder: Social sorting for email triage. In Proceedings of the Conference on Email and Anti-Spam (CEAS), Mountain View, CA, 2005.
• 22: Chris Pal and Andrew McCallum. CC prediction with graphical models. In Conference on Email and Anti-Spam (CEAS), 2006.
• 23: Owen Rambow, Lokesh Shrestha, John Chen, and Chirsty Lauridsen. Summarizing email threads. In HLT/NAACL, 2004.
• 24: Richard B. Segal , Jeffrey O. Kephart, MailCat: an intelligent assistant for organizing e-mail, Proceedings of the third annual conference on Autonomous Agents, p.276-282, April 1999, Seattle, Washington, United States [doi>10.1145/301136.301209]
• 25: S Sood, S Owsley, K Hammond, and L Birnbaum. Tag Assist: Automatic tag suggestion for blog posts. In ICWSM, 2007.
• 26: Mark Steyvers and Tom Griffiths. Probabilistic topic models. In D McNamara, S Dennis, and W Kintsch, editors, Latent Semantic Analysis: A Road to Meaning. Laurence Erlbaum, in press.
• 27: G. Venolia, L. Dabbish, J. J. Cadiz, and A. Gupta. Supporting email workflow. Technical Report MSR-TR-2001-88, Microsoft Research, 2001.
• 28: Hanna M. Wallach, Topic modeling: beyond bag-of-words, Proceedings of the 23rd international conference on Machine learning, p.977-984, June 25-29, 2006, Pittsburgh, Pennsylvania [doi>10.1145/1143844.1143967]
• 29: Stephen Wan , Kathy McKeown, Generating overview summaries of ongoing email thread discussions, Proceedings of the 20th international conference on Computational Linguistics, p.549-es, August 23-27, 2004, Geneva, Switzerland [doi>10.3115/1220355.1220434]
• 30: Xuerui Wang and Andrew McCallum. A note on topical n-grams. Technical Report UM-CS-2005-071, University of Massachusetts Amherst, 2005.
• 31: Xing Wei , W. Bruce Croft, LDA-based document models for ad-hoc retrieval, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148204]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

General Terms:
Design, Human Factors

Keywords:
email, foldering, keyword generation, recipient prediction, topic modeling