Automatically classifying emails into activities

ABSTRACT

Email-based activity management systems promise to give users better tools for managing increasing volumes of email, by organizing email according to a user's activities. Current activity management systems do not automatically classify incoming messages by the activity to which they belong, instead relying on simple heuristics (such as message threads), or asking the user to manually classify incoming messages as belonging to an activity. This paper presents several algorithms for automatically recognizing emails as part of an ongoing activity. Our baseline methods are the use of message reply-to threads to determine activity membership and a naïve Bayes classifier. Our SimSubset and SimOverlap algorithms compare the people involved in an activity against the recipients of each incoming message. Our SimContent algorithm uses IRR (a variant of latent semantic indexing) to classify emails into activities using similarity based on message contents. An empirical evaluation shows that each of these methods provide a significant improvement to the baseline methods. In addition, we show that a combined approach that votes the predictions of the individual methods performs better than each individual method alone.

REFERENCES

• 1: Manu Aery and Sharma Chakravarthy. eMailSift: mining-based approaches to email classification. In SIGIR '04: Proc. of the 27th annual intl. ACM SIGIR conf. on information retrieval, pages 580--581. ACM Press, 2004.
• 2: Rie Kubota Ando , Lillian Lee, Iterative residual rescaling, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.154-162, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383981]
• 3: Victoria Bellotti , Nicolas Ducheneaut , Mark Howard , Ian Smith, Taking email to task: the design and evaluation of a task management centered email tool, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642672]
• 4: W. Cohen, V. Carvalho, and T. Mitchell. Learning to classify email into "speech acts". In Proc. Conf. Empirical Methods in Natural Language Processing, 2004.
• 5: Alex Cozzi, Tom Moran, and Clemens Drews. The shared checklist: Reorganizing the user experience around unified activities. In 10th Intl Conf on Human-Computer Interaction (INTERACT 2005), Sept. 2005.
• 6: S. Deerwester, S. T. Dumais, T. K. Landauer, G. W. Furnas, and R. A. Harshman. Indexing by latent semantic analysis. Journal of the Society for Information Science, 41(6):391--407, 1990.
• 7: Anton N. Dragunov , Thomas G. Dietterich , Kevin Johnsrude , Matthew McLaughlin , Lida Li , Jonathan L. Herlocker, TaskTracer: a desktop environment to support multi-tasking knowledge workers, Proceedings of the 10th international conference on Intelligent user interfaces, January 10-13, 2005, San Diego, California, USA [doi>10.1145/1040830.1040855]
• 8: Nicolas Ducheneaut , Victoria Bellotti, E-mail as habitat: an exploration of embedded personal information management, interactions, v.8 n.5, p.30-38, Sept./Oct. 2001 [doi>10.1145/382899.383305]
• 9: Y. Huang, D. Govindaraju, T. Mitchell, V. Rocha de Carvalho, and W. Cohen. Inferring ongoing activities of workstation users by clustering email. In Proc. of the 1st Conf. on Email and Anti-Spam, July 2004.
• 10: R. Khoussainov and N. Kushmerick. Email task management: An iterative relational learning approach. In Proc. Conf. Email and Anti-Spam, 2005.
• 11: S. Kiritchenko, S. Matwin, and S. Abu-Hakima. Email classification with temporal features. In Proceedings of Intelligent Information Systems, New Trends in Intelligent Information Processing and Web Mining (IIPWM) 2004, pages 523--534. Springer Verlag, 2004.
• 12: Svetlana Kiritchenko , Stan Matwin, Email classification with co-training, Proceedings of the 2001 conference of the Centre for Advanced Studies on Collaborative research, p.8, November 05-07, 2001, Toronto, Ontario, Canada
• 13: Nicholas Kushmerick , Tessa Lau, Automated email activity management: an unsupervised learning approach, Proceedings of the 10th international conference on Intelligent user interfaces, January 10-13, 2005, San Diego, California, USA [doi>10.1145/1040830.1040854]
• 14: Andrew McCallum, Andres Corrada-Emmanuel, and Xuerui Wang. Topic and Role Discovery in Social Networks. In Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, July 2005.
• 15: Luke McDowell , Oren Etzioni , Alon Halevy , Henry Levy, Semantic email, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988706]
• 16: Thomas P. Moran , Alex Cozzi , Stephen P. Farrell, Unified activity management: supporting people in e-business, Communications of the ACM, v.48 n.12, December 2005 [doi>10.1145/1101779.1101811]
• 17: Michael J. Muller , Werner Geyer , Beth Brownholtz , Eric Wilcox , David R. Millen, One-hundred days in an activity-centric collaboration environment based on shared objects, Proceedings of the SIGCHI conference on Human factors in computing systems, p.375-382, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985740]
• 18: M.F. Porter. An algorithm for suffix stripping. Program, 14(3):130--137, 1980.
• 19: Mehran Sahami, Susan Dumais, David Heckerman, and Eric Horvitz. A bayesian approach to filtering junk E-mail. In Learning for Text Categorization: Papers from the 1998 Workshop, Madison, Wisconsin, 1998. AAAI Technical Report WS-98-05.
• 20: R. Segal, J. Crawford, J. Kephart, and B. Leiba. SpamGuru: An Enterprise Anti-Spam Filtering System. In Proceedings of the First Conference on Email and Anti-Spam, July 2004.
• 21: Richard Segal , Jeffrey O. Kephart, Incremental Learning in SwiftFile, Proceedings of the Seventeenth International Conference on Machine Learning, p.863-870, June 29-July 02, 2000
• 22: A. Surendran, J. Platt, and E. Renshaw. Automatic discovery of personal topics to organize email. In Proc. of the 2nd Conf. on Email and Anti-Spam, July 2005.
• 23: Steve Whittaker , Candace Sidner, Email overload: exploring personal information management of email, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.276-283, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238530]
• 24: Ian H. Witten and Eibe Frank. Data Mining: Practical machine learning tools and techniques, 2nd ed. Morgan Kaufmann, 2005.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)

General Terms:
Algorithms, Experimentation

Keywords:
activity management, email, machine learning, text classification