SWISH: semantic analysis of window titles and switching history

ABSTRACT

Information workers are often involved in multiple tasks and activities that they must perform in parallel or in rapid succession. In consequence, task management itself becomes yet another task that information workers need to perform in order to get the rest of their work done. Recognition of this problem has led to research on task management systems, which can help by allowing fast task switching, fast task resumption, and automatic task identification. In this paper we focus on the latter: we tackle the problem of automatically detecting the tasks that the user is involved in, by identifying which of the windows on the user's desktop are related to each other. The underlying assumption is that windows that belong to the same task share some common properties with one another that we can detect from data. We will refer to this problem as the task assignment problem.To address this problem, we have built a prototype named Swish that: (1) constantly monitors users' desktop activities using a stream of windows events; (2) logs and processes this raw event stream, and (3) implements two criteria of window "relatedness", namely the semantic similarity of their titles, and the temporal closeness in their access patterns.In addition to describing the Swish prototype in detail, we validate it with 4 hours of user data, obtaining task classification accuracies of about 70%. We also discuss our plans on including Swish in a number of intelligent user interfaces and future lines of research.

REFERENCES

• 1: Akiko Aizawa, An information-theoretic perspective of tf—idf measures, Information Processing and Management: an International Journal, v.39 n.1, p.45-65, January 2003 [doi>10.1016/S0306-4573(02)00021-3]
• 2: Liam Bannon , Allen Cypher , Steven Greenspan , Melissa L. Monty, Evaluation and analysis of users' activity organization, Proceedings of the SIGCHI conference on Human Factors in Computing Systems, p.54-57, December 12-15, 1983, Boston, Massachusetts, United States [doi>10.1145/800045.801580]
• 3: Patrick Baudisch , Nathaniel Good , Paul Stewart, Focus plus context screens: combining display technology with visualization techniques, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502354]
• 4: Benjamin B. Bederson , James D. Hollan, Pad++: a zooming graphical interface for exploring alternate interface physics, Proceedings of the 7th annual ACM symposium on User interface software and technology, p.17-26, November 02-04, 1994, Marina del Rey, California, United States [doi>10.1145/192426.192435]
• 5: Blaine A. Bell , Steven K. Feiner, Dynamic space management for user interfaces, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.239-248, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354790]
• 6: Victoria Bellotti , Nicolas Ducheneaut , Mark Howard , Ian Smith, Taking email to task: the design and evaluation of a task management centered email tool, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642672]
• 7: S. A. Bly , J. K. Rosenberg, A comparison of tiled and overlapping windows, Proceedings of the SIGCHI conference on Human factors in computing systems, p.101-106, April 13-17, 1986, Boston, Massachusetts, United States
• 8: Coen Bron , Joep Kerbosch, Algorithm 457: finding all cliques of an undirected graph, Communications of the ACM, v.16 n.9, p.575-577, Sept. 1973 [doi>10.1145/362342.362367]
• 9: S.C. Deerwester, S. Dumais, T.K. Landauer, G.W. Fumas, and R.A. Harshman. Indexing by latent semantic analysis. Journal of the American Society of Information Science, 41(6):pp. 391--407, 1990.
• 10: A.P. Dempster, N.M. Laird, and D.B Rubin. Maximum likelihood from incomplete data via de $em$ algorithm. Journal of the Royal Statistical Society, 39-B:pp. 1--38, 1977.
• 11: Kurt D. Fenstermacher , Mark Ginsburg, A Lightweight Framework for Cross-Application User Monitoring, Computer, v.35 n.3, p.51-59, March 2002
• 12: A. Goldberg. Smalltalk-80. Addison-Wesley, 1983.
• 13: D. Austin Henderson, Jr. , Stuart Card, Rooms: the use of multiple virtual workspaces to reduce space contention in a window-based graphical user interface, ACM Transactions on Graphics (TOG), v.5 n.3, p.211-243, July 1986 [doi>10.1145/24054.24056]
• 14: Thomas Hofmann, Probabilistic latent semantic indexing, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.50-57, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312649]
• 15: Dugald Ralph Hutchings , Greg Smith , Brian Meyers , Mary Czerwinski , George Robertson, Display space usage and window management operation comparisons between single monitor and multiple monitor users, Proceedings of the working conference on Advanced visual interfaces, May 25-28, 2004, Gallipoli, Italy [doi>10.1145/989863.989867]
• 16: Finn V. Jensen, Bayesian Networks and Decision Graphs, Springer-Verlag New York, Inc., Secaucus, NJ, 2001
• 17: Eser Kandogan , Ben Shneiderman, Elastic Windows: evaluation of multi-window operations, Proceedings of the SIGCHI conference on Human factors in computing systems, p.250-257, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258720]
• 18: Blair MacIntyre , Elizabeth D. Mynatt , Stephen Voida , Klaus M. Hansen , Joe Tullio , Gregory M. Corso, Support for multitasking and background awareness using interactive peripheral displays, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502355]
• 19: Marina Meila , David Heckerman, An experimental comparison of model-based clustering methods, Machine Learning, v.42 n.1/2, p.9-29, Jan. 2001 [doi>10.1023/A:1007648401407]
• 20: M. Porter. An algorithm for suffix stripping. Program, 14(3):pp. 130--137, 1980.
• 21: Jun Rekimoto, Time-machine computing: a time-centric approach for the information environment, Proceedings of the 12th annual ACM symposium on User interface software and technology, p.45-54, November 07-10, 1999, Asheville, North Carolina, United States [doi>10.1145/320719.322582]
• 22: George Robertson , Eric Horvitz , Mary Czerwinski , Patrick Baudisch , Dugald Ralph Hutchings , Brian Meyers , Daniel Robbins , Greg Smith, Scalable Fabric: flexible task management, Proceedings of the working conference on Advanced visual interfaces, May 25-28, 2004, Gallipoli, Italy [doi>10.1145/989863.989874]
• 23: George Robertson , Maarten van Dantzich , Daniel Robbins , Mary Czerwinski , Ken Hinckley , Kirsten Risden , David Thiel , Vadim Gorokhovsky, The Task Gallery: a 3D window manager, Proceedings of the SIGCHI conference on Human factors in computing systems, p.494-501, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332482]
• 24: G. Smith, P. Baudisch, G. Robertson, M. Czewinski, B. Meyers, D. Robbins, and D. Andrews. Groupbar: The taskbar evolved. In Proc. OZCHI'03, pages pp. 41--50, 2003.
• 25: S. Stumpf, X. Bao, A. Dragunov, T.G. Dietterich, J.L. Herlocker, K. Johnsrude, L. Li, and J. Shen. Predicting user tasks: I know what you're doing! In National Conference on Artificial Intelligence (AAAI'05), pages pp. --, 2005.
• 26: Warren Teitelman, Ten years of window systems—a retrospective view, Proceedings of an Alvey Workshop on Methodology of window management, p.35-46, March 1986, Cosener's House, Abingdon, United Kingdom
• 27: H. Wurnig. Design of a collaborative multi-user desktop system for augmented reality. In Proc. Central European Seminar in Computer Graphics, 1998.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.3 Clustering

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces

  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Clustering

General Terms:
Algorithms

Keywords:
automatic task identification, clustering, user activity monitoring, user task modeling