An infrastructure for extending applications' user experiences across multiple personal devices

ABSTRACT

Users increasingly interact with a heterogeneous collection of computing devices. The applications that users employ on those devices, however, still largely provide user experiences that assume the use of a single computer. This failure is due in part to the difficulty of creating user experiences that span multiple devices, particularly the need to manage identifying, connecting to, and communicating with other devices. In this paper we present an infrastructure based on instant messaging that simplifies adding that additional functionality to applications. Our infrastructure elevates device ownership to a first class property, allowing developers to provide functionality that spans personal devices without writing code to manage users' devices or establish connections among them. It also provides simple mechanisms for applications to send information, events, or commands between a user's devices. We demonstrate the effectiveness of our infrastructure by presenting a set of sample applications built with it and a user study demonstrating that developers new to the infrastructure can implement all of the cross-device functionality for three applications in, on average, less than two and a half hours.

REFERENCES

• 1: Juwon Ahn , Jeffrey S. Pierce, SEREFE: serendipitous file exchange between users and devices, Proceedings of the 7th international conference on Human computer interaction with mobile devices & services, September 19-22, 2005, Salzburg, Austria [doi>10.1145/1085777.1085785]
• 2: Jakob Bardram , Jonathan Bunde-Pedersen , Mads Soegaard, Support for activity-based computing in a personal computing operating system, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124805]
• 3: Beagle. http://beagle-project.org/Main_Page.
• 4: Ramón Cáceres , Casey Carter , Chandra Narayanaswami , Mandayam Raghunath, Reincarnating PCs with portable SoulPads, Proceedings of the 3rd international conference on Mobile systems, applications, and services, June 06-08, 2005, Seattle, Washington [doi>10.1145/1067170.1067179]
• 5: David Dearman , Jeffery S. Pierce, It's on my other computer!: computing with multiple devices, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357177]
• 6: Extensible Messaging and Presence Protocol. Core, Instant Messaging and Presence. http://www.ietf.org/rfc/rfc3920.txt and http://www.ietf.org/rfc/rfc3921.txt.
• 7: eyeOS. http://eyeos.org.
• 8: FolderShare is a Windows Live Service. https://www.foldershare.com/
• 9: Armando Fox , Brad Johanson , Pat Hanrahan , Terry Winograd, Integrating Information Appliances into an Interactive Workspace, IEEE Computer Graphics and Applications, v.20 n.3, p.54-65, May 2000 [doi>10.1109/38.844373]
• 10: Heather M. Hutchings , Jeffrey S. Pierce, Understanding the whethers, hows, and whys of divisible interfaces, Proceedings of the working conference on Advanced visual interfaces, May 23-26, 2006, Venezia, Italy [doi>10.1145/1133265.1133320]
• 11: Brad Johanson , Shankar Ponnekanti , Caesar Sengupta , Armando Fox, Multibrowsing: Moving Web Content across Multiple Displays, Proceedings of the 3rd international conference on Ubiquitous Computing, p.346-353, September 30-October 02, 2001, Atlanta, Georgia, USA
• 12: Microsoft Office Groove. http://office.microsoft.com/en-us/groove/default.aspx.
• 13: Mark W. Newman , Shahram Izadi , W. Keith Edwards , Jana Z. Sedivy , Trevor F. Smith, User interfaces when and where they are needed: an infrastructure for recombinant computing, Proceedings of the 15th annual ACM symposium on User interface software and technology, October 27-30, 2002, Paris, France [doi>10.1145/571985.572009]
• 14: Dan R. Olsen, Jr. , S. Travis Nielsen , David Parslow, Join and capture: a model for nomadic interaction, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502367]
• 15: Antti Oulasvirta , Lauri Sumari, Mobile kits and laptop trays: managing multiple devices in mobile information work, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240795]
• 16: Trevor Pering , Rafael Ballagas , Roy Want, Spontaneous marriages of mobile devices and interactive spaces, Communications of the ACM, v.48 n.9, September 2005 [doi>10.1145/1081992.1082020]
• 17: Mandayam Raghunath , Chandra Narayanaswami , Claudio Pinhanez, Fostering a Symbiotic Handheld Environment, Computer, v.36 n.9, p.56-65, September 2003 [doi>10.1109/MC.2003.1231195]
• 18: Tristan Richardson , Quentin Stafford-Fraser , Kenneth R. Wood , Andy Hopper, Virtual Network Computing, IEEE Internet Computing, v.2 n.1, p.33-38, January 1998 [doi>10.1109/4236.656066]
• 19: Manuel Román , Christopher Hess , Renato Cerqueira , Anand Ranganathan , Roy H. Campbell , Klara Nahrstedt, A Middleware Infrastructure for Active Spaces, IEEE Pervasive Computing, v.1 n.4, p.74-83, October 2002 [doi>10.1109/MPRV.2002.1158281]
• 20: Mahadev Satyanarayanan , James J. Kistler , Puneet Kumar , Maria E. Okasaki , Ellen H. Siegel , David C. Steere, Coda: A Highly Available File System for a Distributed Workstation Environment, IEEE Transactions on Computers, v.39 n.4, p.447-459, April 1990 [doi>10.1109/12.54838]
• 21: Satyanarayanan, M., Kozuch, M., Helfrich, C., and O'Hallaron, D. R. Towards Seamless Mobility on Pervasive Hardware. Pervasive & Mobile Computing, 1, 2 (July 2005), pp. 157--189.
• 22: Bill N. Schilit , Uttam Sengupta, Device Ensembles, Computer, v.37 n.12, p.56-64, December 2004 [doi>10.1109/MC.2004.241]
• 23: Synergy. http://synergy2.sourceforge.net/.
• 24: Peter Tandler, The BEACH application model and software framework for synchronous collaboration in ubiquitous computing environments, Journal of Systems and Software, v.69 n.3, p.267-296, 15 January 2004 [doi>10.1016/S0164-1212(03)00055-4]
• 25: John C. Tang , James Lin , Jeffrey Pierce , Steve Whittaker , Clemens Drews, Recent shortcuts: using recent interactions to support shared activities, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240816]
• 26: Min Yin , Shumin Zhai, Dial and see: tackling the voice menu navigation problem with cross-device user experience integration, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095066]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

General Terms:
Design, Human Factors

Keywords:
application development, infrastructure, multi-device services, multi-device user experiences