Touch & interact: touch-based interaction of mobile phones with displays

ABSTRACT

The limited screen size and resolution of current mobile devices can still be problematic for map, multimedia and browsing applications. In this paper we present Touch & Interact: an interaction technique in which a mobile phone is able to touch a display, at any position, to perform selections. Through the combination of the output capabilities of the mobile phone and display, applications can share the entire display space. Moreover, there is potential to realize new interaction techniques between the phone and display. For example, select & pick and select & drop are interactions whereby entities can be picked up onto the phone or dropped onto the display. We report the implementation of Touch & Interact, its usage for a tourist guide application and experimental comparison. The latter shows that the performance of Touch & Interact is comparable to approaches based on a touch screen; it also shows the advantages of our system regarding ease of use, intuitiveness and enjoyment.

REFERENCES

• 1: George H. Forman , John Zahorjan, The Challenges of Mobile Computing, Computer, v.27 n.4, p.38-47, April 1994 [doi>10.1109/2.274999]
• 2: Ferscha, A., Kathan, G. Vogl, S. WebWall - An Architecture for Public Display WWW Services. World Wide Web Conference, Honolulu, Hawaii, 2002.
• 3: Keith Cheverst , Alan Dix , Daniel Fitton , Chris Kray , Mark Rouncefield , Corina Sas , George Saslis-Lagoudakis , Jennifer G. Sheridan, Exploring bluetooth based mobile phone interaction with the hermes photo display, Proceedings of the 7th international conference on Human computer interaction with mobile devices & services, September 19-22, 2005, Salzburg, Austria [doi>10.1145/1085777.1085786]
• 4: SpotCode Interfaces, http://www.cl.cam.ac.uk/Research/SRG/netos/uid/spotcode.html
• 5: Rafael Ballagas , Michael Rohs , Jennifer G. Sheridan, Sweep and point and shoot: phonecam-based interactions for large public displays, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1056876]
• 6: Rafael Ballagas , Jan Borchers , Michael Rohs , Jennifer G. Sheridan, The Smart Phone: A Ubiquitous Input Device, IEEE Pervasive Computing, v.5 n.1, p.70, January 2006 [doi>10.1109/MPRV.2006.18]
• 7: Sharp, R., Scott, J., Beresford, A. R. Secure Mobile Computing Via Public Terminals. In Proc. Pervasive'06. Springer. 238--253. 2006
• 8: M. C. Maguire, A review of user-interface design guidelines for public information kiosk systems, International Journal of Human-Computer Studies, v.50 n.3, p.263-286, March 1999 [doi>10.1006/ijhc.1998.0243]
• 9: Daniel Vogel , Patrick Baudisch, Shift: a technique for operating pen-based interfaces using touch, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240727]
• 10: George W. Fitzmaurice, Situated information spaces and spatially aware palmtop computers, Communications of the ACM, v.36 n.7, p.39-49, July 1993 [doi>10.1145/159544.159566]
• 11: Brad A. Myers , Herb Stiel , Robert Gargiulo, Collaboration using multiple PDAs connected to a PC, Proceedings of the 1998 ACM conference on Computer supported cooperative work, p.285-294, November 14-18, 1998, Seattle, Washington, United States [doi>10.1145/289444.289503]
• 12: Roy Want , Kenneth P. Fishkin , Anuj Gujar , Beverly L. Harrison, Bridging physical and virtual worlds with electronic tags, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.370-377, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303111]
• 13: John Boyd, Here comes the wallet phone, IEEE Spectrum, v.42 n.11, p.12-14, November 2005 [doi>10.1109/MSPEC.2005.1526896]
• 14: Derek Reilly , Malcolm Rodgers , Ritchie Argue , Mike Nunes , Kori Inkpen, Marked-up maps: combining paper maps and electronic information resources, Personal and Ubiquitous Computing, v.10 n.4, p.215-226, March 2006 [doi>10.1007/s00779-005-0043-6]
• 15: Madhavapeddy, A., Scott, D., Sharp, R., Upton, E. Using Camera-Phones to Enhance Human Computer Interaction. Adjunct Proceedings of Ubicomp 2004, Nottingham, UK, 2004
• 16: Kento Miyaoku , Suguru Higashino , Yoshinobu Tonomura, C-blink: a hue-difference-based light signal marker for large screen interaction via any mobile terminal, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029657]
• 17: Clifton Forlines , Ravin Balakrishnan , Paul Beardsley , Jeroen van Baar , Ramesh Raskar, Zoom-and-pick: facilitating visual zooming and precision pointing with interactive handheld projectors, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095046]
• 18: Miika Silfverberg , I. Scott MacKenzie , Tatu Kauppinen, An isometric joystick as a pointing device for handheld information terminals, No description on Graphics interface 2001, p.119-126, June 07-09, 2001, Ottawa, Ontario, Canada
• 19: Jun Rekimoto, Pick-and-drop: a direct manipulation technique for multiple computer environments, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.31-39, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263505]
• 20: Paul Dietz , Darren Leigh, DiamondTouch: a multi-user touch technology, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502389]
• 21: Microsoft Surface, http://www.microsoft.com/surface/
• 22: Johannes Vetter , John Hamard , Massimo Paolucci , Enrico Rukzio , Albrecht Schmidt, Physical mobile interaction with dynamic physical object, Proceedings of the 9th international conference on Human computer interaction with mobile devices and services, p.339-340, September 09-12, 2007, Singapore [doi>10.1145/1377999.1378030]
• 23: Hitachi Global News Release, http://www.hitachi.com/New/cnews/030902.html
• 24: Nokia Wireless Presenter, http://europe.nokia.com/A4149109
• 25: Sarah A. Douglas , Arthur E. Kirkpatrick , I. Scott MacKenzie, Testing pointing device performance and user assessment with the ISO 9241, Part 9 standard, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.215-222, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303042]
• 26: Venkatesh, V., Morris, M. G., Davies, G. B., Davies, D. D., User Acceptance of Information Technology: Toward a unified view. MIS Quarterly Vol. 27 No. 3, pp. 425--478/September 2003.
• 27: Mulitag project, http://eis.comp.lancs.ac.uk/multitag/

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Prototyping

General Terms:
Design, Experimentation, Human Factors, Measurement

Keywords:
dynamic display, mobile interaction, touch & interact