Spatially aware handhelds for high-precision tangible interaction with large displays

ABSTRACT

While touch-screen displays are becoming increasingly popular, many factors affect user experience and performance. Surface quality, parallax, input resolution, and robustness, for instance, can vary with sensing technology, hardware configurations, and environmental conditions.

We have developed a framework for exploring how we could overcome some of these dependencies, by leveraging the higher visual and input resolution of small, coarsely tracked mobile devices for direct, precise, and rapid interaction on large digital displays.

The results from a formal user study show no significant differences in performance when comparing four techniques we developed for a tracked mobile device, where two existing touch-screen techniques served as baselines. The mobile techniques, however, had more consistent performance and smaller variations among participants, and an overall higher user preference in our setup. Our results show the potential of spatially aware handhelds as an interesting complement or substitute for direct touch-interaction on large displays.

REFERENCES

• 1: Pär-Anders Albinsson , Shumin Zhai, High precision touch screen interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642631]
• 2: Rafael Ballagas , Jan Borchers , Michael Rohs , Jennifer G. Sheridan, The Smart Phone: A Ubiquitous Input Device, IEEE Pervasive Computing, v.5 n.1, p.70, January 2006 [doi>10.1109/MPRV.2006.18]
• 3: Patrick Baudisch , Nathaniel Good , Victoria Bellotti , Pamela Schraedley, Keeping things in context: a comparative evaluation of focus plus context screens, overviews, and zooming, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503423]
• 4: Hrvoje Benko , Andrew D. Wilson , Patrick Baudisch, Precise selection techniques for multi-touch screens, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124963]
• 5: Hrvoje Benko , Edward W. Ishak , Steven Feiner, Collaborative Mixed Reality Visualization of an Archaeological Excavation, Proceedings of the 3rd IEEE/ACM International Symposium on Mixed and Augmented Reality, p.132-140, November 02-05, 2004 [doi>10.1109/ISMAR.2004.23]
• 6: George W. Fitzmaurice, Situated information spaces and spatially aware palmtop computers, Communications of the ACM, v.36 n.7, p.39-49, July 1993 [doi>10.1145/159544.159566]
• 7: Robert Hardy , Enrico Rukzio, Touch & interact: touch-based interaction of mobile phones with displays, Proceedings of the 10th international conference on Human computer interaction with mobile devices and services, September 02-05, 2008, Amsterdam, The Netherlands [doi>10.1145/1409240.1409267]
• 8: Hart, S. G., and Staveland, L. E. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Human Mental Workload (1988), 239--250.
• 9: Anders Henrysson , Mark Ollila, UMAR: Ubiquitous Mobile Augmented Reality, Proceedings of the 3rd international conference on Mobile and ubiquitous multimedia, p.41-45, October 27-29, 2004, College Park, Maryland [doi>10.1145/1052380.1052387]
• 10: Steve Hodges , Shahram Izadi , Alex Butler , Alban Rrustemi , Bill Buxton, ThinSight: versatile multi-touch sensing for thin form-factor displays, Proceedings of the 20th annual ACM symposium on User interface software and technology, October 07-10, 2007, Newport, Rhode Island, USA [doi>10.1145/1294211.1294258]
• 11: James R. Lewis, IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use, International Journal of Human-Computer Interaction, v.7 n.1, p.57-78, Jan.-March 1995 [doi>10.1080/10447319509526110]
• 12: I. Scott MacKenzie , Aleks Oniszczak, A comparison of three selection techniques for touchpads, Proceedings of the SIGCHI conference on Human factors in computing systems, p.336-343, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274691]
• 13: Tomer Moscovich , John F. Hughes, Navigating documents with the virtual scroll ring, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029642]
• 14: Alex Olwal , Steven Feiner , Susanna Heyman, Rubbing and tapping for precise and rapid selection on touch-screen displays, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357105]
• 15: Alex Olwal, LightSense: enabling spatially aware handheld interaction devices, Proceedings of the 2006 Fifth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'06), p.119-122, October 22-25, 2006 [doi>10.1109/ISMAR.2006.297802]
• 16: Derek Reilly , Malcolm Rodgers , Ritchie Argue , Mike Nunes , Kori Inkpen, Marked-up maps: combining paper maps and electronic information resources, Personal and Ubiquitous Computing, v.10 n.4, p.215-226, March 2006 [doi>10.1007/s00779-005-0043-6]
• 17: Jun Rekimoto, A multiple device approach for supporting whiteboard-based interactions, Proceedings of the SIGCHI conference on Human factors in computing systems, p.344-351, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274692]
• 18: Jun Rekimoto , Katashi Nagao, The world through the computer: computer augmented interaction with real world environments, Proceedings of the 8th annual ACM symposium on User interface and software technology, p.29-36, November 15-17, 1995, Pittsburgh, Pennsylvania, United States [doi>10.1145/215585.215639]
• 19: Rohs, M., Schöning, J., Krüger, A., and Hecht, B. Towards real-time markerless tracking of magic lenses on paper maps. Adjunct Proc. Pervasive '07 (2007), 69--72.
• 20: Rukzio, E., Broll, G., Leichtenstern, K., and Schmidt, A. Mobile interaction with the real world: An evaluation and comparison of physical mobile interaction techniques. Proc. AmI-07 (2007), 1--18.
• 21: G. M. Smith , m. c. schraefel, The radial scroll tool: scrolling support for stylus- or touch-based document navigation, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029641]
• 22: Wagner, D., Reitmayr, G., Mulloni, A., Drummond, T., and Schmalstieg, D. Pose tracking from natural features on mobile phones. Proc. ISMAR '08 (2008), 125--134.
• 23: Daniel Wagner , Dieter Schmalstieg, First Steps Towards Handheld Augmented Reality, Proceedings of the 7th IEEE International Symposium on Wearable Computers, p.127, October 21-23, 2003
• 24: Andrew D. Wilson , Raman Sarin, BlueTable: connecting wireless mobile devices on interactive surfaces using vision-based handshaking, Proceedings of Graphics Interface 2007, May 28-30, 2007, Montreal, Canada [doi>10.1145/1268517.1268539]
• 25: Jacob O. Wobbrock , Andrew D. Wilson , Yang Li, Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes, Proceedings of the 20th annual ACM symposium on User interface software and technology, October 07-10, 2007, Newport, Rhode Island, USA [doi>10.1145/1294211.1294238]
• 26: Ka-Ping Yee, Peephole displays: pen interaction on spatially aware handheld computers, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642613]

INDEX TERMS

Keywords:
LightSense, MobileButtons, MobileDrag, MobileGesture, MobileRub, interaction technique, mobile, spatially aware, tangible, touch, touch-screen