Interaction techniques in large display environments using hand-held devices

ABSTRACT

Hand-held devices possess a large potential as an interaction device for their today's ubiquity, and present us with an opportunity to devise new and unique ways of interaction as a smart device with multi-modal sensing and display capabilities. This paper introduces user interaction techniques (for selection, translation, scaling and rotation of objects) using a camera-equipped hand-held device such as a mobile phone or a PDA for large shared environments. We propose three intuitive interaction techniques for 2D and 3D objects in such an environment. The first approach uses the motion flow information to estimate the relative motion of the hand-held device and interact with the large display. The marker-object and marker-cursor approaches both use software markers on the interaction object or on the cursor for the various interactive tasks. The proposed interaction techniques can be further combined with many auxiliary functions and wireless services (of the hand-held devices) for seamless information sharing and exchange among multiple users. A formal usability analysis is currently on-going.

REFERENCES

• 1: Bayon, V. and Griffiths, G. Co-located interaction in virtual environments via de-coupled interfaces. In HCI International 2003. 2003.
• 2: Rafael Ballagas , Michael Rohs , Jennifer G. Sheridan, Sweep and point and shoot: phonecam-based interactions for large public displays, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1056876]
• 3: Madhavapeddy, A., Scott, D., Sharp, R., and Upton, E. Using camera-phones to enhance human-computer interaction. In 6th International Conference on Ubiquitous Computing. 2004.
• 4: Manolis I. A. Lourakis , Antonis A. Argyros, Efficient, causal camera tracking in unprepared environments, Computer Vision and Image Understanding, v.99 n.2, p.259-290, August 2005 [doi>10.1016/j.cviu.2005.02.001]
• 5: Hachet, M. and Kitamura, Y. 3D Interaction With and From Handheld Computers. In IEEE VR 2005 Workshop: New Directions in 3D User Interfaces. 2005.
• 6: Martin Hachet , Joachim Pouderoux , Pascal Guitton, A camera-based interface for interaction with mobile handheld computers, Proceedings of the 2005 symposium on Interactive 3D graphics and games, April 03-06, 2005, Washington, District of Columbia [doi>10.1145/1053427.1053438]
• 7: Kosuke Miyahara , Hiroshi Inoue , Yuji Tsunesada , Masanori Sugimoto, Intuitive manipulation techniques for projected displays of mobile devices, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1056990]
• 8: Bouguet, J.Y. Pyramidal Implementation of the Lucas Kanade Feature Tracker Description of the Algorithm. 2000.
• 9: Shi, J. and Tomasi, C. Good features to track. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 1994. Seattle, USA.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.6 Methodology and Techniques
          Subjects: Interaction techniques

General Terms:
Design, Performance

Keywords:
hand-held device, interaction techniques, large shared display