XWand: UI for intelligent spaces

ABSTRACT

The XWand is a novel wireless sensor package that enables styles of natural interaction with intelligent environments. For example, a user may point the wand at a device and control it using simple gestures. The XWand system leverages the intelligence of the environment to best determine the user's intention. We detail the hardware device, signal processing algorithms to recover position and orientation, gesture recognition techniques, a multimodal (wand and speech) computational architecture and a preliminary user study examining pointing performance under conditions of tracking availability and audio feedback.

REFERENCES

• 1: Brummitt, B., and JJ Cadiz. "Let There Be Light": Examining Interfaces for Homes of the Future. in Human-Computer Interaction: INTERACT '01. 2001.
• 2: Michael Patrick Johnson , Andrew Wilson , Bruce Blumberg , Christopher Kline , Aaron Bobick, Sympathetic interfaces: using a plush toy to direct synthetic characters, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.152-158, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303028]
• 3: Teresa Marrin, Possibilities for the digital baton as a general-purpose gestural interface, CHI '97 extended abstracts on Human factors in computing systems: looking to the future, March 22-27, 1997, Atlanta, Georgia [doi>10.1145/1120212.1120409]
• 4: Toshiyuki Masui , Itiro Siio, Real-World Graphical User Interfaces, Proceedings of the 2nd international symposium on Handheld and Ubiquitous Computing, p.72-84, September 25-27, 2000, Bristol, UK
• 5: Colin Swindells , Kori M. Inkpen , John C. Dill , Melanie Tory, That one there! Pointing to establish device identity, Proceedings of the 15th annual ACM symposium on User interface software and technology, October 27-30, 2002, Paris, France [doi>10.1145/571985.572007]
• 6: Nissanka B. Priyantha , Anit Chakraborty , Hari Balakrishnan, The Cricket location-support system, Proceedings of the 6th annual international conference on Mobile computing and networking, p.32-43, August 06-11, 2000, Boston, Massachusetts, United States [doi>10.1145/345910.345917]
• 7: Cliff Randell , Henk L. Muller, Low Cost Indoor Positioning System, Proceedings of the 3rd international conference on Ubiquitous Computing, p.42-48, September 30-October 02, 2001, Atlanta, Georgia, USA
• 8: Nebojsa Jojic , Thomas Huang , Barry Brumitt , Brian Meyers , Steve Harris, Detection and Estimation of Pointing Gestures in Dense Disparity Maps, Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000, p.468, March 26-30, 2000
• 9: Berthold K. Horn, Robot Vision, McGraw-Hill Higher Education, 1986
• 10: MacNeil, D., Hand and Mind. 1992: University of Chicago Press.
• 11: Sharon Oviatt, Taming recognition errors with a multimodal interface, Communications of the ACM, v.43 n.9, p.45-51, Sept. 2000 [doi>10.1145/348941.348979]
• 12: Judea Pearl, Probabilistic reasoning in intelligent systems: networks of plausible inference, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1988
• 13: Dan R. Olsen, Jr. , Travis Nielsen, Laser pointer interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.17-22, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365030]
• 14: Brad A. Myers , Rishi Bhatnagar , Jeffrey Nichols , Choon Hong Peck , Dave Kong , Robert Miller , A. Chris Long, Interacting at a distance: measuring the performance of laser pointers and other devices, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503383]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Prototyping

Additional Classification:
  B. Hardware
  B.4 INPUT/OUTPUT AND DATA COMMUNICATIONS
      B.4.2 Input/Output Devices
          Subjects: Channels and controllers; Voice

  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Voice I/O; Input devices and strategies (e.g., mouse, touchscreen)

  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.4 Applications
          Subjects: Computer vision
      I.5.5 Implementation
          Subjects: Interactive systems; Special architectures

General Terms:
Algorithms, Design, Human Factors

Keywords:
Bayesian networks, accelerometer, gesture recognition, intelligent environments, magnetometer, multimodal interfaces, sensing