Boom chameleon: simultaneous capture of 3D viewpoint, voice and gesture annotations on a spatially-aware display

ABSTRACT

We introduce the Boom Chameleon, a novel input/output device consisting of a flat-panel display mounted on a tracked mechanical boom. The display acts as a physical window into 3D virtual environments, through which a one-to-one mapping between real and virtual space is preserved. The Boom Chameleon is further augmented with a touch-screen and a microphone/speaker combination. We present a 3D annotation application that exploits this unique configuration in order to simultaneously capture viewpoint, voice and gesture information. Design issues are discussed and results of an informal user study on the device and annotation software are presented. The results show that the Boom Chameleon annotation facilities have the potential to be an effective, easy to learn and operate 3D design review system.

REFERENCES

• 1: Art+Com. http://www.artcom.de.
• 2: Ravin Balakrishnan , George W. Fitzmaurice , Gordon Kurtenbach, User Interfaces for Volumetric Displays, Computer, v.34 n.3, p.37-45, March 2001 [doi>10.1109/2.910892]
• 3: Mark Billinghurst , Hirokazu Kato, Collaborative augmented reality, Communications of the ACM, v.45 n.7, July 2002 [doi>10.1145/514236.514265]
• 4: Richard A. Bolt, “Put-that-there”: Voice and gesture at the graphics interface, ACM SIGGRAPH Computer Graphics, v.14 n.3, p.262-270, July 1980
• 5: Bourguignon, D., Cani, M-P. and Drettakis, G. (2001). Drawing for Illustration and Annotation in 3D, In Proc. of EUROGRAPHICS'01, 114-122.
• 6: William Buxton, Less is more (more is less), The invisible future: the seamless integration of technology into everyday life, McGraw-Hill, Inc., New York, NY, 2001
• 7: Bill Buxton , George W. Fitzmaurice, HMDs, caves & chameleon: a human-centric analysis of interaction in virtual space, ACM SIGGRAPH Computer Graphics, v.32 n.4, p.69-74, Nov. 1998 [doi>10.1145/307710.307732]
• 8: Barbara L. Chalfonte , Robert S. Fish , Robert E. Kraut, Expressive richness: a comparison of speech and text as media for revision, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.21-26, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108848]
• 9: Philip Cohen , David McGee , Sharon Oviatt , Lizhong Wu , Joshua Clow , Robert King , Simon Julier , Lawrence Rosenblum, Multimodal Interaction for 2D and 3D Environments, IEEE Computer Graphics and Applications, v.19 n.4, p.10-13, July 1999 [doi>10.1109/38.773958]
• 10: Fakespace Labs. http://www.fakespacelabs.com.
• 11: George W. Fitzmaurice, Situated information spaces and spatially aware palmtop computers, Communications of the ACM, v.36 n.7, p.39-49, July 1993 [doi>10.1145/159544.159566]
• 12: George W. Fitzmaurice , Shumin Zhai , Mark H. Chignell, Virtual reality for palmtop computers, ACM Transactions on Information Systems (TOIS), v.11 n.3, p.197-218, July 1993 [doi>10.1145/159161.159160]
• 13: David Frohlich , Owen Daly-Jones, Voicefax: a shared workspace for voicemail partners, Conference companion on Human factors in computing systems, p.308-309, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223355.223688]
• 14: Gary Hardock , Gordon Kurtenbach , William Buxton, A marking based interface for collaborative writing, Proceedings of the 6th annual ACM symposium on User interface software and technology, p.259-266, December 1993, Atlanta, Georgia, United States [doi>10.1145/168642.168669]
• 15: Reid Harmon , Walter Patterson , William Ribarsky , Jay Bolter, The Virtual Annotation System, Proceedings of the 1996 Virtual Reality Annual International Symposium (VRAIS 96), p.239, March 30-April 03, 1996
• 16: Iris Annotator. http://www.sgi.com/software/annotator.
• 17: Thomas Jung , Mark D. Gross , Ellen Yi-Luen Do, Annotating and sketching on 3D web models, Proceedings of the 7th international conference on Intelligent user interfaces, January 13-16, 2002, San Francisco, California, USA [doi>10.1145/502716.502733]
• 18: LaViola, J., Holden, A., Forsberg, D. and Bhuphaibool, D. (1998). Collaborative Conceptual Modeling Using the SKETCH Framework, In Proc. of the first IASTED International Conference on Computer Graphics and Imaging, 154-158.
• 19: Stephen R. Levine , Susan F. Ehrlich, The Freestyle system: a design perspective, Human-computer interaction: toward the year 2000, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1995
• 20: Rainer Lienhart, A System for Effortless Content Annotation to Unfold the Semantics in Videos, Proceedings of the IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'00), p.45, June 16-16, 2000
• 21: Maria M. Loughlin , John F. Hughes, An annotation system for 3D fluid flow visualization, Proceedings of the conference on Visualization '94, October 17-21, 1994, Washinton, D.C.
• 22: Maeda, J., Greenwold, S. Installation. Aesthetics and Computation Group, MIT. http://acg.media.mit.edu/people/simong/installationNew/cover.html.
• 23: Catherine C. Marshall, Annotation: from paper books to the digital library, Proceedings of the second ACM international conference on Digital libraries, p.131-140, July 23-26, 1997, Philadelphia, Pennsylvania, United States [doi>10.1145/263690.263806]
• 24: Matsuoka, H., Onozawa, A., Hosoya, E., Sato, H., and Nojima, H. (2002). Regeneration of Real Objects in the Real World. Presented at SIGGRAPH Symposium on Interactive 3D Graphics 2002 - Emerging technologies. http://kankyo.lelab.ecl.ntt.co.jp/real_in_real/eng/index.htm.
• 25: Kenton O'Hara , Abigail Sellen, A comparison of reading paper and on-line documents, Proceedings of the SIGCHI conference on Human factors in computing systems, p.335-342, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258787]
• 26: Sharon Oviatt, Ten myths of multimodal interaction, Communications of the ACM, v.42 n.11, p.74-81, Nov. 1999 [doi>10.1145/319382.319398]
• 27: Elin Rønby Pedersen , Kim McCall , Thomas P. Moran , Frank G. Halasz, Tivoli: an electronic whiteboard for informal workgroup meetings, Proceedings of the SIGCHI conference on Human factors in computing systems, p.391-398, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169309]
• 28: I. Poupyrev , N. Tomokazu , S. Weghorst, Virtual Notepad: Handwriting in Immersive VR, Proceedings of the Virtual Reality Annual International Symposium, p.126, March 14-18, 1998
• 29: Jun Rekimoto , Katashi Nagao, The world through the computer: computer augmented interaction with real world environments, Proceedings of the 8th annual ACM symposium on User interface and software technology, p.29-36, November 15-17, 1995, Pittsburgh, Pennsylvania, United States [doi>10.1145/215585.215639]
• 30: Dieter Schmalstieg , L. Miguel Encarnação , Zsolt Szalavári, Using transparent props for interaction with the virtual table, Proceedings of the 1999 symposium on Interactive 3D graphics, p.147-153, April 26-29, 1999, Atlanta, Georgia, United States [doi>10.1145/300523.300542]
• 31: David Small , Hiroshi Ishii, Design of spatially aware graspable displays, CHI '97 extended abstracts on Human factors in computing systems: looking to the future, March 22-27, 1997, Atlanta, Georgia [doi>10.1145/1120212.1120437]
• 32: Stellingwerff, Martijn. (1999). SketchBoX. In Proc. of Architectural Computing from Turing to 2000. eCAADe, ISBN 0-9523687-5-7, Liverpool (UK) 15-17 September 1999, 491-497.
• 33: Richard Stoakley , Matthew J. Conway , Randy Pausch, Virtual reality on a WIM: interactive worlds in miniature, Proceedings of the SIGCHI conference on Human factors in computing systems, p.265-272, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223938]
• 34: Karon Weher , Alex Poon, Marquee: a tool for real-time video logging, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.58-64, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191697]
• 35: Steve Whittaker , Richard Davis , Julia Hirschberg , Urs Muller, Jotmail: a voicemail interface that enables you to see what was said, Proceedings of the SIGCHI conference on Human factors in computing systems, p.89-96, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332411]
• 36: Steve Whittaker , Patrick Hyland , Myrtle Wiley, FILOCHAT: handwritten notes provide access to recorded conversations, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.271-277, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191763]
• 37: Lynn D. Wilcox , Bill N. Schilit , Nitin Sawhney, Dynomite: a dynamically organized ink and audio notebook, Proceedings of the SIGCHI conference on Human factors in computing systems, p.186-193, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258700]
• 38: WindowsVR. http://www.abs-tech.com/Produtos/3D_VR/VR-Hardware/hmds1/Virtual_Research/win_vr.html.

INDEX TERMS

Primary Classification:
  F. Theory of Computation
  F.1 COMPUTATION BY ABSTRACT DEVICES
      F.1.2 Modes of Computation
          Subjects: Interactive and reactive computation

Additional Classification:
  B. Hardware
  B.4 INPUT/OUTPUT AND DATA COMMUNICATIONS
      B.4.2 Input/Output Devices
          Subjects: Image display; Voice

  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Artificial, augmented, and virtual realities
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)
      H.5.3 Group and Organization Interfaces
          Subjects: Synchronous interaction

  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.1 Hardware architecture
          Subjects: Input devices
      I.3.4 Graphics Utilities
          Subjects: Virtual device interfaces

General Terms:
Design, Experimentation, Human Factors

Keywords:
3D navigation, annotation, gesture, spatially-aware display, voice