A virtual space teleconferencing system that supports intuitive interaction for creative and cooperative work

ABSTRACT

Since their advent, interaction with computers has been a very fascinating field of research. Though, we have come a long way from turning knobs and punching cards to using keyboards and pointing devices, natural language interaction has not seen widespread use as a general means of interaction. The thesis of this paper is that some application fields, specially those dealing with computer graphics, can benefit from the interaction of natural language and hand gestures. This bimodal means of interaction in computer-graphics-based task, complements the deficiencies of just applying either one of the two modes. This paper, describes current research efforts taking place at ATR for combining hand gestures and verbal descriptions for generating, modifying and manipulating 3D computer graphics objects.

REFERENCES

• 1: Altman, Edward .I., Normal Form Analysis of Chua's Circl,it with Applications for Trajectory Recognition. In IEEE Transactions on Circuit and Systems, volume 40, pp. 675-682. IEEE, October 1993.
• 2: Gardiner, M. The superellipse: a curve between the ellipse anti the rectangle. In Scientific A mcrican, volume 213, pp. 222-234. 1965.
• 3: Ishibuchi, Koichi., Takemura, Haruo., and Kishino, Fumio. Re'M-time hand gesture recognition using a 3-D prediction model. In Proceedings of IEEE International Conference on Systems, Man and Cybernetics, pp. 324- 328, 1993.
• 4: Johansson, G. Configurations in Event Perception. Almqvist and Wiksell, Stockholm, 1950.
• 5: Kishino, Fumio., Ohya, .lun., 'l'akemura, Haruo., and Terashima, Nobuyoshi. Virtual space teleconfcrcncing system-Real-time detection and reproduction of 3-D human images. In Proceedings of HCI International '.93, pp. 669-674, 1991.
• 6: Mochizuki, Kenji., Takemura, Haruo., and Kishino, Fumio. Object manipulation and layout in a 3-D virtual space using a combination of natural language and hand pointing. In Proceedings of 4th Sapporo International Computer Graphics Symposium, HW-I, pp. 38- 4'9 199')
• 7: Nishihara, H. Intensity, visible-surface and volumetric representations. In Artificial Intelligence, volume 28, pp. 293-331. 1981.
• 8: Ohya, J,ln., Kit.amura, Yas,ichi., T~tkemura, }laruo., Kishino, Fumio., and Terashima, Nobuyoshi. Realtime reproduction of 3D human images in Virtual Space Teleconferencing. In Proceedings of IEEE Virtual Reality Annual International Symposium, pp. 408--414, September 1993.
• 9: Rosch, E. On the internal structure of perceptual and semantic categories. In Moore, T.E., editor, Congnitit,e Det,eloprne:nt and the A cquizition of Language. Academic Press, New York, 1973.
• 10: Sagayama, S., Takami, J., N agai, A., Singer, H., Yamaguchi, K., Ohkura, K., Kita, K., and Kurematsu, A. ATREUS: a Speech Recognition Front-end for a Speech Translation System. In Proceedings of EURO SP 1993, pp. 1287-1290, 1993.
• 11: A. Smith , Y. Kitamura , H. Takemura , F. Kishino, A simple and efficient method for accurate collision detection among deformable polyhedral objects in arbitrary motion, Proceedings of the Virtual Reality Annual International Symposium (VRAIS'95), p.136, March 11-15, 1995
• 12: Stevens, S. Patterns in Nature. Brown Books, Boston, MA., 1974.
• 13: Takahashi, Tomoichi., Hakata, A., Kobayashi, Yukio., and Yamashita, K. User interface using language and visual information. In Proceedings of Computer World'88, Kobc, Japan, pp. 192-199, 1988.
• 14: Demetri Terzopoulos , Dimitri Metaxas, Dynamic 3D Models with Local and Global Deformations: Deformable Superquadrics, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.13 n.7, p.703-714, July 1991 [doi>10.1109/34.85659]
• 15: Tetsutani, Nobuji., Nagashima, Yoshio., Tomono, Akira., and Kishino, Fumio. Stereoscopic display method employing eye-position tracking. In Proceedings of International Symposium on Three Dimensional Image Technology and Arts(Tokyo, Japan, February 5- 7, 199~2), pp. 101-107, 1992.
• 16: Thompson, l). A. On Growth and Form. U.K., 1942. 2nd ed.
• 17: Tijerino, Yuri A., Mochizuki, Kenji., and Kishino, Fumio. Interactive 3-D Computer Graphics Driven through Verbal Instructions: Previous and Current Activities at A'I'R. In Comput. ~ Graphics., volume 18, pp. 621-631. Elsevier Science, Great Britain, 1994.
• 18: Tijcrino, Yuri A. "WYSIWYS"-Interactive generation, manipulation and modification of 3-D shapes based on verbal descriptions. In A1 Review Journal, volume 8. Kluwer Academic. unpublished.
• 19: D. Weimer , S. K. Ganapathy, A synthetic visual environment with hand gesturing and voice input, Proceedings of the SIGCHI conference on Human factors in computing systems: Wings for the mind, p.235-240, March 1989
• 20: Wertheimer, M. Laws of organization in perceptual forms. In W. D., Ellis, editor, Gestalt Psychology. Harcourt Brace, New York, 1923.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Virtual reality

Additional Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.3 Communications Applications
          Subjects: Computer conferencing, teleconferencing, and videoconferencing
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces
          Subjects: Synchronous interaction

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.1 Applications and Expert Systems
          Subjects: Natural language interfaces
      I.2.7 Natural Language Processing
  I.3 COMPUTER GRAPHICS
      I.3.1 Hardware architecture
          Subjects: Input devices

General Terms:
Algorithms, Human Factors, Languages