| Integration and synchronization of input modes during multimodal human-computer interaction |
| Full text |
Pdf
(1.18 MB)
|
| Source
|
Conference on Human Factors in Computing Systems
archive
Proceedings of the SIGCHI conference on Human factors in computing systems
table of contents
Atlanta, Georgia, United States
Pages: 415 - 422
Year of Publication: 1997
ISBN:0-89791-802-9
|
|
Authors
|
|
Sharon Oviatt
|
Center for Human-Computer Communication, Department of Computer Science and Engineering, Oregon Graduate Institute of Science & Technology, P. O. Box 91000, Portland, OR
|
|
Antonella DeAngeli
|
Center for Human-Computer Communication, Department of Computer Science and Engineering, Oregon Graduate Institute of Science & Technology, P. O. Box 91000, Portland, OR
|
|
Karen Kuhn
|
Center for Human-Computer Communication, Department of Computer Science and Engineering, Oregon Graduate Institute of Science & Technology, P. O. Box 91000, Portland, OR and Psychology Dept., University of Trieste, and Linguistics Dept., Portland State University
|
|
| Sponsor |
|
| Publisher |
|
| Bibliometrics |
Downloads (6 Weeks): 8, Downloads (12 Months): 78, Citation Count: 73
|
|
|
REFERENCES
Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.
 |
1
|
|
 |
2
|
|
| |
3
|
Kendon, A. Gesticulation and speech: Two aspects of the process of utterance, The Relationship of Verbal and Nonverbal Communication (ed. by M. Key), The Hague: Mouton, 1980, 207-227.
|
| |
4
|
David B. Koons , Carlton J. Sparrell , Kristinn R. Thorisson, Integrating simultaneous input from speech, gaze, and hand gestures, Intelligent multimedia interfaces, American Association for Artificial Intelligence, Menlo Park, CA, 1993
|
| |
5
|
Levelt, W., Richardson, G. & Heu, W. Pointing and voicing in deictic expressions, Jour. of Memory and Language, 1985, 24, 133-164.
|
| |
6
|
McNeill, D. Hand and Mind: What gestures reveal about thought, Univ. of Chicago Press: Chicago, II1., 1992.
|
| |
7
|
McNeill, D. Language as gesture (Gesture as language), Proc. of the Workshop on the Integration of Gesture in Language & Speech, ed. by L. Messing, Univ. of Delaware, Oct. 1996, 1-20.
|
| |
8
|
Naughton, K. Spontaneous gesture and sign: A study of ASL signs co-occurring with speech, Proc. of the Workshop on the Integration of Gesture in Language & Speech, ed. by L. Messing, Univ. of Delaware, Oct. 1996, 125-34.
|
 |
9
|
|
 |
10
|
|
| |
11
|
Oviatt, S., Cohen, P., Fong, M., & Frank, M. A rapid semi-automatic simulation technique for investigating interactive speech and handwriting, Proc. of the Intl. Conf. on Spoken Language Processing, 1992, 2, 1351-54.
|
| |
12
|
Oviatt, S., Cohen, P. Johnston, M. & Kuhn, K. Multimodal language: Linguistic features and processing requirements, forthcoming.
|
| |
13
|
|
| |
14
|
Oviatt, S. & Olsen, E. Integration themes in multimodal human-computer interaction, Proc. of the lntl Conf. on Spoken Language Processing, 1994, 2, 551-554.
|
| |
15
|
Oviatt, S. L. & vanGent, R. Error resolution during multimodal human-computer interaction, Proc. of the Intl. Conf. on Spoken Language Processing, 1996.
|
| |
16
|
Pittman, J., Cohen, P., Smith, I., Yang, T. & Oviatt, S. Quickset: A multimodal interface for distributed interactive simulations, Proc. of the 6th Conf. on Computer-Generated Forces & Behavior Representation, Univ. of Central Florida, Orlando, FL., 1996, 217-24.
|
CITED BY 73
|
|
|
|
|
Philip R. Cohen , Michael Johnston , David McGee , Sharon Oviatt , Jay Pittman , Ira Smith , Liang Chen , Josh Clow, QuickSet: multimodal interaction for distributed applications, Proceedings of the fifth ACM international conference on Multimedia, p.31-40, November 09-13, 1997, Seattle, Washington, United States
|
|
|
|
|
|
|
|
|
|
|
|
Sandrine Robbe-Reiter , Noëlle Carbonell , Pierre Dauchy, Expression constraints in multimodal human-computer interaction, Proceedings of the 5th international conference on Intelligent user interfaces, p.225-228, January 09-12, 2000, New Orleans, Louisiana, United States
|
|
|
Zouheir Trabelsi , Sung-Hyuk Cha , Darshan Desai , Charles Tappert, A voice and ink XML multimodal architecture for mobile e-commerce systems, Proceedings of the 2nd international workshop on Mobile commerce, September 28-28, 2002, Atlanta, Georgia, USA
|
|
|
Francis Quek , David McNeill , Robert Bryll , Susan Duncan , Xin-Feng Ma , Cemil Kirbas , Karl E. McCullough , Rashid Ansari, Multimodal human discourse: gesture and speech, ACM Transactions on Computer-Human Interaction (TOCHI), v.9 n.3, p.171-193, September 2002
|
|
|
|
|
|
|
|
|
Joyce Y. Chai , Zahar Prasov , Joseph Blaim , Rong Jin, Linguistic theories in efficient multimodal reference resolution: an empirical investigation, Proceedings of the 10th international conference on Intelligent user interfaces, January 10-13, 2005, San Diego, California, USA
|
|
|
Manpreet Kaur , Marilyn Tremaine , Ning Huang , Joseph Wilder , Zoran Gacovski , Frans Flippo , Chandra Sekhar Mantravadi, Where is "it"? Event Synchronization in Gaze-Speech Input Systems, Proceedings of the 5th international conference on Multimodal interfaces, November 05-07, 2003, Vancouver, British Columbia, Canada
|
|
|
Antonella De Angeli , Walter Gerbino , Giulia Cassano , Daniela Petrelli, Visual display, pointing, and natural language: the power of multimodal interaction, Proceedings of the working conference on Advanced visual interfaces, May 24-27, 1998, L'Aquila, Italy
|
|
|
|
|
|
Sharon Oviatt , Rachel Coulston , Stefanie Tomko , Benfang Xiao , Rebecca Lunsford , Matt Wesson , Lesley Carmichael, Toward a theory of organized multimodal integration patterns during human-computer interaction, Proceedings of the 5th international conference on Multimodal interfaces, November 05-07, 2003, Vancouver, British Columbia, Canada
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Michael Johnston , Philip R. Cohen , David McGee , Sharon L. Oviatt , James A. Pittman , Ira Smith, Unification-based multimodal integration, Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics, p.281-288, July 07-12, 1997, Madrid, Spain
|
|
|
David R. McGee , Philip R. Cohen , Sharon Oviatt, Confirmation in multimodal systems, Proceedings of the 36th annual meeting on Association for Computational Linguistics, p.823-829, August 10-14, 1998, Montreal, Quebec, Canada
|
|
|
Philip R. Cohen , Michael Johnston , David McGee , Sharon Oviatt , Jay Pittman , Ira Smith , Liang Chen , Josh Clow, QuickSet: multimodal interaction for simulation set-up and control, Proceedings of the fifth conference on Applied natural language processing, p.20-24, March 31-April 03, 1997, Washington, DC
|
|
|
|
|
|
|
|
|
|
|
|
Helen Meng , P. C. Ching , Shuk Fong Chan , Yee Fong Wong , Cheong Chat Chan, ISIS: an adaptive, trilingual conversational system with interleaving interaction and delegation dialogs, ACM Transactions on Computer-Human Interaction (TOCHI), v.11 n.3, p.268-299, September 2004
|
|
|
Benfang Xiao , Rebecca Lunsford , Rachel Coulston , Matt Wesson , Sharon Oviatt, Modeling multimodal integration patterns and performance in seniors: toward adaptive processing of individual differences, Proceedings of the 5th international conference on Multimodal interfaces, November 05-07, 2003, Vancouver, British Columbia, Canada
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Xugang Wang , Junfeng Li , Xiang Ao , Gang Wang , Guozhong Dai, Multimodal error correction for continuous handwriting recognition in pen-based user interfaces, Proceedings of the 11th international conference on Intelligent user interfaces, January 29-February 01, 2006, Sydney, Australia
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Cristian Hesselman , Pablo Cesar , Ishan Vaishnavi , Mathieu Boussard , Ralf Kernchen , Stefan Meissner , Antonietta Spedalieri , Albert Sinfreu , Christian Räck, Delivering interactive multimedia services in dynamic pervasive computing environments, Proceedings of the 1st international conference on Ambient media and systems, p.1-8, February 11-14, 2008, Quebec, Canada
|
|
|
Paulo Barthelmess , Edward Kaiser , Rebecca Lunsford , David McGee , Philip Cohen , Sharon Oviatt, Human-centered collaborative interaction, Proceedings of the 1st ACM international workshop on Human-centered multimedia, October 27-27, 2006, Santa Barbara, California, USA
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Joyce Y. Chai , Pengyu Hong , Michelle X. Zhou , Zahar Prasov, Optimization in multimodal interpretation, Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, p.1-es, July 21-26, 2004, Barcelona, Spain
|
|
|
Sangyoon Lee , Tian Chen , Jongseo Kim , Gerard Jounghyun Kim , Sung Ho Han , Zhigeng Pan, Effects of tactile augmentation and self-body visualization on affective property evaluation of virtual mobile phone designs, Presence: Teleoperators and Virtual Environments, v.16 n.1, p.45-64, February 2007
|
|
|
|
|
|
Edward C. Kaiser , Paulo Barthelmess , Candice Erdmann , Phil Cohen, Multimodal redundancy across handwriting and speech during computer mediated human-human interactions, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA
|
|
|
|
|
|
|
|
|
Pilar Manchón , Carmen del Solar , Gabriel Amores , Guillermo Pérez, Multimodal interaction analysis in a smart house, Proceedings of the 9th international conference on Multimodal interfaces, November 12-15, 2007, Nagoya, Aichi, Japan
|
|
|
|
|
|
|
|
|
Sharon Oviatt , Phil Cohen , Lizhong Wu , John Vergo , Lisbeth Duncan , Bernhard Suhm , Josh Bers , Thomas Holzman , Terry Winograd , James Landay , Jim Larson , David Ferro, Designing the user interface for multimodal speech and pen-based gesture applications: state-of-the-art systems and future research directions, Human-Computer Interaction, v.15 n.4, p.263-322, December 2000
|
|
|
|
|
|
Barbara Di Eugenio , Davide Fossati , Susan Haller , Dan Yu , Michael Glass, Be Brief, And They Shall Learn: Generating Concise Language Feedback for a Computer Tutor, International Journal of Artificial Intelligence in Education, v.18 n.4, p.317-345, December 2008
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
INDEX TERMS
Primary Classification:
H.
Information Systems
H.5
INFORMATION INTERFACES AND PRESENTATION (I.7)
Additional Classification:
H.
Information Systems
H.1
MODELS AND PRINCIPLES
H.5
INFORMATION INTERFACES AND PRESENTATION (I.7)
General Terms:
Design,
Human Factors,
Management,
Measurement,
Performance,
Theory
Keywords:
dynamic interactive maps,
integration and synchronization,
multimodal interaction,
predictive modeling,
spatial location information,
speech and pen input
|