Multimodal system processing in mobile environments

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

Multimodal system processing in mobile environments

Full text

Pdf (579 KB)

Source

Symposium on User Interface Software and Technology archive
Proceedings of the 13th annual ACM symposium on User interface software and technology table of contents

San Diego, California, United States

Pages: 21 - 30

Year of Publication: 2000

ISBN:1-58113-212-3

Author

Sharon Oviatt

Department of Computer Science and Engineering, Oregon Graduate Institute of Science and Technology, 20000 N.W. Walker Road, Beaverton, Oregon

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques
SIGSOFT: ACM Special Interest Group on Software Engineering
SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 16, Downloads (12 Months): 90, Citation Count: 21

Additional Information:

references cited by index terms collaborative colleagues

Tools and Actions:	Request Permissions Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/354401.354408 What is a DOI?

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	Clow, J. & Oviatt, S. L. STAMP: A suite of tools for analyzing multimodal system processing, Proceedings of the International Conference on Spoken Language Processing, Sydney: ASSTA Inc., 1998, 2, pp. 277-280.
	2	Cohen, P. R., Cheyer, A., Wang, M., & Baeg, S. C. An open agent architecture. AAAI '94 Spring Symposium Series on Software Agents, Menlo Park: AAAI press, 1994, pp. 1-8.
	3	Philip R. Cohen , Michael Johnston , David McGee , Sharon Oviatt , Jay Pittman , Ira Smith , Liang Chen , Josh Clow, QuickSet: multimodal interaction for distributed applications, Proceedings of the fifth ACM international conference on Multimedia, p.31-40, November 09-13, 1997, Seattle, Washington, United States [doi>10.1145/266180.266328]
	4	Das, S., Bakis, R., Nadas, A., Nahamoo, D. & Picheny, M. Influence of background noise and microphone on the performance of the IBM TANGORA speech recognition system, Proceedings of the IEEE International Conference on Acoustic Speech Signal Processing, 1993, 2, pp. 71-74.
	5	Dreher, J.J. & O'Neill, J.J. Effects of ambient noise on speaker intelligibility for words and phrases, Journal of the Acoustical Society of America, 1957, 29 (12), pp. 1320-1323.
	6	Yifan Gong, Speech recognition in noisy environments: a survey, Speech Communication, v.16 n.3, p.261-291, April 1995 [doi>10.1016/0167-6393(94)00059-J]
	7	Hanley, T.D.& Steer, M.D. Effect of level of distracting noise upon speaking rate, duration and intensity, Journal of Speech and Hearing Disorders, 1949, 14, pp. 363-368.
	8	Paul Iverson , Lynn E. Bernstein , Edward T. Auer, Jr., Modeling the interaction of phonemic intelligibility and lexical structure in audiovisual word recognition, Speech Communication, v.26 n.1-2, p.45-63, Oct. 1998 [doi>10.1016/S0167-6393(98)00049-1]
	9	Michael Johnston , Philip R. Cohen , David McGee , Sharon L. Oviatt , James A. Pittman , Ira Smith, Unification-based multimodal integration, Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics, p.281-288, July 07-12, 1997, Madrid, Spain
	10	Junqua, J.C. The Lombard reflex and its role on human listeners & automatic speech recognizers, Journal of the Acoustical Society of America, 1993, 93 (1), pp.510-24.
	11	The Adaptive Agent Architecture: Achieving Fault-Tolerance Using Persistent Broker Teams, Proceedings of the Fourth International Conference on MultiAgent Systems (ICMAS-2000), p.159, July 10-12, 2000
	12	P. Lockwood , J. Boudy, Experiments with a Nonlinear Spectral Subtractor (NSS), Hidden Markov Models and the projection, for robust speech recognition in cars, Speech Communication, v.11 n.2-3, p.215-228, June 1992 [doi>10.1016/0167-6393(92)90016-Z]
	13	Lombard, E. Le signe de l'elevation de la voix, Annals Maladiers Oreille, Larynx, Nez, Pharynx, 1911, 37, pp. 101-119.
	14	Sharon Oviatt, Ten myths of multimodal interaction, Communications of the ACM, v.42 n.11, p.74-81, Nov. 1999 [doi>10.1145/319382.319398]
	15	Sharon Oviatt, Mutual disambiguation of recognition errors in a multimodel architecture, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.576-583, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303163]
	16	Pick, H.L., Siegel, G.M., Fox, P.W., Garber, S.R. & Kearney, J.K. Inhibiting the Lombard effect, Journal of the Acoustical Society of America, 1989, 85 (2), pp. 894-900.
	17	Potash, L.M. A signal detection problem and a possible solution in Japanese quail, Animal Behavior, 1972, 20, pp. 192-195.
	18	Schulman, R. Articulatory dynamics of loud and normal speech, Journal of the Acoustical Society of America, 1989, 85, pp. 295-312.
	19	Siegel, G.M., Pick, H.L., Olsen, M.G. & Sawin, L. Auditory feedback in the regulation of vocal intensity of preschool children, Developmental Psychology, 1976, 12, pp. 255-261.
	20	Sinott, J.M., Stebbins, W.C. & Moody,D.B. Regulation of voice amplitude by the monkey, Journal of the Acoustical Society of America, 1975, 58, pp. 412-14.
	21	Van Summers, W.V., Pisoni, D.B., Bernacki, R.H., Pedlow, R.I. and Stokes, M.A. Effects of noise on speech production: Acoustic and perceptual analyses, Journal of the Acoustical Society of America, 1988, 84, pp. 917-928.

CITED BY 24

	Joaquim A Jorge, Adaptive tools for the elderly: new devices to cope with age-induced cognitive disabilities, Proceedings of the 2001 EC/NSF workshop on Universal accessibility of ubiquitous computing: providing for the elderly, May 22-25, 2001, Alcácer do Sal, Portugal
	Joyce Y. Chai , Zahar Prasov , Joseph Blaim , Rong Jin, Linguistic theories in efficient multimodal reference resolution: an empirical investigation, Proceedings of the 10th international conference on Intelligent user interfaces, January 10-13, 2005, San Diego, California, USA
	Sharon Oviatt, Multimodal interfaces, The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 2002
	Hicham Djenidi , Amar Ramdane-Cherif , Chakib Tadj , Nicole Levy, Architectures multi-agents génériques à base de réseaux de Pétri colorés temporisés pour la fusion multimodale en entrée, Proceedings of the 14th French-speaking conference on Human-computer interaction (Conférence Francophone sur l'Interaction Homme-Machine), p.33-40, November 26-29, 2002, Poitiers, France
	Joyce Y. Chai , Pengyu Hong , Michelle X. Zhou, A probabilistic approach to reference resolution in multimodal user interfaces, Proceedings of the 9th international conference on Intelligent user interface, January 13-16, 2004, Funchal, Madeira, Portugal
	Sharon Oviatt, Advances in Robust Multimodal Interface Design, IEEE Computer Graphics and Applications, v.23 n.5, p.62-68, September 2003
	André D. Milota, Modality fusion for graphic design applications, Proceedings of the 6th international conference on Multimodal interfaces, October 13-15, 2004, State College, PA, USA
	Sharon Oviatt , Rachel Coulston , Rebecca Lunsford, When do we interact multimodally?: cognitive load and multimodal communication patterns, Proceedings of the 6th international conference on Multimodal interfaces, October 13-15, 2004, State College, PA, USA
	Matthias Jöst , Jochen Häußler , Matthias Merdes , Rainer Malaka, Multimodal interaction for pedestrians: an evaluation study, Proceedings of the 10th international conference on Intelligent user interfaces, January 10-13, 2005, San Diego, California, USA
	Mathias Koelsch , Ryan Bane , Tobias Hoellerer , Matthew Turk, Multimodal Interaction with a Wearable Augmented Reality System, IEEE Computer Graphics and Applications, v.26 n.3, p.62-71, May 2006
	Moustapha Zouinar , Marc Relieu , Pascal Salembier , Guillaume Calvet, Observation and capture of multimodal interaction in mobile situations, Proceedings of the 1st French-speaking conference on Mobility and ubiquity computing, June 01-03, 2004, Nice, France
	Sharon Oviatt, Advances in the robust processing of multimodal speech and pen systems, Multimodal interface for human-machine communication, World Scientific Publishing Co., Inc., River Edge, NJ, 2002
	Guillaume Calvet , Pascal Salembier , Julien Kahn , Moustapha Zouinar, Etude empirique de l'interaction multimodale en mobilité: approche méthodologique et premiers résultats, Proceedings of the 17th conference on 17ème Conférence Francophone sur l'Interaction Homme-Machine, p.283-286, September 27-30, 2005, Toulouse, France
	Sanjeev Kumar , Philip R. Cohen , Rachel Coulston, Multimodal interaction under exerted conditions in a natural field setting, Proceedings of the 6th international conference on Multimodal interfaces, October 13-15, 2004, State College, PA, USA
	Sharon Oviatt, Designing robust multimodal systems for universal access, Proceedings of the 2001 EC/NSF workshop on Universal accessibility of ubiquitous computing: providing for the elderly, May 22-25, 2001, Alcácer do Sal, Portugal
	Marcos Serrano , Laurence Nigay , Rachel Demumieux , Jérôme Descos , Patrick Losquin, Multimodal interaction on mobile phones: development and evaluation using ACICARE, Proceedings of the 8th conference on Human-computer interaction with mobile devices and services, September 12-15, 2006, Helsinki, Finland
	H. Djenidi , S. Benarif , A. Ramdane-Cherif , C. Tadj , N. Levy, Generic multimedia multimodal agents paradigms and their dynamic reconfiguration at the architectural level, EURASIP Journal on Applied Signal Processing, v.2004 n.1, p.1688-1707, 1 January 2004
	Mathias Koelsch , Ryan Bane , Tobias Hoellerer , Matthew Turk, Multimodal Interaction with a Wearable Augmented Reality System, IEEE Computer Graphics and Applications, v.26 n.3, p.62-71, May 2006
	D. Wilson , J. Doyle , J. Weakliam , M. Bertolotto , D. Lynch, Personalised maps in multimodal mobile GIS, International Journal of Web Engineering and Technology, v.3 n.2, p.196-216, January 2007
	Sharon Oviatt , Phil Cohen , Lizhong Wu , John Vergo , Lisbeth Duncan , Bernhard Suhm , Josh Bers , Thomas Holzman , Terry Winograd , James Landay , Jim Larson , David Ferro, Designing the user interface for multimodal speech and pen-based gesture applications: state-of-the-art systems and future research directions, Human-Computer Interaction, v.15 n.4, p.263-322, December 2000
	Andreas Ratzka, Explorative studies on multimodal interaction in a PDA- and desktop-based scenario, Proceedings of the 10th international conference on Multimodal interfaces, October 20-22, 2008, Chania, Crete, Greece
	Tim Paek , Bo Thiesson , Yun-Cheng Ju , Bongshin Lee, Search Vox: leveraging multimodal refinement and partial knowledge for mobile voice search, Proceedings of the 21st annual ACM symposium on User interface software and technology, October 19-22, 2008, Monterey, CA, USA
	Joyce Y. Chai , Zahar Prasov , Shaolin Qu, Cognitive principles in robust multimodal interpretation, Journal of Artificial Intelligence Research, v.27 n.1, p.55-83, September 2006
	Pui-Yu Hui , Helen M. Meng, Cross-modality semantic integration with hypothesis rescoring for robust interpretation of multimodal user interactions, IEEE Transactions on Audio, Speech, and Language Processing, v.17 n.3, p.486-500, March 2009