Exploiting referential context in spoken language interfaces for data-poor domains

ABSTRACT

This paper describes an implementation of a shell-like programming interface that utilizes referential context (that is, information about the current state of an interfaced application) in order to achieve accurate recognition -- even in user-defined domains with no available domain-specific training corpora. The interface incorporates a knowledge of context into its model of syntax, yielding a referential semantic language model. Interestingly, the referential semantic language model exploits context dynamically, unlike other recent systems, by using incremental processing and the limited stack memory of an HMM-like time series model.

REFERENCES

• 1: G. Aist, J. Allen, E. Campana, C. Gallo, S. Stoness, M. Swift, and M. Tanenhaus. Incremental understanding in human-computer dialogue and experimental evidence for advantages over nonincremental methods. In Proc. DECALOG, pages 149--154, 2007.
• 2: J. Baker. The dragon system: an overivew. IEEE Transactions on Acoustics, Speech and Signal Processing, 23(1):24--29, 1975.
• 3: R. J. Brachman and J. G. Schmolze. An overview of the kl-one knolewdge representation system. Cognitive Science, 9(2):171--216, Apr. 1985.
• 4: G. Bugmann, E. Klein, S. Lauria, and T. Kyriacou. Corpus-based robotics: A route instruction example. In Proceedings of Intelligent Autonomous Systems, pages 96--103, 2004.
• 5: Ciprian Chelba , Frederick Jelinek, Exploiting syntactic structure for language modeling, Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, August 10-14, 1998, Montreal, Quebec, Canada
• 6: G. Chung, S. Seneff, C. Wang, and I. Hetherington. A dynamic vocabulary spoken dialogue interface. In Proc. ICSLP, 2004.
• 7: A. Church. A formulation of the simple theory of types. Journal of Symbolic Logic, 5(2):56--68, 1940.
• 8: D. DeVault and M. Stone. Domain inference in incremental interpretation. In Proc. ICoS, 2003.
• 9: W. M. Fisher, G. R. Doddington, and K. M. Goudie-Marshall. The darpa speech recognition research database: Specifications and status. In Proceedings of DARPA Workshop on Speech Recognition, pages 93--99, Feb. 1986.
• 10: W. M. Fisher, V. Zue, J. Bernstein, and D. S. Pallet. An acoustic-phonetic data base. Journal of the Acoustical Society of America, 81:S92--S93, 1987.
• 11: P. Gorniak and D. Roy. Grounded semantic composition for visual scenes. Journal of Artificial Intelligence Research, 21:429--470, 2004.
• 12: Jerry R. Hobbs, Ontological promiscuity, Proceedings of the 23rd annual meeting on Association for Computational Linguistics, p.60-69, July 08-12, 1985, Chicago, Illinois [doi>10.3115/981210.981218]
• 13: F. Jelinek, L. R. Bahl, and R. L. Mercer. Design of a linguistic statistical decoder for the recognition of continuous speech. IEEE Transactions on Information Theory, 21:250--256, 1975.
• 14: Oliver Lemon , Alexander Gruenstein, multithreaded context for robust conversational interfaces: Context-sensitive speech recognition and interpretation of corrective fragments, ACM Transactions on Computer-Human Interaction (TOCHI), v.11 n.3, p.241-267, September 2004 [doi>10.1145/1017494.1017496]
• 15: K. P. Murphy and M. A. Paskin. Linear time inference in hierarchical HMMs. In Proc. NIPS, pages 833--840, 2001.
• 16: T. Robinson. An application of recurrent nets to phone probability estimation. In IEEE Transactions on Neural Networks, 1994.
• 17: D. Roy and N. Mukherjee. Towards situated speech understanding: visual context priming of language models. Computer Speech & Language, 19(2):227--248, 2005.
• 18: Paul E. Rybski , Kevin Yoon , Jeremy Stolarz , Manuela M Veloso, Interactive robot task training through dialog and demonstration, Proceedings of the ACM/IEEE international conference on Human-robot interaction, March 10-12, 2007, Arlington, Virginia, USA [doi>10.1145/1228716.1228724]
• 19: William Schuler, Computational properties of environment-based disambiguation, Proceedings of the 39th Annual Meeting on Association for Computational Linguistics, p.474-481, July 06-11, 2001, Toulouse, France [doi>10.3115/1073012.1073073]
• 20: A. Tarski. The concept of truth in the languages of the deductive sciences (polish). Prace Towarzystwa Naukowego Warszawskiego, Wydzial III Nauk Matematyczno-Fizycznych, 34, 1933. translated as 'The concept of truth in formalized languages', in: J. Corcoran (Ed.), Logic, Semantics, Metamathematics: papers from 1923 to 1938, Hackett Publishing Company, Indianapolis, IN, 1983, pp. 152--278.
• 21: Geoffrey Zweig , Stuart Russell, Speech recognition with dynamic Bayesian networks, Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence, p.173-180, July 1998, Madison, Wisconsin, United States

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Natural language

Additional Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Time series analysis

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Speech recognition and synthesis

General Terms:
Design, Human Factors