Jabberwocky: you don't have to be a rocket scientist to change slides for a hydrogen combustion lecture

ABSTRACT

In designing Jabberwocky—a speech-based interface to Microsoft PowerPoint—we have tried to go beyond simple commands like “Next slide, please” and make a tool that aids speakers as they present and even learns as they rehearse their presentations. Jabberwocky looks at the contents of the slides, extracting key words and phrases and associating them with their places in the presentation. By listening for these phrases (and synonymous phrases derived using syntactic rules) Jabberwocky is able to follow along with the presentation, switching slides at the appropriate moments. In this paper, we discuss the implementation of this system—a component of our Intelligent Classroom project—and look at how we are using it.

REFERENCES

• 1: Allen, J., Miller, B., Ringger, E., and Sikorski, T. (1996). Robust Understanding in a Dialogue System. In Proceedings of 34" Meeting of the Association for Computational Linguistics.
• 2: Bradshaw, S., and Hammond, K. (1999). Constructing Indices from Citations in Collections of Research Papers. In Proceedings of the 62"d Annual Meeting of the American Society for Information Science.
• 3: Carroll, L. (1871). Through the Looking Glass.
• 4: Michael H. Coen, Design principles for intelligent environments, Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence, p.547-554, July 1998, Madison, Wisconsin, United States
• 5: Everett, S., Wauchope, K., and Perez-Quinones, M . (1998). Creating Natural Language Interfaces to VR Systems: Experiences, Observations and Lessons Learned. In Proceedings of Fourth International Conference on Virtual Systems and Multimedia.
• 6: David Franklin, Cooperating with people: the intelligent classroom, Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence, p.555-560, July 1998, Madison, Wisconsin, United States
• 7: Franklin, D., and Flachsbart, J. (1998). All gadget and no representation makes Jack a dull environment. In AAAI Spring Symposium on Intelligent Environments. AAAI TR SS-98-02.
• 8: Lucente, M., Zwart, G. and George, A. (1998). Visualization Space: A Testbed for Deviceless Multimodal User Interface. In AAAI Spring Symposium on Intelligent Environments. AAAI TR SS-98-02.
• 9: Heckerman, D., and Horvitz, E. (1998). Inferring Informational Goals from Free-Text Queries: A Bayesian Approach. In Proceedings of Fourteenth Conference on Uncertainty in Artificial Intelligence.
• 10: Wilson, D., and Bradshaw, S. (1999). CBR Textuality. In Proceedings of the Fourth UK Case-Based Reasoning Workshop.
• 11: WordNet. Available and described at http:llwww.- cogsci.princeton.ed&wn/w3wn.html
• 12: Xerox Fart of Speech Tagger. Available at parcftp.xerox.com/pub/tagger/tagger-1-0.tar.z and described at http://kb.rxrc.xerox.com/grenoble/mltt/- fsNLP/tagger.html

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE

General Terms:
Design, Experimentation, Human Factors

Keywords:
approximate natural language understanding, intelligent environments, speech-based user interfaces