Implementation of DomCAT: the domain complexity analysis tool for natural language dialog processing

ABSTRACT

While dialog system technology is advancing, there is a lack of theory allowing the vastly different domains for systems to be compared. As a result, all predictions of the cost of building a new dialog system must be made by a dialog expert based on intuition and experience. Recently it has been proposed that entropy can be used as a complexity measure for dialog systems. These calculations would require domain specifications and understanding of information theory. This paper introduces the Domain Complexity Analysis Tool, or DomCAT. With this tool, anyone with basic knowledge of dialog systems can calculate system complexities and create new dialog domain specifications, and dialog complexity calculations can become standard for the field.

REFERENCES

• 1: Allen, J., Ferguson, G., Miller, B., and Ringger, E. Spoken dialogue and interactive planning. Proceedings of the ARPA Spoken Language Systems Technology Workshop (SLST), 1995.
• 2: Madeleine Bates , Damaris Ayuso, A proposal for incremental dialogue evaluation, Proceedings of the workshop on Speech and Natural Language, p.319-322, February 19-22, 1991, Pacific Grove, California [doi>10.3115/112405.112472]
• 3: Charfuelan, M., Lopez, C., Gril J., Rodriguez, C., and Gomez, L. A general framework to assess spoken language dialogue systems: Experience with call center agent systems. Proceedings of the Conference on Natural Language Processing Workshop on Robust Methods in Analysis of Natural Language Data. 2000.
• 4: Frank, G., Guinn, C., and Hubal, R. JUST-TALK: An application of responsive virtual human technology. Proceedings of the 24^th Intersevice/Industrial Training, Simulation, and Education Conference, 2002.
• 5: Frankie James , Manny Rayner , Beth A Hockey, "Do That Again": Evaluating Spoken Dialogue Interfaces, RIACS, 2000
• 6: Keim, G., Fulkerson, M., and Biermann, A. Initiative in tutorial dialogue systems. Proceedings of the American Association for Artificial Intelligence (AAAI) Spring Symposium on Computational Models for Mixed-Initiative Interaction, 1997.
• 7: Shannon Pollard , Ronnie W. Smith, Defining the complexity of natural language dialogue system domains, Duke University, Durham, NC, 2006
• 8: Ronnie W. Smith , D. Richard Hipp, Spoken natural language dialog systems: a practical approach, Oxford University Press, Inc., New York, NY, 1995
• 9: Marilyn A. Walker , Diane J. Litman , Candace A. Kamm , Alicia Abella, PARADISE: a framework for evaluating spoken dialogue agents, Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics, p.271-280, July 07-12, 1997, Madrid, Spain

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing

Additional Classification:
  D. Software
  D.0 GENERAL

  E. Data
  E.4 CODING AND INFORMATION THEORY

General Terms:
Design, Management, Measurement

Keywords:
complexity, dialog processing