A look under the hood: design and development of the first SmartWeb system demonstrator

ABSTRACT

Experience shows that decisions in the early phases of the development of a multimodal system prevail throughout the life-cycle of a project. The distributed architecture and the requirement for robust multimodal interaction in our project SmartWeb resulted in an approach that uses and extends W3C standards like EMMA and RDFS. These standards for the interface structure and content allowed us to integrate available tools and techniques. However, the requirements in our system called for various extensions, e.g., to introduce result feedback tags for an extended version of EMMA. The interconnection framework depends on a commercial telephone voice dialog system platform for the dialog-centric components while the information access processes are linked using web service technology. Also in the area of this underlying infrastructure, enhancements and extensions were necessary. The first demonstration system is operable now and will be presented at the Football World Cup 2006 in Germany.

REFERENCES

• 1: J. Alexandersson and T. Becker. The Formal Foundations Underlying Overlay. In Proceedings of the Fifth International Workshop on Computational Semantics (IWCS-5), Tilburg, The Netherlands, 2003.
• 2: J. Alexandersson, T. Becker, R. Engel, M. Löckelt, E. Pecourt, P. Poller, N. Pfleger, and N. Reithinger. Ends-based dialogue processing. In Proceedings, Second International Workshop on Scalable Natural Language Understanding, Boston, 2004.
• 3: J. Alexandersson, R. Engel, M. Kipp, S. Koch, U. Küssner, N. Reithinger, and M. Stede. Modeling negotiation dialogs. In W. Wahlster, editor, Verbmobil: Foundations of Speech-to-Speech Translation, pages 441 -- 451. Springer, 2000.
• 4: R. Amant and C. Healey. Usability guidelines for interactive search in direct manipulation systems. In Proceedings of The International Joint Conference on Artificial Intelligence (IJCAI), 2001.
• 5: Kalina Bontcheva , Valentin Tablan , Diana Maynard , Hamish Cunningham, Evolving GATE to meet new challenges in language engineering, Natural Language Engineering, v.10 n.3-4, p.349-373, September 2004 [doi>10.1017/S1351324904003468]
• 6: Adam Cheyer , David Martin, The Open Agent Architecture, Autonomous Agents and Multi-Agent Systems, v.4 n.1-2, p.143-148, March-June 2001 [doi>10.1023/A:1010091302035]
• 7: R. Engel. SPIN: Language understanding for spoken dialogue systems using a production system approach. In Proceedings of 7th International Conference on Spoken Language Processing (ICSLP-2002), pages 2717--2720, Denver, Colorado, USA, 2002.
• 8: Dieter Fensel , Wolfgang Wahlster , Henry Lieberman, Spinning the Semantic Web: Bringing the World Wide Web to Its Full Potential, MIT Press, Cambridge, MA, 2002
• 9: L. Gorlenko , R. Merrick, No wires attached: Usability challenges in the connected mobile world, IBM Systems Journal, v.42 n.4, p.639-651, October 2003
• 10: N. Guarino and C. A. Welty. An overview of ontoclean. In S. Staab and R. Studer, editors, Handbook on Ontologies, International Handbooks on Information Systems, pages 151--172. Springer, 2004.
• 11: Gerd Herzog , Alassane Ndiaye , Stefan Merten , Heinz Kirchmann , Tilman Becker , Peter Poller, Large-scale software integration for spoken language and multimodal dialog systems, Natural Language Engineering, v.10 n.3-4, p.283-305, September 2004 [doi>10.1017/S1351324904003444]
• 12: Glenn E. Krasner , Stephen T. Pope, A cookbook for using the model-view controller user interface paradigm in Smalltalk-80, Journal of Object-Oriented Programming, v.1 n.3, p.26-49, Aug./Sept. 1988
• 13: Roman Longoria, Designing Software for the Mobile Context: A Practitioner's Guide (Computer Communications and Networks), SpringerVerlag, 2004
• 14: Mark T. Maybury , Wolfgang Wahlster, Readings in intelligent user interfaces, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1998
• 15: Sharon Oviatt, Ten myths of multimodal interaction, Communications of the ACM, v.42 n.11, p.74-81, Nov. 1999 [doi>10.1145/319382.319398]
• 16: Norbert Reithinger , Jan Alexandersson , Tilman Becker , Anselm Blocher , Ralf Engel , Markus Löckelt , Jochen Müller , Norbert Pfleger , Peter Poller , Michael Streit , Valentin Tschernomas, SmartKom: adaptive and flexible multimodal access to multiple applications, Proceedings of the 5th international conference on Multimodal interfaces, November 05-07, 2003, Vancouver, British Columbia, Canada [doi>10.1145/958432.958454]
• 17: N. Reithinger, D. Fedeler, A. Kumar, C. Lauer, E. Pecourt, and L. Romary. MIAMM - A Multimodal Dialogue System Using Haptics. In J. van Kuppevelt, L. Dybkjaer, and N. O. Bernsen, editors, Advances in Natural Multimodal Dialogue Systems. Springer, 2005.
• 18: S. Seneff, R. Lau, and J. Polifroni. Organization, Communication, and Control in the Galaxy-II Conversational System. In Proc. of Eurospeech'99, pages 1271--1274, Budapest, Hungary, 1999.
• 19: Kristinn R. Thórisson , Christopher Pennock , Thor List , John DiPirro, Artificial intelligence in computer graphics: a constructionist approach, ACM SIGGRAPH Computer Graphics, v.38 n.1, p.26-30, February 2004 [doi>10.1145/1012272.1012275]
• 20: W. Wahlster, editor. VERBMOBIL: Foundations of Speech-to-Speech Translation. Springer, 2000.
• 21: W. Wahlster. Smartweb: Mobile applications of the semantic web. In P. Dadam and M. Reichert, editors, GI Jahrestagung 2004, pages 26--27. Springer, 2004.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Natural language

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.6 Methodology and Techniques
          Subjects: Interaction techniques

General Terms:
Algorithms, Design, Human Factors

Keywords:
interaction design, multimodality, semantic web