Tangible interaction with real and virtual products: designing a shopping assistant for rural communities

ABSTRACT

This work describes the design and development process of an interactive ordering system for rural corner stores, extending the limited assortment of a small store with virtual items on touch-sensitive displays embedded into shelves. The interface blends tangible interaction of both real products in a shelf and virtual products iconically represented on nearby screens. The tangible interaction component is complemented by a natural language interface, supporting comparison and inspection of multiple products of the real and virtual world.

REFERENCES

• 1: O. Kühne, H. Zweck, and T. Gross. Plan zur Entwicklung des Ländlichen Raums im Saarland. Technischer Bericht, Saarländisches Ministerium für Umwelt, Illingen, 2005.
• 2: Mark T. Maybury , Wolfgang Wahlster, Readings in intelligent user interfaces, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1998
• 3: Sharon Oviatt, Ten myths of multimodal interaction, Communications of the ACM, v.42 n.11, p.74-81, Nov. 1999 [doi>10.1145/319382.319398]
• 4: Sharon Oviatt, Multimodal interfaces, The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications, L. Erlbaum Associates Inc., Hillsdale, NJ, 2002
• 5: W. Wahlster. Towards symmetric multimodality: Fusion and fission of speech, gesture, and facial expression. In Proceedings of the 26th German Conference on Artificial Intelligence, September 2003, pages 1--18. Springer, 2003.
• 6: R. Wasinger, A. Krüger, and O. Jacobs. Integrating intra and extra gestures into a mobile and multimodal shopping assistant. In In: Proceedings of the 3rd International Conference on Pervasive Computing (Pervasive2005, pages 297--314. Springer LNCS, 2005.

INDEX TERMS

Keywords:
multi-modal interaction, shopping assistant, tangible user interface