Multimodal multiplayer tabletop gaming

ABSTRACT

There is a large disparity between the rich physical interfaces of co-located arcade games and the generic input devices seen in most home console systems. In this article we argue that a digital table is a conducive form factor for general co-located home gaming as it affords: (a) seating in collaboratively relevant positions that give all equal opportunity to reach into the surface and share a common view; (b) rich whole-handed gesture input usually seen only when handling physical objects; (c) the ability to monitor how others use space and access objects on the surface; and (d) the ability to communicate with each other and interact on top of the surface via gestures and verbal utterance. Our thesis is that multimodal gesture and speech input benefits collaborative interaction over such a digital table. To investigate this thesis, we designed a multimodal, multiplayer gaming environment that allows players to interact directly atop a digital table via speech and rich whole-hand gestures. We transform two commercial single-player computer games, representing a strategy and simulation game genre, to work within this setting.

REFERENCES

• 1: Richard A. Bolt, “Put-that-there”: Voice and gesture at the graphics interface, Proceedings of the 7th annual conference on Computer graphics and interactive techniques, p.262-270, July 14-18, 1980, Seattle, Washington, United States
• 2: CLARK, H. 1996. Using Language. Cambridge University Press.
• 3: Multimodal Interaction During Multiparty Dialogues: Initial Results, Proceedings of the 4th IEEE International Conference on Multimodal Interfaces, p.448, October 14-16, 2002 [doi>10.1109/ICMI.2002.1167037]
• 4: COHEN, P.R, 2000. Speech can't do everything: A case for multimodal systems. Speech Technol. Mag. 5, 4.
• 5: Philip R. Cohen , Michael Johnston , David McGee , Sharon Oviatt , Jay Pittman , Ira Smith , Liang Chen , Josh Clow, QuickSet: multimodal interaction for distributed applications, Proceedings of the fifth ACM international conference on Multimedia, p.31-40, November 09-13, 1997, Seattle, Washington, United States [doi>10.1145/266180.266328]
• 6: Paul Dietz , Darren Leigh, DiamondTouch: a multi-user touch technology, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502389]
• 7: GREEBBERG, S. 1999. Designing computers as public artifacts. Int. J. Design Computing: Special Issue on Design Computing on the Net (Nov.30--Dec.3). University of Sydney.
• 8: Saul Greenberg, Personalizable groupware: accommodating individual roles and group differences, Proceedings of the second conference on European Conference on Computer-Supported Cooperative Work, p.17-31, September 25-27, 1991, Amsterdam, The Netherlands
• 9: S. Greenberg, Sharing views and interactions with single-user applications, Proceedings of the ACM SIGOIS and IEEE CS TC-OA conference on Office information systems, p.227-237, April 25-27, 1990, Cambridge, Massachusetts, United States
• 10: GUTWIN, C. AND GREENBERG, S. 2004. The importance of awareness for team cognition in distributed collaboration. In Team Cognition: Understanding the Factors that Drive Process and Performance, E. Salas and S. Fiore (eds.), APA Press, 177--201.
• 11: Christian Heath , Paul Luff, Collaborative activity and technological design: task coordination in London underground control rooms, Proceedings of the second conference on European Conference on Computer-Supported Cooperative Work, p.65-80, September 25-27, 1991, Amsterdam, The Netherlands
• 12: Carsten Magerkurth , Maral Memisoglu , Timo Engelke , Norbert Streitz, Towards the next generation of tabletop gaming experiences, Proceedings of Graphics Interface 2004, p.73-80, May 17-19, 2004, London, Ontario, Canada
• 13: Sharon Oviatt, Ten myths of multimodal interaction, Communications of the ACM, v.42 n.11, p.74-81, Nov. 1999 [doi>10.1145/319382.319398]
• 14: OVIATT, S. 1997. .Multimodal interactive maps: Designing for human performance. Human-Computer Interaction 12.
• 15: David Pinelle , Carl Gutwin , Saul Greenberg, Task analysis for groupware usability evaluation: Modeling shared-workspace tasks with the mechanics of collaboration, ACM Transactions on Computer-Human Interaction (TOCHI), v.10 n.4, p.281-311, December 2003 [doi>10.1145/966930.966932]
• 16: SEGAL, L. 1994. Effects of checklist interface on non-verbal crew communications. NASA Ames Research Center, Contractor Rep.177639.
• 17: John C. Tang, Findings from observational studies of collaborative work, International Journal of Man-Machine Studies, v.34 n.2, p.143-160, Feb. 1991 [doi>10.1016/0020-7373(91)90039-A]
• 18: Edward Tse , Chia Shen , Saul Greenberg , Clifton Forlines, Enabling interaction with single user applications through speech and gestures on a multi-user tabletop, Proceedings of the working conference on Advanced visual interfaces, May 23-26, 2006, Venezia, Italy [doi>10.1145/1133265.1133336]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

General Terms:
Design, Human Factors

Keywords:
computer supported cooperative work, multimodal speech and gesture interfaces, tabletop interaction, visual-spatial displays