Cooperative gestures: multi-user gestural interactions for co-located groupware

ABSTRACT

Multi-user, touch-sensing input devices create opportunities for the use of cooperative gestures -- multi-user gestural interactions for single display groupware. Cooperative gestures are interactions where the system interprets the gestures of more than one user as contributing to a single, combined command. Cooperative gestures can be used to enhance users' sense of teamwork, increase awareness of important system events, facilitate reachability and access control on large, shared displays, or add a unique touch to an entertainment-oriented activity. This paper discusses motivating scenarios for the use of cooperative gesturing and describes some initial experiences with CollabDraw, a system for collaborative art and photo manipulation. We identify design issues relevant to cooperative gesturing interfaces, and present a preliminary design framework. We conclude by identifying directions for future research on cooperative gesturing interaction techniques.

REFERENCES

• 1: Thomas Baudel , Michel Beaudouin-Lafon, Charade: remote control of objects using free-hand gestures, Communications of the ACM, v.36 n.7, p.28-35, July 1993 [doi>10.1145/159544.159562]
• 2: Baudisch, P., Cutrell, E., Robbins, D., Czerwinski, M., Tandler, P., Bederson, B., and Zierlinger, A. Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch- and Pen-operated Systems. Interact 2003.
• 3: Bratman, M. 1999. Faces of Intention. Cambridge, MA: Cambridge University Press.
• 4: Buxton, W. Chunking and Phrasing and the Design of Human-Computer Dialogues. IFIP Conference, 1986, 475--480.
• 5: Alessandro Cappelletti , Giulia Gelmini , Fabio Pianesi , Franca Rossi , Massimo Zancanaro, Enforcing Cooperative Storytelling: First Studies, Proceedings of the IEEE International Conference on Advanced Learning Technologies (ICALT'04), p.281-285, August 30-September 01, 2004
• 6: Paul Dietz , Darren Leigh, DiamondTouch: a multi-user touch technology, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502389]
• 7: Katherine Everitt , Chia Shen , Kathy Ryall , Clifton Forlines, Modal spaces: spatial multiplexing to mediate direct-touch input on large displays, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1056916]
• 8: Jörg Geißler, Shuffle, throw or take it! working efficiently with an interactive wall, CHI 98 conference summary on Human factors in computing systems, p.265-266, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/286498.286745]
• 9: Tovi Grossman , Daniel Wigdor , Ravin Balakrishnan, Multi-finger gestural interaction with 3d volumetric displays, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029644]
• 10: François Guimbretière , Maureen Stone , Terry Winograd, Fluid interaction with high-resolution wall-size displays, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502353]
• 11: Guiard, Y. Asymmetric division of labor in human skilled bimanual action: the kinematic chain as a model. Journal of Motor Behaviour. 19 (4), 1987, 486--517.
• 12: Hall, E.T. The Hidden Dimension. New York: Anchor Books, 1966.
• 13: Ken Hinckley, Synchronous gestures for multiple persons and computers, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.149-158, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964713]
• 14: Ken Hinckley , Gonzalo Ramos , Francois Guimbretiere , Patrick Baudisch , Marc Smith, Stitching: pen gestures that span multiple displays, Proceedings of the working conference on Advanced visual interfaces, May 25-28, 2004, Gallipoli, Italy [doi>10.1145/989863.989866]
• 15: Lars Erik Holmquist , Friedemann Mattern , Bernt Schiele , Petteri Alahuhta , Michael Beigl , Hans-Werner Gellersen, Smart-Its Friends: A Technique for Users to Easily Establish Connections between Smart Artefacts, Proceedings of the 3rd international conference on Ubiquitous Computing, p.116-122, September 30-October 02, 2001, Atlanta, Georgia, USA
• 16: Joyce, W.B. On the Free-Rider Problem in Cooperative Learning. Journal of Educational Business. 74(5), 271--274, May-June 1999.
• 17: Myron W. Krueger , Thomas Gionfriddo , Katrin Hinrichsen, VIDEOPLACE—an artificial reality, Proceedings of the SIGCHI conference on Human factors in computing systems, p.35-40, April 1985, San Francisco, California, United States
• 18: Shahzad Malik , Abhishek Ranjan , Ravin Balakrishnan, Interacting with large displays from a distance with vision-tracked multi-finger gestural input, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095042]
• 19: Techniques for Interactive Audience Participation, Proceedings of the 4th IEEE International Conference on Multimodal Interfaces, p.15, October 14-16, 2002 [doi>10.1109/ICMI.2002.1166962]
• 20: McCullagh, P. and Nelder, J.A. Generalized Linear Models. Chapman and Hall, 2nd edition, 1989.
• 21: Meredith Ringel Morris , Kathy Ryall , Chia Shen , Clifton Forlines , Frederic Vernier, Beyond "social protocols": multi-user coordination policies for co-located groupware, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, 2004, Chicago, Illinois, USA [doi>10.1145/1031607.1031648]
• 22: Kenji Oka , Yoichi Sato , Hideki Koike, Real-Time Tracking of Multiple Fingertips and Gesture Recognition for Augmented Desk Interface Systems, Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition, p.429, May 20-21, 2002
• 23: Piper, A.M., O'Brien, E., Morris, M.R., and Winograd, T. SIDES: A Collaborative Tabletop Computer Game for Social Skills Development. Stanford Univ. Technical Report, 2005.
• 24: Jun Rekimoto, SmartSkin: an infrastructure for freehand manipulation on interactive surfaces, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503397]
• 25: Jun Rekimoto, SyncTap: synchronous user operation for spontaneous network connection, Personal and Ubiquitous Computing, v.8 n.2, p.126-134, May 2004 [doi>10.1007/s00779-004-0262-2]
• 26: Matthew Reynolds , Bernd Schoner , Joey Richards , Kelly Dobson , Neil Gershenfeld, An immersive, multi-user, musical stage environment, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.553-560, August 2001 [doi>10.1145/383259.383324]
• 27: Meredith Ringel , Kathy Ryall , Chia Shen , Clifton Forlines , Frederic Vernier, Release, relocate, reorient, resize: fluid techniques for document sharing on multi-user interactive tables, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.986085]
• 28: Meredith Ringel , Henry Berg , Yuhui Jin , Terry Winograd, Barehands: implement-free interaction with a wall-mounted display, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634284]
• 29: Rogers, Y. and Lindley, S. Collaborating Around Large Interactive Displays: Which Way is Best to Meet? Interacting with Computers, 2004.
• 30: Ryall, K., Esenther, A., Everitt, K., Forlines, C., Morris, M.R., Shen, C., Shipman, S. and Vernier, F. iDwidgets: Parameterizing Widgets by User Identity. Interact 2005.
• 31: Searle, J.R. Collective intentions and actions. In P. R. Cohen, J. Morgan, and M. E. Pollack, editors, Intentions in Communication, chapter 19, pages 401--416. MIT Press, Camridge, MA, 1990.
• 32: Stacey D. Scott , M. Sheelagh , T. Carpendale , Kori M. Inkpen, Territoriality in collaborative tabletop workspaces, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, 2004, Chicago, Illinois, USA [doi>10.1145/1031607.1031655]
• 33: Chia Shen , Frédéric D. Vernier , Clifton Forlines , Meredith Ringel, DiamondSpin: an extensible toolkit for around-the-table interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.167-174, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985714]
• 34: Jason Stewart , Benjamin B. Bederson , Allison Druin, Single display groupware: a model for co-present collaboration, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.286-293, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303064]
• 35: Taxén, G., Hellström, S.-O., Tobiasson, H., Back, M., and Bowers, J. The Well of Inventions -- Learning, Interaction and Participatory Design in Museum Installations. ICHIM 2003.
• 36: Ryan Ulyate , David Bianciardi, The interactive dance club: avoiding chaos in a multi participant environment, Proceedings of the 2001 conference on New interfaces for musical expression, April 01-02, 2001, Seattle, Washington
• 37: Daniel Vogel , Ravin Balakrishnan, Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029656]
• 38: Pierre Wellner, Interacting with paper on the DigitalDesk, Communications of the ACM, v.36 n.7, p.87-96, July 1993 [doi>10.1145/159544.159630]
• 39: Catherine G. Wolf , James R. Rhyne, Gesturing with shared drawing tools, INTERACT '93 and CHI '93 conference companion on Human factors in computing systems, p.137-138, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/259964.260150]
• 40: Mike Wu , Ravin Balakrishnan, Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.193-202, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964718]
• 41: Mike Wu , Chia Shen , Kathy Ryall , Clifton Forlines , Ravin Balakrishnan, Gesture Registration, Relaxation, and Reuse for Multi-Point Direct-Touch Surfaces, Proceedings of the First IEEE International Workshop on Horizontal Interactive Human-Computer Systems, p.185-192, January 05-07, 2006 [doi>10.1109/TABLETOP.2006.19]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces

Keywords:
computer-supported cooperative work, cooperative gestures, gestures, single display groupware