Synchronous gestures for multiple persons and computers

ABSTRACT

This research explores distributed sensing techniques for mobile devices using synchronous gestures. These are patterns of activity, contributed by multiple users (or one user with multiple devices), which take on a new meaning when they occur together in time, or in a specific sequence in time. To explore this new area of inquiry, this work uses tablet computers augmented with touch sensors and two-axis linear accelerometers (tilt sensors). The devices are connected via an 802.11 wireless network and synchronize their time-stamped sensor data. This paper describes a few practical examples of interaction techniques using synchronous gestures such as dynamically tiling together displays by physically bumping them together, discusses implementation issues, and speculates on further possibilities for synchronous gestures.

REFERENCES

• 1: 1. Analog Devices Inc., Low-Cost +/-2g Dual-Axis Accelerometer with Duty Cycle Output, http:// www.analog.com/UploadedFiles/Datasheets/ 567227477ADXL202E_a.pdf, 2002.
• 2: Yuji Ayatsuka , Nobuyuki Matsushita , Jun Rekimoto, HyperPalette: a hybrid computing environment for small computing devices, CHI '00 extended abstracts on Human factors in computing systems, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/633292.633368]
• 3: 3. Bahl, P., Padmanabhan, V., RADAR: An In-Building RF-Based User Location and Tracking System, IEEE 19th Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM 2000), 775-784.
• 4: 4. Baxter, L.K., Capacitive Sensors: Design and Applications. IEEE Press Series on Electronics Technology, ed. R.J. Herrick. 1997, New York: The Institute of Electrical and Electronics Engineers.
• 5: 5. Buxton, W., Chunking and Phrasing and the Design of Human-Computer Dialogues, Information Processing '86, Proc. of the IFIP 10th World Computer Congress, 1986: Amsterdam: North Holland Publishers, 475-480.
• 6: 6. Chen, J.C., Yip, L., Wang, H., Maniezzo, D., Hudson, R.E., Elson, J., Yao, K., Estrin, D., DSP Implementation of a Distributed Acoustical Beamformer on a Wireless Sensor Platform, Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2003), 2003, Hong Kong, China.
• 7: 7. Choudhury, T., Pentland, A., The Sociometer: A Wearable Device for Understanding Human Networks, Conference on Computer Supported Cooperative Work (Workshop: Ad hoc Communications and Collaboratin in Ubiquitous Computing Environments), 2002.
• 8: 8. Dannenberg, R., Aura as a Platform for Distributed Sensing and Control, Symposium on Sensing and Input for Media-Centric Systems (SIMS 02), 2002, University of Santa Barbara: Center for Research in Electronic Art Technology.
• 9: 9. Dey, A., Abowd, G., Salber, D., A Conceptual Framework and a Toolkit for Supporting the Rapid Prototyping of Context-Aware Applications. Journal of Human-Computer Interaction, 2001. 16(2-4): p. 97-166.
• 10: Paul Dourish , Sara Bly, Portholes: supporting awareness in a distributed work group, Proceedings of the SIGCHI conference on Human factors in computing systems, p.541-547, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142982]
• 11: Jeremy Elson , Lewis Girod , Deborah Estrin, Fine-grained network time synchronization using reference broadcasts, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060304]
• 12: Matthew G. Gorbet , Maggie Orth , Hiroshi Ishii, Triangles: tangible interface for manipulation and exploration of digital information topography, Proceedings of the SIGCHI conference on Human factors in computing systems, p.49-56, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274652]
• 13: Jonathan Grudin, Partitioning digital worlds: focal and peripheral awareness in multiple monitor use, Proceedings of the SIGCHI conference on Human factors in computing systems, p.458-465, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365312]
• 14: Beverly L. Harrison , Kenneth P. Fishkin , Anuj Gujar , Carlos Mochon , Roy Want, Squeeze me, hold me, tilt me! An exploration of manipulative user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.17-24, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274647]
• 15: Ken Hinckleyss, Distributed and local sensing techniques for face-to-face collaboration, Proceedings of the 5th international conference on Multimodal interfaces, November 05-07, 2003, Vancouver, British Columbia, Canada [doi>10.1145/958432.958450]
• 16: Ken Hinckley , Jeff Pierce , Mike Sinclair , Eric Horvitz, Sensing techniques for mobile interaction, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.91-100, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354417]
• 17: Ken Hinckley , Mike Sinclair, Touch-sensing input devices, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.223-230, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303045]
• 18: Lars Erik Holmquist , Friedemann Mattern , Bernt Schiele , Petteri Alahuhta , Michael Beigl , Hans-Werner Gellersen, Smart-Its Friends: A Technique for Users to Easily Establish Connections between Smart Artefacts, Proceedings of the 3rd international conference on Ubiquitous Computing, p.116-122, September 30-October 02, 2001, Atlanta, Georgia, USA
• 19: Eric Horvitz , Carl Kadie , Tim Paek , David Hovel, Models of attention in computing and communication: from principles to applications, Communications of the ACM, v.46 n.3, March 2003 [doi>10.1145/636772.636798]
• 20: Hua Jiang , G. Drew Kessler , Jean Nonnemaker, DEMIS: a dynamic event model for interactive systems, Proceedings of the ACM symposium on Virtual reality software and technology, November 11-13, 2002, Hong Kong, China [doi>10.1145/585740.585757]
• 21: Brad Johanson , Armando Fox, The Event Heap: A Coordination Infrastructure for Interactive Workspaces, Proceedings of the Fourth IEEE Workshop on Mobile Computing Systems and Applications, p.83, June 20-21, 2002
• 22: 22. Pierce, J., Mahaney, H., Abowd, G., Opportunistic Annexing for Handheld Devices: Opportunities and Challenges, Tech Report #GIT-GVU-03-31, 2003. http://www.cc.gatech.edu/ gvu/research/techreports.html.
• 23: Jun Rekimoto, Pick-and-drop: a direct manipulation technique for multiple computer environments, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.31-39, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263505]
• 24: 24. Schilit, B.N., Adams, N.I., Want, R., Context-Aware Computing Applications, Proc. IEEE Workshop on Mobile Computing Systems and Applications, 1994, Santa Cruz, CA: IEEE Computer Society, 85-90.
• 25: Peter Tandler , Thorsten Prante , Christian Müller-Tomfelde , Norbert Streitz , Ralf Steinmetz, Connectables: dynamic coupling of displays for the flexible creation of shared workspaces, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502351]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.9 Robotics
          Subjects: Sensors

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software
          Subjects: Distributed systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen); Graphical user interfaces (GUI)

General Terms:
Design, Management

Keywords:
context awareness, distributed sensor systems, input devices, multi-user interfaces, sensors, ubiquitous computing