Touch and toys: new techniques for interaction with a remote group of robots

ABSTRACT

Interaction with a remote team of robots in real time is a difficult human-robot interaction (HRI) problem exacerbated by the complications of unpredictable real-world environments, with solutions often resorting to a larger-than-desirable ratio of operators to robots. We present two innovative interfaces that allow a single operator to interact with a group of remote robots. Using a tabletop computer the user can configure and manipulate groups of robots directly by either using their fingers (touch) or by manipulating a set of physical toys (tangible user interfaces). We recruited participants to partake in a user study that required them to interact with a small group of remote robots in simple tasks, and present our findings as a set of design considerations.

REFERENCES

• 1: Michel Beaudouin-Lafon, Instrumental interaction: an interaction model for designing post-WIMP user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.446-453, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332473]
• 2: Breazeal, C. Toward sociable robots. Robot. and Auton. Syst. 42, 3--4 (2003), 167--175.
• 3: Where the action is: the foundations of embodied interaction, MIT Press, Cambridge, MA, 2001
• 4: Mauro Dragone , Thomas Holz , Gregory M.P. O'Hare, Mixing robotic realities, Proceedings of the 11th international conference on Intelligent user interfaces, January 29-February 01, 2006, Sydney, Australia [doi>10.1145/1111449.1111504]
• 5: Drury, J. L., Scholtz, J., and Yanco, H. A. Awareness in Human-Robot Interactions. In Proc. SMC '03, IEEE (2003), 912--918.
• 6: George W. Fitzmaurice , William Buxton, An empirical evaluation of graspable user interfaces: towards specialized, space-multiplexed input, Proceedings of the SIGCHI conference on Human factors in computing systems, p.43-50, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258578]
• 7: George W. Fitzmaurice , Hiroshi Ishii , William A. S. Buxton, Bricks: laying the foundations for graspable user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.442-449, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223964]
• 8: Phil Frei , Victor Su , Bakhtiar Mikhak , Hiroshi Ishii, curlybot: designing a new class of computational toys, Proceedings of the SIGCHI conference on Human factors in computing systems, p.129-136, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332416]
• 9: Kevin Gold , Brian Scassellati, Using context and sensory data to learn first and second person pronouns, Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction, March 02-03, 2006, Salt Lake City, Utah, USA [doi>10.1145/1121241.1121262]
• 10: Goodrich, M., and Olsen, D. Seven principles of efficient human robot interaction. In Proc. SMC '03, IEEE (2003), 3943--3948.
• 11: Saul Greenberg , Bill Buxton, Usability evaluation considered harmful (some of the time), Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357074]
• 12: Cheng Guo , Ehud Sharlin, Exploring the use of tangible user interfaces for human-robot interaction: a comparative study, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357076]
• 13: Uta Hinrichs , Sheelagh Carpendale , Stacey D. Scott, Evaluating the effects of fluid interface components on tabletop collaboration, Proceedings of the working conference on Advanced visual interfaces, May 23-26, 2006, Venezia, Italy [doi>10.1145/1133265.1133272]
• 14: Hiroshi Ishii , Brygg Ullmer, Tangible bits: towards seamless interfaces between people, bits and atoms, Proceedings of the SIGCHI conference on Human factors in computing systems, p.234-241, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258715]
• 15: Jones, H., and Snyder, M. Supervisory control of multiple robots based on a real-time strategy game interaction paradigm. In Proc. SMC '01, IEEE (2001), 383--388.
• 16: Sara Kiesler , Pamela Hinds, Introduction to this special issue on human-robot interaction, Human-Computer Interaction, v.19 n.1, p.1-8, June 2004 [doi>10.1207/s15327051hci1901&2_1]
• 17: Russell Kruger , Sheelagh Carpendale , Stacey D. Scott , Anthony Tang, Fluid integration of rotation and translation, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055055]
• 18: Paul Lapides , Ehud Sharlin , Mario Costa Sousa, Three dimensional tangible user interface for controlling a robotic team, Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction, March 12-15, 2008, Amsterdam, The Netherlands [doi>10.1145/1349822.1349867]
• 19: Mandryk, R., Scott, S., and Inkpen, K. Display factors influencing co-located collaboration. In Ext. Abst. CSCW'02, ACM (2002), 137--138.
• 20: MIT Media Group. Leonardo. WWW, http://robotic.media.mit.edu/projects/Leonardo/Leo-intro.html, Visited May 22nd, 2006, 2006.
• 21: Norman, D. Emotional design: why we love (or hate) everyday things. Basic Books, NY, 2004.
• 22: Norman, D. A. The Design of Everyday Things. Doubleday, New York, NY, 1988.
• 23: Quigley, M., Goodrich, M., and Beard, R. Semi-autonomous human-uav interfaces for fixed- wing mini-uavs. In Proc. IROS '04, IEEE (2004).
• 24: Hayes Solos Raffle , Amanda J. Parkes , Hiroshi Ishii, Topobo: a constructive assembly system with kinetic memory, Proceedings of the SIGCHI conference on Human factors in computing systems, p.647-654, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985774]
• 25: Justin Richer , Jill L. Drury, A video game-based framework for analyzing human-robot interaction: characterizing interface design in real-time interactive multimedia applications, Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction, March 02-03, 2006, Salt Lake City, Utah, USA [doi>10.1145/1121241.1121287]
• 26: Jan Richter , Bruce H. Thomas , Maki Sugimoto , Masahiko Inami, Remote active tangible interactions, Proceedings of the 1st international conference on Tangible and embedded interaction, February 15-17, 2007, Baton Rouge, Louisiana [doi>10.1145/1226969.1226977]
• 27: Rogers, Y., and Lindley, S. Collaborating around large interactive displays: Which way is best to meet? Interact. Comput. 16, 6 (2004), 1133--1152.
• 28: Stacey D. Scott , Karen D. Grant , Regan L. Mandryk, System guidelines for co-located, collaborative work on a tabletop display, Proceedings of the eighth conference on European Conference on Computer Supported Cooperative Work, p.159-178, September 14-18, 2003, Helsinki, Finland
• 29: Scott, S., Wan, J., Rico, A., Furusho, C., and Cummings, M. Aiding team supervision in command and control operations with large-screen displays. In Proc. HSIS '07, ASNE (2007).
• 30: Setalaphruk, V., Ueno, A., Kume, I., and Kono, Y. Robot navigation in corridor environments using a sketch floor map. In Proc. CIRA '03, IEEE, 552--557.
• 31: Ehud Sharlin , Benjamin Watson , Yoshifumi Kitamura , Fumio Kishino , Yuichi Itoh, On tangible user interfaces, humans and spatiality, Personal and Ubiquitous Computing, v.8 n.5, p.338-346, September 2004 [doi>10.1007/s00779-004-0296-5]
• 32: Noriyoshi Shimizu , Naoya Koizumi , Maki Sugimoto , Hideaki Nii , Dairoku Sekiguchi , Masahiko Inami, A teddy-bear-based robotic user interface, Computers in Entertainment (CIE), v.4 n.3, July 2006 [doi>10.1145/1146816.1146830]
• 33: Candace L. Sidner , Christopher Lee , Louis-Philippe Morency , Clifton Forlines, The effect of head-nod recognition in human-robot conversation, Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction, March 02-03, 2006, Salt Lake City, Utah, USA [doi>10.1145/1121241.1121291]
• 34: Marjorie Skubic , Derek Anderson , Samuel Blisard , Dennis Perzanowski , Alan Schultz, Using a hand-drawn sketch to control a team of robots, Autonomous Robots, v.22 n.4, p.399-410, May 2007 [doi>10.1007/s10514-007-9023-1]
• 35: Strauss, A., and Corbin, J. Basics of Qualitative Research. Sage, London, 1998.
• 36: Sebastian Thrun, Toward a framework for human-robot interaction, Human-Computer Interaction, v.19 n.1, p.9-24, June 2004 [doi>10.1207/s15327051hci1901&2_2]
• 37: Xin, M., and Sharlin, E. Exploring human-robot interaction through telepresence board games. In Adv. in Artif. Real. and Tele- Exist., vol. 4282/2006 of Lect. Notes in Comput. Sci. Springer, 2006, 249--261.
• 38: Yanco, H. A., and Drury, J. Classifying human-robot interaction: an updated taxonomy. In Proc. SMC '04, IEEE (2004), 2841--2846.
• 39: Young, J., Igarashi, T., and Sharlin, E. Puppet master: Designing reactive character behavior by demonstration. In SIGGRAPH SCA '08, Eurographics Association (2008), 183--191.
• 40: Young, J., and Sharlin, E. Sharing spaces with robots: an integrated environment for human-robot interaction. In Proc. ISIE '06, MSR (2006), 103--110.
• 41: James E. Young , Min Xin , Ehud Sharlin, Robot expressionism through cartooning, Proceedings of the ACM/IEEE international conference on Human-robot interaction, March 10-12, 2007, Arlington, Virginia, USA [doi>10.1145/1228716.1228758]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation); Theory and methods

General Terms:
Design

Keywords:
human-robot interaction, robot teams, tabletop computing, tangible user interfaces, touch interfaces