Magic cards: a paper tag interface for implicit robot control

ABSTRACT

Typical Human Robot Interaction (HRI) assumes that the user explicitly interacts with robots. However, explicit control with robots can be unnecessary or even undesirable in certain cases, such as dealing with domestic services (or housework). In this paper, we propose an alternative strategy of interaction: the user implicitly controls a robot by issuing commands on corresponding real world objects and the environment. Robots then discover these commands and complete them in the background. We implemented a paper-tag-based interface to support such implicit robot control in a sensor-augmented home environment. Our initial user studies indicated that the paper-tag-based interface is particularly simple to use and provides users with flexibility in planning and controlling their housework tasks in a simulated home environment.

REFERENCES

• 1: ARToolKit. Http://www.hitl.washington.edu/artoolkit/
• 2: The National Survey of Families and Household. Http: //www.ssc.wisc.edu/nsfh/.
• 3: Ubisense. Http://www.ubisense.net.
• 4: Few DA, Bruemmer DJ, and Walton MC. Improved human-robot teaming through facilitated initiative. Proc. RO-MAN '06, (2006), 171--176.
• 5: Mark Fiala, ARTag, a Fiducial Marker System Using Digital Techniques, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2, p.590-596, June 20-26, 2005 [doi>10.1109/CVPR.2005.74]
• 6: Terrence Fong , Charles Thorpe, Vehicle Teleoperation Interfaces, Autonomous Robots, v.11 n.1, p.9-18, July 2001 [doi>10.1023/A:1011295826834]
• 7: Jodi Forlizzi, How robotic products become social products: an ethnographic study of cleaning in the home, Proceedings of the ACM/IEEE international conference on Human-robot interaction, March 10-12, 2007, Arlington, Virginia, USA [doi>10.1145/1228716.1228734]
• 8: Jodi Forlizzi , Carl DiSalvo, Service robots in the domestic environment: a study of the roomba vacuum in the home, Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction, March 02-03, 2006, Salt Lake City, Utah, USA [doi>10.1145/1121241.1121286]
• 9: Grimm J and Grimm W. The elves and the shoemaker. Fairy tales.
• 10: Jeremy M. Heiner , Scott E. Hudson , Kenichiro Tanaka, Linking and messaging from real paper in the Paper PDA, Proceedings of the 12th annual ACM symposium on User interface software and technology, p.179-186, November 07-10, 1999, Asheville, North Carolina, United States [doi>10.1145/320719.322600]
• 11: Hersch J and Stratton LS. Housework and wages. The Journal of Human Resources 2002; 37(1): 217--229.
• 12: Hiroshi Ishii , Brygg Ullmer, Tangible bits: towards seamless interfaces between people, bits and atoms, Proceedings of the SIGCHI conference on Human factors in computing systems, p.234-241, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258715]
• 13: Walter Johnson , Herbert Jellinek , Leigh Klotz, Jr. , Ramana Rao , Stuart K. Card, Bridging the paper and electronic worlds: the paper user interface, Proceedings of the INTERACT '93 and CHI '93 conference on Human factors in computing systems, p.507-512, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169445]
• 14: Cory D. Kidd , Robert Orr , Gregory D. Abowd , Christopher G. Atkeson , Irfan A. Essa , Blair MacIntyre , Elizabeth D. Mynatt , Thad Starner , Wendy Newstetter, The Aware Home: A Living Laboratory for Ubiquitous Computing Research, Proceedings of the Second International Workshop on Cooperative Buildings, Integrating Information, Organization, and Architecture, p.191-198, October 01-02, 1999
• 15: Beth M. Lange , Mark A. Jones , James L. Meyers, Insight lab: an immersive team environment linking paper, displays, and data, Proceedings of the SIGCHI conference on Human factors in computing systems, p.550-557, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274718]
• 16: Thomas P. Moran , Eric Saund , William Van Melle , Anuj U. Gujar , Kenneth P. Fishkin , Beverly L. Harrison, Design and technology for Collaborage: collaborative collages of information on physical walls, Proceedings of the 12th annual ACM symposium on User interface software and technology, p.197-206, November 07-10, 1999, Asheville, North Carolina, United States [doi>10.1145/320719.322602]
• 17: Olson JT. Role conflict between housework and child care. Work and Occupations 1979; 6(4): 430--456.
• 18: Patel SN, Truong KN, and Abowd GD. PowerLine positioning: A practical sub-room-level indoor location system for domestic use. Proc. Ubicomp '06, (2006), 441--458.
• 19: Jun Rekimoto , Yuji Ayatsuka, CyberCode: designing augmented reality environments with visual tags, Proceedings of DARE 2000 on Designing augmented reality environments, p.1-10, April 2000, Elsinore, Denmark [doi>10.1145/354666.354667]
• 20: Robinson JA and Robertson C. The LivePaper system: augmenting paper on an enhanced tabletop. Computers&Graphics 2001; 25(5): 731--743.
• 21: Rogers Y. Moving on from Weiser's vision of calm computing: Engaging UbiComp experience. Proc. Ubicomp '06, (2006), 404--421.
• 22: Saund E. Image mosaicing and a diagrammatic user interface for an office whiteboard scanner. Proc. CoBuild '99, (1999).
• 23: Itiro Siio , Yoshiaki Mima, IconStickers: Converting Computer Icons into Real Paper Icons, Proceedings of HCI International (the 8th International Conference on Human-Computer Interaction) on Human-Computer Interaction: Ergonomics and User Interfaces-Volume I, p.271-275, August 22-26, 1999
• 24: Quentin Stafford-Fraser , Peter Robinson, BrightBoard: a video-augmented environment, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.134-141, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238457]
• 25: Ja-Young Sung , Rebecca E. Grinter , Henrik I. Christensen , Lan Guo, Housewives or technophiles?: understanding domestic robot owners, Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction, March 12-15, 2008, Amsterdam, The Netherlands [doi>10.1145/1349822.1349840]
• 26: Tao Q, Oyster girl, Shou shen hou ji.
• 27: Alex S. Taylor , Laurel Swan, Artful systems in the home, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055060]
• 28: Weiser M. The computer for the 21st century. Scientific American 1991; 265(3): 94--104.
• 29: Pierre Wellner, Interacting with paper on the DigitalDesk, Communications of the ACM, v.36 n.7, p.87-96, July 1993 [doi>10.1145/159544.159630]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.9 Robotics
          Subjects: Operator interfaces; Commercial robots and applications

General Terms:
Design, Human Factors

Keywords:
domestic service, implicit human robot interaction, magic cards, paper-tag-based interface