Mobile human-robot teaming with environmental tolerance

ABSTRACT

We demonstrate that structured light-based depth sensing with standard perception algorithms can enable mobile peer-to-peer interaction between humans and robots. We posit that the use of recent emerging devices for depth-based imaging can enable robot perception of non-verbal cues in human movement in the face of lighting and minor terrain variations. Toward this end, we have developed an integrated robotic system capable of person following and responding to verbal and non-verbal commands under varying lighting conditions and uneven terrain. The feasibility of our system for peer-to-peer HRI is demonstrated through two trials in indoor and outdoor environments.

REFERENCES

• 1: H. G. Barrow, J. M. Tenenbaum, R. C. Boles, and H. C. Wolf. Parametric correspondence and chamfer matching: Two new techniques for image matching. In IJCAI, pages 659--663, 1977.
• 2: Cepstral, 2008. http://www.cepstral.com.
• 3: C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/ cjlin/libsvm.
• 4: N. Dalal, B. Triggs, and C. Schmid. Human detection using oriented histograms of flow and appearance. In European Conference on Computer Vision, 2006.
• 5: Rachel Gockley , Jodi Forlizzi , Reid Simmons, Natural person-following behavior for social robots, Proceedings of the ACM/IEEE international conference on Human-robot interaction, March 10-12, 2007, Arlington, Virginia, USA [doi>10.1145/1228716.1228720]
• 6: A. Haasch, S. Hohenner, S. Huwel, M. Kleinehagenbrock, S. Lang, I. Toptsis, G. Fink, J. Fritsch, B. Wrede, and G. Sagerer. BIRON - the bielefeld robot companion. In Int. Workshop on Advances in Service Robotics, pages 27--32, Stuttgart, Germany, 2004.
• 7: O. Jenkins, G. Gonzalez, and M. Loper. Interactive human pose and action recognition using dynamical motion primitives. International Journal of Humanoid Robotics, 4(2):365--385, Jun 2007.
• 8: W. G. Kennedy, M. Buga jska, M. Marge, W. Adams, P. Fransen, B. R., A. C. D., Schultz, and J. G. Trafton. Spatial representation and reasoning for human-robot collaboration. In Twenty-second National Conference on Artificial Intel ligence (AAAI-07), 2007.
• 9: S. Knoop, S. Vacek, and R. Dillmann. Sensor fusion for 3d human body tracking with an articulated 3d body model. In ICRA 2006: Proceedings 2006 IEEE International Conference on Robotics and Automation, pages 1686--1691, May 2006.
• 10: N. Ko jo, T. Inamura, K. Okada, and M. Inaba. Gesture recognition for humanoids using proto-symbol space. In Humanoid Robots, 2006 6th IEEE-RAS International Conference on, pages 76--81, 2006.
• 11: Monica N. Nicolescu , Maja J. Mataric, Natural methods for robot task learning: instructive demonstrations, generalization and practice, Proceedings of the second international joint conference on Autonomous agents and multiagent systems, July 14-18, 2003, Melbourne, Australia [doi>10.1145/860575.860614]
• 12: Michael Oren , Constantine Papageorgiou , Pawan Sinha , Edgar Osuna , Tomaso Poggio, Pedestrian Detection Using Wavelet Templates, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.193, June 17-19, 1997
• 13: Lawrence R. Rabiner, A tutorial on hidden Markov models and selected applications in speech recognition, Readings in speech recognition, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1990
• 14: O. Rogalla, M. Ehrenmann, R. Zollner, R. Becher, and R. Dillmann. Using gesture and speech control for commanding a robot assistant. In Proceedings. 11th IEEE International Workshop on Robot and Human Interactive Communication, pages 454--459, 2002.
• 15: Paul E. Rybski , Kevin Yoon , Jeremy Stolarz , Manuela M Veloso, Interactive robot task training through dialog and demonstration, Proceedings of the ACM/IEEE international conference on Human-robot interaction, March 10-12, 2007, Arlington, Virginia, USA [doi>10.1145/1228716.1228724]
• 16: D. Schulz. A probabilistic exemplar approach to combine laser and vision for person tracking. In Proceedings of Robotics: Science and Systems, Philadelphia, USA, August 2006.
• 17: H. Shimizu and T. Piggio. Direction estimation of pedestrian from multiple still images. In Intel ligent Vehicles Symposium, pages 596--600, June 2004.
• 18: C. Sminchisescu and A. Telea. Human pose estimation from silhouettes - a consistent approach using distance level sets. In WSCG International Conference on Computer Graphics,Visualization and Computer Vision, pages 413--420, 2002.
• 19: Sphinx-3, 2008. http://cmusphinx.sourceforge.net.
• 20: R. Stiefelhagen, C. Fugen, R. Gieselmann, H. Holzapfel, K. Nickel, and A. Waibel. Natural human-robot interaction using speech, head pose and gestures. In IEEE/RSJ International Conference Intel ligent Robots and Systems, volume 3, pages 2422--2427, Sendai, Japan, 2004.
• 21: SwissRanger specifications, 2008. http://www.swissranger.ch/main.php.
• 22: Sebastian Thrun, Toward a framework for human-robot interaction, Human-Computer Interaction, v.19 n.1, p.9-24, June 2004 [doi>10.1207/s15327051hci1901&2_2]
• 23: Stefan Waldherr , Roseli Romero , Sebastian Thrun, A Gesture Based Interface for Human-Robot Interaction, Autonomous Robots, v.9 n.2, p.151-173, September 2000 [doi>10.1023/A:1008918401478]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.9 Robotics
          Subjects: Operator interfaces

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Tracking; Range data
  I.5 PATTERN RECOGNITION
      I.5.4 Applications
          Subjects: Computer vision

General Terms:
Design, Human Factors

Keywords:
gesture recognition, human-robot interaction, person following