High speed obstacle avoidance using monocular vision and reinforcement learning

ABSTRACT

We consider the task of driving a remote control car at high speeds through unstructured outdoor environments. We present an approach in which supervised learning is first used to estimate depths from single monocular images. The learning algorithm can be trained either on real camera images labeled with ground-truth distances to the closest obstacles, or on a training set consisting of synthetic graphics images. The resulting algorithm is able to learn monocular vision cues that accurately estimate the relative depths of obstacles in a scene. Reinforcement learning/policy search is then applied within a simulator that renders synthetic scenes. This learns a control policy that selects a steering direction as a function of the vision system's output. We present results evaluating the predictive ability of the algorithm both on held out test data, and in actual autonomous driving experiments.

REFERENCES

• 1: J. L. Barron , D. J. Fleet , S. S. Beauchemin, Performance of optical flow techniques, International Journal of Computer Vision, v.12 n.1, p.43-77, Feb. 1994 [doi>10.1007/BF01420984]
• 2: Blthoff, I., Blthoff, H., & Sinha, P. (1998). Top-down influences on stereoscopic depth-perception. Nature Neuroscience, 1, 254--257.
• 3: A. Criminisi , I. Reid , A. Zisserman, Single View Metrology, International Journal of Computer Vision, v.40 n.2, p.123-148, Nov. 2000 [doi>10.1023/A:1026598000963]
• 4: E. R. Davies, Machine Vision: Theory, Algorithms, Practicalities, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2004
• 5: Gini, G., & Marchi, A. (2002). Indoor robot navigation with single camera vision. Proc. Pattern Recognition in Information Systems, PRIS, Spain.
• 6: Honig, J., Heit, B., & Bremont, J. (1996). Visual depth perception based on optical blur. Proc. of Int'l Conf. on Image Processing (pp. 721--724).
• 7: Huber, P. (1981). Robust statistics. New York: Wiley.
• 8: Jahne, B., & Geissler, P. (1994). Depth from focus with one image. Proc. IEEE Conf. on Computer Vision and Pattern Recognition CVPR (pp. 713--717).
• 9: Kardas, E. (2005). Monocular cues in depth perception. {Online}: http://peace.saumag.edu/faculty/Kardas/Courses/GP Weiten/C4SandP/MonoCues.html.
• 10: Michael Kearns , Satinder Singh, Finite-sample convergence rates for Q-learning and indirect algorithms, Proceedings of the 1998 conference on Advances in neural information processing systems II, p.996-1002, July 1999
• 11: Kim, M., & Uther, W. (2003). Automatic gait optimisation for quadruped robots. Proc. Australasian Conf. on Robotics and Automation (pp. 1--9).
• 12: Klarquist, W., Geisler, W., & Bovik, A. (1995). Maximum-likelihood depth-from-defocus for active vision. Proc. Int'l Conf. on Intelligent Robots and Systems (pp. 374--379).
• 13: Kohl, N., & Stone, P. (2004). Policy gradient reinforcement learning for fast quadrupedal locomotion. Proc. IEEE Int'l Conf. Robotics and Automation.
• 14: Kudo, H., Saito, M., Yamamura, T., & Ohnishi, N. (1999). Measurement of the ability in monocular depth perception during gazing at near visual target-effect of the ocular parallax cue. Proc. IEEE Int'l Conf. Systems, Man & Cybernetics (pp. 34--37).
• 15: LeCun, Y. (2003). Presentation at Navigation, Locomotion and Articulation workshop. Washington DC.
• 16: Loomis, J. M. (2001). Looking down is looking up. Nature News and Views, 414, 155--156.
• 17: Nagai, T., Naruse, T., Ikehara, M., & Kurematsu, A. (2002). Hmm-based surface reconstruction from single images. Proc. IEEE Int'l Conf. on Image Processing (pp. II--561 -- 11--564).
• 18: Andrew Y. Ng , Michael I. Jordan, PEGASUS: A policy search method for large MDPs and POMDPs, Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, p.406-415, June 30-July 03, 2000
• 19: Pomerleau, D. (1989). An autonomous land vehicle in a neural network. NIPS 1. Morgan Kaufmann.
• 20: Daniel Scharstein , Richard Szeliski, A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms, International Journal of Computer Vision, v.47 n.1-3, p.7-42, April-June 2002 [doi>10.1023/A:1014573219977]
• 21: Shao, M., Simchony, T., & Chellappa, R. (1988). New algorithms from reconstruction of a 3-d depth map from one or more images. Proc. IEEE Conf. on Computer Vision and Pattern Recognition CVPR (pp. 530--535).
• 22: Richard S. Sutton , Andrew G. Barto, Introduction to Reinforcement Learning, MIT Press, Cambridge, MA, 1998
• 23: Wu, B., Ooi, T. L., & He, Z. J. (2004). Perceiving distance accurately by a directional process of integrating ground information. Letters to Nature, 428, 73 77.