Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

We introduce flexible algorithms that can automatically learn mappings from images to actions by interacting with their environment. They work by introducing an image classifier in front of a Reinforcement Learning algorithm. The classifier partitions the visual space according to the presence or absence of highly informative local descriptors. The image classifier is incrementally refined by selecting new local descriptors when perceptual aliasing is detected. Thus, we reduce the visual input domain down to a size manageable by Reinforcement Learning, permitting us to learn direct percept-to-action mappings. Experimental results on a continuous visual navigation task illustrate the applicability of the framework.

REFERENCES

• 1: Richard Ernest Bellman, Dynamic Programming, Dover Publications, Incorporated, 2003
• 2: Dimitri P. Bertsekas , John N. Tsitsiklis, Neuro-Dynamic Programming, Athena Scientific, 1996
• 3: Chapman, D., & Kaelbling, L. (1991). Input generalization in delayed reinforcement learning: An algorithm and performance comparisons. Proc. of the 12th International Joint Conference on Artificial Intelligence (IJCAI) (pp. 726--731). Sydney.
• 4: Chrisman, L. (1992). Reinforcement learning with perceptual aliasing: The perceptual distinctions approach. National Conference on Artificial Intelligence (pp. 183--188).
• 5: Coelho, J., Piater, J., & Grupen, R. (2001). Developing haptic and visual perceptual categories for reaching and grasping with a humanoid robot. Robotics and Autonomous Systems, 37, 195--218.
• 6: Gibson, E., & Spelke, E. (1983). The development of perception. Handbook of child psychology vol. iii: Cognitive development, chapter 1, 2--76. Wiley.
• 7: V. Gouet , N. Boujemaa, Object-based queries using color points of interest, Proceedings of the IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'01), p.30, December 14-14, 2001
• 8: David G. Lowe, Object Recognition from Local Scale-Invariant Features, Proceedings of the International Conference on Computer Vision-Volume 2, p.1150, September 20-25, 1999
• 9: Andrew Kachites Mccallum , Dana Ballard, Reinforcement learning with selective perception and hidden state, 1996
• 10: Mikolajczyk, K., & Schmid, C. (2003). A performance evaluation of local descriptors. IEEE Conference on Computer Vision and Pattern Recognition (pp. 257--263). Madison (WI, USA).
• 11: Rémi Munos , Andrew Moore, Variable Resolution Discretization in Optimal Control, Machine Learning, v.49 n.2-3, p.291-323, November-December 2002 [doi>10.1023/A:1017992615625]
• 12: Justus H. Piater , Roderic A. Grupen, Visual feature learning, 2001
• 13: J. Ross Quinlan, C4.5: programs for machine learning, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1993
• 14: Sébastien Jodogne , Fabien Scalzo , Justus H. Piater, Task-Driven Learning of Spatial Combinations of Visual Features, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops, p.48, June 20-26, 2005 [doi>10.1109/CVPR.2005.539]
• 15: Cordelia Schmid , Roger Mohr, Local Grayvalue Invariants for Image Retrieval, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.5, p.530-535, May 1997 [doi>10.1109/34.589215]
• 16: Cordelia Schmid , Roger Mohr , Christian Bauckhage, Evaluation of Interest Point Detectors, International Journal of Computer Vision, v.37 n.2, p.151-172, June 2000 [doi>10.1023/A:1008199403446]
• 17: Schyns, P., & Rodet, L. (1997). Categorization creates functional features. Journ. of Experimental Psychology: Learning, Memory and Cognition, 23, 681--696.
• 18: Singh, S., Jaakkola, T., & Jordan, M. (1995). Reinforcement learning with soft state aggregation. Advances in Neural Information Processing Systems (pp. 361--368). MIT Press.
• 19: Richard S. Sutton, Reinforcement Learning, Kluwer Academic Publishers, Norwell, MA, 1992
• 20: William T. B. Uther , Manuela M. Veloso, Tree based discretization for continuous state space reinforcement learning, Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence, p.769-774, July 1998, Madison, Wisconsin, United States
• 21: Watkins, C. (1989). Learning from delayed rewards. Doctoral dissertation, King's College, Cambridge.
• 22: Steven D. Whitehead , Dana H. Ballard, Learning to Perceive and Act by Trial and Error, Machine Learning, v.7 n.1, p.45-83, July 1991 [doi>10.1023/A:1022619109594]