Convergence analysis of temporal-difference learning algorithms with linear function approximation

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

Convergence analysis of temporal-difference learning algorithms with linear function approximation

Full text

Pdf (919 KB)

Source

Annual Workshop on Computational Learning Theory archive
Proceedings of the twelfth annual conference on Computational learning theory table of contents

Santa Cruz, California, United States

Pages: 193 - 202

Year of Publication: 1999

ISBN:1-58113-167-4

Author

Vladislav Tadić

Mihajlo Pupin Institute, Volgina 15, 11000 Belgrade, Serbia, Yugoslavia

Sponsors

SIGACT: ACM Special Interest Group on Algorithms and Computation Theory
SIGART: ACM Special Interest Group on Artificial Intelligence
Univ. of California, : University of California at Santa Cruz

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 1, Downloads (12 Months): 16, Citation Count: 0

Additional Information:

references index terms collaborative colleagues peer to peer

Tools and Actions:	Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/307400.307438 What is a DOI?

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	Albert Benveniste , Pierre Priouret , Michel Métivier, Adaptive algorithms and stochastic approximations, Springer-Verlag New York, Inc., New York, NY, 1990
	2	Dimitri P. Bertsekas, Dynamic Programming and Optimal Control, Athena Scientific, 2001
	3	Dimitri P. Bertsekas , John N. Tsitsiklis, Neuro-Dynamic Programming, Athena Scientific, 1996
	4	D. S. Clark, Necessary and sufficient conditions for the Robbins-Monro method, Stochastic Processes and their Applications, vol. 17, pp. 359-367, 1984.
	5	Han-Fu Chen , Lei Guo, Identification and stochastic adaptive control, Birkhauser Boston Inc., Cambridge, MA, 1991
	6	Peter Dayan, The Convergence of TD(λ) for General λ, Machine Learning, v.8 n.3-4, p.341-362, May 1992 [doi>10.1007/BF00992701]
	7	Peter Dayan , Terrence J. Sejnowski, TD(λ) Converges with Probability 1, Machine Learning, v.14 n.3, p.295-301, March 1994 [doi>10.1023/A:1022657612745]
	8	T. Jaakola, M. I. Jordan, S. P. Singh, On the convergence of stochastic iterative dynamic programming algorithms, Neural Computation, vol. 6, pp. 1185-1201, 1994.
	9	S. R. Kulkarni and C. S. Horn, An alternative proof for convergence of stochastic approximation algorithms, IEEE Transactions of Automatic Control, vol. 41, pp. 419-424, 1996.
	10	P. R. Kumar , Pravin Varaiya, Stochastic systems: estimation, identification and adaptive control, Prentice-Hall, Inc., Upper Saddle River, NJ, 1986
	11	H. J. Kushner and D. S. Clark, Stochastic Approximation Methods for Constrained and Unconstrained Systems, Springer Verlag, 1978.
	12	Lennart Ljung , Georg Pflug , Harro Walk, Stochastic approximation and optimization of random systems, Birkhauser Verlag, Basel, Switzerland, 1992
	13	S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability, Springer-Verlag, 1993.
	14	Victor Solo , Xuan Kong, Adaptive signal processing algorithms: stability and performance, Prentice-Hall, Inc., Upper Saddle River, NJ, 1994
	15	W. F. Stout, Almost Sure Convergence, Academic Press, 1974.
	16	Richard S. Sutton, Learning to Predict by the Methods of Temporal Differences, Machine Learning, v.3 n.1, p.9-44, August 1988 [doi>10.1023/A:1022633531479]
	17	Richard S. Sutton , Andrew G. Barto, Introduction to Reinforcement Learning, MIT Press, Cambridge, MA, 1998
	18	V. Tadid, Convergence of stochastic approximation under general noise and stability conditions, Proceedings of the 36 IEEE Conference on Decision and Control, 1997.
	19	John N. Tsitsiklis , Benjamin van Roy, Feature-based methods for large scale dynamic programming, Machine Learning, v.22 n.1-3, p.59-94, Jan./Feb./March 1996 [doi>10.1007/BF00114724]
	20	J. N. Tsitsiklis and B. Van Roy, An analysis of temporal-difference learning with function approximation, IEEE Transactions on Automatic Control, vol. 42, pp. 674-690, 1997.
	21	I.-J. Wang, E. K. P. Chong, and S. R. Kulkarni, Equivalent and sufficient conditions on noise sequences for stochastic approximation algorithms, Advances in Applied Probability, vol. 28, pp. 784- 801, 1996.