## Neural Computation

October 1, 1997, Vol. 9, No. 7, Pages 1403-1419
(doi: 10.1162/neco.1997.9.7.1403)
© 1997 Massachusetts Institute of Technology
Mean-Field Theory for Batched TD(λ)
Article PDF (126.63 KB)
Abstract

A representation-independent mean-field dynamics is presented for batched TD(λ). The task is learning to predict the outcome of an indirectly observed absorbing Markov process. In the case of linear representations, the discrete-time deterministic iteration is an affine map whose fixed point can be expressed in closed form without the assumption of linearly independent observation vectors. Batched linear TD(λ) is proved to converge with probability 1 for all λ. Theory and simulation agree on a random walk example.