Monthly
288 pp. per issue
6 x 9, illustrated
ISSN
0899-7667
E-ISSN
1530-888X
2014 Impact factor:
2.21

Neural Computation

July 1993, Vol. 5, No. 4, Pages 613-624.
(doi: 10.1162/neco.1993.5.4.613)
© 1993 Massachusetts Institute of Technology
Improving Generalization for Temporal Difference Learning: The Successor Representation
Article PDF (848.46 KB)
Abstract

Estimation of returns over time, the focus of temporal difference (TD) algorithms, imposes particular constraints on good function approximators or representations. Appropriate generalization between states is determined by how similar their successors are, and representations should follow suit. This paper shows how TD machinery can be used to learn such representations, and illustrates, using a navigation task, the appropriately distributed nature of the result.