Learning the Initial State of a Second-Order Recurrent Neural Network during Regular-Language Inference

Mikel L. Forcada; Rafael C. Carrasco

doi:10.1162/neco.1995.7.5.923

More Information

Monthly

288 pp. per issue

6 x 9, illustrated

ISSN

0899-7667

E-ISSN

1530-888X

2014 Impact factor:

2.21

Neural Computation

September 1995, Vol. 7, No. 5, Pages 923-930

(doi: 10.1162/neco.1995.7.5.923)

Learning the Initial State of a Second-Order Recurrent Neural Network during Regular-Language Inference

Article PDF (396.42 KB)

Mikel L. Forcada

Department de Lienguatges i Sistemes Informàtics, Universitat d'Alacant, E-03071 Alacant, Spain

Rafael C. Carrasco

Department de Lienguatges i Sistemes Informàtics, Universitat d'Alacant, E-03071 Alacant, Spain

Abstract

Recent work has shown that second-order recurrent neural networks (2ORNNs) may be used to infer regular languages. This paper presents a modified version of the real-time recurrent learning (RTRL) algorithm used to train 2ORNNs, that learns the initial state in addition to the weights. The results of this modification, which adds extra flexibility at a negligible cost in time complexity, suggest that it may be used to improve the learning of regular languages when the size of the network is small.