Markovian architectural bias of recurrent neural networks

Abstract

In this paper, we elaborate upon the claim that clustering in the recurrent layer of recurrent neural networks (RNNs) reflects meaningful information processing states even prior to training. By concentrating on activation clusters in RNNs, while not throwing away the continuous state space network dynamics, we extract predictive models that we call neural… (More)
DOI: 10.1109/TNN.2003.820839

Topics

5 Figures and Tables

Cite this paper

@article{Tio2004MarkovianAB, title={Markovian architectural bias of recurrent neural networks}, author={Peter Ti{\~n}o and Michal Cernansk{\'y} and Lubica Benuskov{\'a}}, journal={IEEE Transactions on Neural Networks}, year={2004}, volume={15}, pages={6-15} }