Hierarchical Recurrent Neural Networks for Long-Term Dependencies

  title={Hierarchical Recurrent Neural Networks for Long-Term Dependencies},
  author={Salah El Hihi and Yoshua Bengio},
We have already shown that extracting long-term dependencies from sequential data is difficult, both for determimstic dynamical systems such as recurrent networks, and probabilistic models such as hidden Markov models (HMMs) or input/output hidden Markov models (IOHMMs). In practice, to avoid this problem, researchers have used domain specific a-priori knowledge to give meaning to the hidden or state variables representing past context. In this paper, we propose to use a more general type of a… CONTINUE READING



Citations per Year

272 Citations

Semantic Scholar estimates that this publication has 272 citations based on the available data.

See our FAQ for additional information.