Corpus ID: 12201880

The Statistical Recurrent Unit

@article{Oliva2017TheSR,
  title={The Statistical Recurrent Unit},
  author={J. Oliva and B. P{\'o}czos and J. Schneider},
  journal={ArXiv},
  year={2017},
  volume={abs/1703.00381}
}
  • J. Oliva, B. Póczos, J. Schneider
  • Published 2017
  • Computer Science, Mathematics
  • ArXiv
  • Sophisticated gated recurrent neural network architectures like LSTMs and GRUs have been shown to be highly effective in a myriad of applications. [...] Key Result We show the efficacy of SRUs as compared to LSTMs and GRUs in an unbiased manner by optimizing respective architectures' hyperparameters for both synthetic and real-world tasks.Expand Abstract
    25 Citations
    Do RNN and LSTM have Long Memory?
    • 3
    • PDF
    Learning Long Term Dependencies via Fourier Recurrent Units
    • 16
    • Highly Influenced
    • PDF
    Recurrent Attentive Neural Process for Sequential Data
    • 9
    • PDF
    How much complexity does an RNN architecture need to learn syntax-sensitive dependencies?
    • 1
    • PDF
    Low-pass Recurrent Neural Networks - A memory architecture for longer-term correlation discovery
    • 2
    • PDF
    Streaming Adaptation of Deep Forecasting Models using Adaptive Recurrent Units
    • 4
    • Highly Influenced
    • PDF
    ALSTM: Adaptive LSTM for Durative Sequential Data
    • 3
    An Empirical Study of Language CNN for Image Captioning
    • 57
    • PDF

    References

    SHOWING 1-10 OF 35 REFERENCES
    Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
    • 5,131
    • PDF
    A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
    • 470
    • Highly Influential
    • PDF
    A Clockwork RNN
    • 319
    • PDF
    Learning Longer Memory in Recurrent Neural Networks
    • 191
    • PDF
    Recurrent Neural Network Regularization
    • 1,599
    • PDF
    Advances in optimizing recurrent networks
    • 378
    • PDF
    Speech recognition with deep recurrent neural networks
    • 5,580
    • PDF
    Long Short-Term Memory
    • 34,907
    • PDF
    On the difficulty of training recurrent neural networks
    • 2,933
    • Highly Influential
    • PDF
    Long-term recurrent convolutional networks for visual recognition and description
    • 3,392
    • PDF