The Statistical Recurrent Unit
@article{Oliva2017TheSR, title={The Statistical Recurrent Unit}, author={J. Oliva and B. P{\'o}czos and J. Schneider}, journal={ArXiv}, year={2017}, volume={abs/1703.00381} }
Sophisticated gated recurrent neural network architectures like LSTMs and GRUs have been shown to be highly effective in a myriad of applications. [...] Key Result We show the efficacy of SRUs as compared to LSTMs and GRUs in an unbiased manner by optimizing respective architectures' hyperparameters for both synthetic and real-world tasks.Expand Abstract
Supplemental Code
Figures, Tables, and Topics from this paper
25 Citations
Towards Non-saturating Recurrent Units for Modelling Long-term Dependencies
- Computer Science, Mathematics
- AAAI
- 2019
- 19
- PDF
Learning Long Term Dependencies via Fourier Recurrent Units
- Computer Science, Mathematics
- ICML
- 2018
- 16
- Highly Influenced
- PDF
How much complexity does an RNN architecture need to learn syntax-sensitive dependencies?
- Computer Science, Biology
- ACL
- 2020
- 1
- PDF
Low-pass Recurrent Neural Networks - A memory architecture for longer-term correlation discovery
- Computer Science, Mathematics
- ArXiv
- 2018
- 2
- PDF
Streaming Adaptation of Deep Forecasting Models using Adaptive Recurrent Units
- Computer Science, Mathematics
- KDD
- 2019
- 4
- Highly Influenced
- PDF
ALSTM: Adaptive LSTM for Durative Sequential Data
- Computer Science
- 2018 IEEE 30th International Conference on Tools with Artificial Intelligence (ICTAI)
- 2018
- 3
An Empirical Study of Language CNN for Image Captioning
- Computer Science
- 2017 IEEE International Conference on Computer Vision (ICCV)
- 2017
- 57
- PDF
References
SHOWING 1-10 OF 35 REFERENCES
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Computer Science
- ArXiv
- 2014
- 5,131
- PDF
A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
- Computer Science
- ArXiv
- 2015
- 470
- Highly Influential
- PDF
Advances in optimizing recurrent networks
- Computer Science
- 2013 IEEE International Conference on Acoustics, Speech and Signal Processing
- 2013
- 378
- PDF
Speech recognition with deep recurrent neural networks
- Computer Science
- 2013 IEEE International Conference on Acoustics, Speech and Signal Processing
- 2013
- 5,580
- PDF
On the difficulty of training recurrent neural networks
- Computer Science, Mathematics
- ICML
- 2013
- 2,933
- Highly Influential
- PDF
Long-term recurrent convolutional networks for visual recognition and description
- Computer Science, Medicine
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- 2015
- 3,392
- PDF