Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN

@article{Li2018IndependentlyRN,
  title={Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN},
  author={Shuai Li and Wanqing Li and Chris Cook and Ce Zhu and Yanbo Gao},
  journal={2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2018},
  pages={5457-5466}
}
Recurrent neural networks (RNNs) have been widely used for processing sequential data. However, RNNs are commonly difficult to train due to the well-known gradient vanishing and exploding problems and hard to learn long-term patterns. Long short-term memory (LSTM) and gated recurrent unit (GRU) were developed to address these problems, but the use of hyperbolic tangent and the sigmoid action functions results in gradient decay over layers. Consequently, construction of an efficiently trainable… CONTINUE READING
Highly Cited
This paper has 39 citations. REVIEW CITATIONS
Related Discussions
This paper has been referenced on Twitter 96 times. VIEW TWEETS

Citations

Publications citing this paper.
Showing 1-10 of 14 extracted citations

Smartphone-sensors Based Activity Recognition Using IndRNN

UbiComp/ISWC Adjunct • 2018
View 3 Excerpts
Method Support
Highly Influenced

Fault Detection of Permanent Magnet Synchronous Motor Based on Deep Learning Method

2018 21st International Conference on Electrical Machines and Systems (ICEMS) • 2018
View 1 Excerpt

References

Publications referenced by this paper.
Showing 1-10 of 49 references

Deep Residual Learning for Image Recognition

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) • 2016
View 7 Excerpts
Highly Influenced

NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) • 2016
View 5 Excerpts
Highly Influenced

Deep Learning on Lie Groups for Skeleton-Based Action Recognition

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) • 2017
View 1 Excerpt

Interpretable 3D Human Action Analysis with Temporal Convolutional Networks

2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) • 2017

J

K. Greff, R. K. Srivastava
Koutnı́k, B. R. Steunebrink, and J. Schmidhuber. Lstm: A search space odyssey. IEEE transactions on neural networks and learning systems • 2017
View 1 Excerpt