Corpus ID: 3276568

Noisin: Unbiased Regularization for Recurrent Neural Networks

@article{Dieng2018NoisinUR,
  title={Noisin: Unbiased Regularization for Recurrent Neural Networks},
  author={Adji B. Dieng and R. Ranganath and Jaan Altosaar and D. Blei},
  journal={ArXiv},
  year={2018},
  volume={abs/1805.01500}
}
Recurrent neural networks (RNNs) are powerful models of sequential data. They have been successfully used in domains such as text and speech. However, RNNs are susceptible to overfitting; regularization is important. In this paper we develop Noisin, a new method for regularizing RNNs. Noisin injects random noise into the hidden states of the RNN and then maximizes the corresponding marginal likelihood of the data. We show how Noisin applies to any RNN and we study many different types of noise… Expand
8 Citations
Adaptive Noise Injection: A Structure-Expanding Regularization for RNN
  • Highly Influenced
  • PDF
State-Regularized Recurrent Neural Networks
  • 12
  • PDF
Regularizing Recurrent Neural Networks via Sequence Mixup
  • PDF
Understanding Generalization in Recurrent Neural Networks
  • 5
Medi-Care AI: Predicting Medications From Billing Codes via Robust Recurrent Neural Networks
  • 1
  • PDF
How Good is the Bayes Posterior in Deep Neural Networks Really?
  • 67
  • PDF

References

SHOWING 1-10 OF 54 REFERENCES
Regularizing and Optimizing LSTM Language Models
  • 686
  • PDF
A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
  • 1,109
  • PDF
Context dependent recurrent neural network language model
  • 488
  • PDF
Z-Forcing: Training Stochastic Recurrent Networks
  • 112
  • PDF
Sequence to Sequence Learning with Neural Networks
  • 11,931
  • PDF
Speech recognition with deep recurrent neural networks
  • 5,780
  • PDF
Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization
  • 83
  • PDF
On the State of the Art of Evaluation in Neural Language Models
  • 342
  • PDF
Recurrent Highway Networks
  • 317
  • PDF
...
1
2
3
4
5
...