Neural Network Regularization via Robust Weight Factorization
@article{Rudy2014NeuralNR, title={Neural Network Regularization via Robust Weight Factorization}, author={Jan Rudy and Weiguang Ding and D. Im and Graham W. Taylor}, journal={ArXiv}, year={2014}, volume={abs/1412.6630} }
Regularization is essential when training large neural networks. As deep neural networks can be mathematically interpreted as universal function approximators, they are effective at memorizing sampling noise in the training data. This results in poor generalization to unseen data. Therefore, it is no surprise that a new regularization technique, Dropout, was partially responsible for the now-ubiquitous winning entry to ImageNet 2012 by the University of Toronto. Currently, Dropout (and related… CONTINUE READING
Figures, Tables, and Topics from this paper
5 Citations
Sparse semi-autoencoders to solve the vanishing information problem in multi-layered neural networks
- Computer Science
- Applied Intelligence
- 2018
- 2
Supervised semi-autoencoder learning for multi-layered neural networks
- Computer Science
- 2017 Joint 17th World Congress of International Fuzzy Systems Association and 9th International Conference on Soft Computing and Intelligent Systems (IFSA-SCIS)
- 2017
Stochastic Synapses Enable Efficient Brain-Inspired Learning Machines
- Computer Science, Medicine
- Front. Neurosci.
- 2016
- 80
- PDF
Repeated potentiality assimilation: Simplifying learning procedures by positive, independent and indirect operation for improving generalization and interpretation
- Computer Science
- 2016 International Joint Conference on Neural Networks (IJCNN)
- 2016
- 10
References
SHOWING 1-10 OF 38 REFERENCES
Dropout Training as Adaptive Regularization
- Computer Science, Mathematics
- NIPS
- 2013
- 407
- Highly Influential
- PDF
Training with Noise is Equivalent to Tikhonov Regularization
- Mathematics, Computer Science
- Neural Computation
- 1995
- 806
- PDF
Dropout: a simple way to prevent neural networks from overfitting
- Computer Science
- J. Mach. Learn. Res.
- 2014
- 20,930
- PDF
Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion
- Computer Science, Mathematics
- J. Mach. Learn. Res.
- 2010
- 4,670
- PDF
Extracting and composing robust features with denoising autoencoders
- Mathematics, Computer Science
- ICML '08
- 2008
- 4,289
- Highly Influential
- PDF
On the importance of initialization and momentum in deep learning
- Computer Science
- ICML
- 2013
- 2,682
- Highly Influential
- PDF
Keeping the neural networks simple by minimizing the description length of the weights
- Computer Science
- COLT '93
- 1993
- 749
- PDF