Corpus ID: 14832074

Improving neural networks by preventing co-adaptation of feature detectors

@article{Hinton2012ImprovingNN,
  title={Improving neural networks by preventing co-adaptation of feature detectors},
  author={Geoffrey E. Hinton and Nitish Srivastava and A. Krizhevsky and Ilya Sutskever and R. Salakhutdinov},
  journal={ArXiv},
  year={2012},
  volume={abs/1207.0580}
}
When a large feedforward neural network is trained on a small training set, it typically performs poorly on held-out test data. This "overfitting" is greatly reduced by randomly omitting half of the feature detectors on each training case. This prevents complex co-adaptations in which a feature detector is only helpful in the context of several other specific feature detectors. Instead, each neuron learns to detect a feature that is generally helpful for producing the correct answer given the… Expand
Understanding Dropout
Fast dropout training
Annealed dropout training of deep networks
Adaptive dropout for training deep neural networks
Fast “dropout” training for logistic regression
Disjunctive normal networks
Compacting Neural Network Classifiers via Dropout Training
Techniques for Learning Binary Stochastic Feedforward Neural Networks
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 30 REFERENCES
Adaptive Mixtures of Local Experts
Learning Multiple Layers of Features from Tiny Images
Acoustic Modeling Using Deep Belief Networks
Reducing the Dimensionality of Data with Neural Networks
Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition
Gradient-based learning applied to document recognition
Machine learning
Bayesian learning for neural networks
...
1
2
3
...