Corpus ID: 7182867

Dropout Training as Adaptive Regularization

@inproceedings{Wager2013DropoutTA,
  title={Dropout Training as Adaptive Regularization},
  author={S. Wager and Sida I. Wang and Percy Liang},
  booktitle={NIPS},
  year={2013}
}

S. Wager, Sida I. Wang, Percy Liang

Published in NIPS 2013

Computer Science, Mathematics

Dropout and other feature noising schemes control overfitting by artificially corrupting the training data. For generalized linear models, dropout performs a form of adaptive regularization. Using this viewpoint, we show that the dropout regularizer is first-order equivalent to an L2 regularizer applied after scaling the features by an estimate of the inverse diagonal Fisher information matrix. We also establish a connection to AdaGrad, an online learning algorithm, and find that a close… CONTINUE READING

View PDF on arXiv

Save to Library

Create Alert

Cite

Launch Research Feed