Corpus ID: 210861129

LipschitzLR: Using theoretically computed adaptive learning rates for fast convergence.

@inproceedings{Yedida2019LipschitzLRUT,
  title={LipschitzLR: Using theoretically computed adaptive learning rates for fast convergence.},
  author={Rahul Yedida and Snehanshu Saha},
  year={2019}
}
  • Rahul Yedida, Snehanshu Saha
  • Published 2019
  • Mathematics, Computer Science
  • Optimizing deep neural networks is largely thought to be an empirical process, requiring manual tuning of several hyper-parameters, such as learning rate, weight decay, and dropout rate. Arguably, the learning rate is the most important of these to tune, and this has gained more attention in recent works. In this paper, we propose a novel method to compute the learning rate for training deep neural networks with stochastic gradient descent. We first derive a theoretical framework to compute… CONTINUE READING

    Create an AI-powered research feed to stay up to date with new papers like this posted to ArXiv