Neural Optimizer Search with Reinforcement Learning

@inproceedings{Bello2017NeuralOS,
  title={Neural Optimizer Search with Reinforcement Learning},
  author={Irwan Bello and Barret Zoph and Vijay Vasudevan and Quoc V. Le},
  booktitle={ICML},
  year={2017}
}
We present an approach to automate the process of discovering optimization methods, with a focus on deep learning architectures. We train a Recurrent Neural Network controller to generate a string in a specific domain language that describes a mathematical update equation based on a list of primitive functions, such as the gradient, running average of the gradient, etc. The controller is trained with Reinforcement Learning to maximize the performance of a model after a few epochs. On CIFAR-10… CONTINUE READING
Highly Cited
This paper has 26 citations. REVIEW CITATIONS

9 Figures & Tables

Topics

Statistics

0204020172018
Citations per Year

Citation Velocity: 17

Averaging 17 citations per year over the last 2 years.

Learn more about how we calculate this metric in our FAQ.
  • GitHub repos referencing this paper

  • Presentations referencing similar topics