# Deep Q-Networks for Accelerating the Training of Deep Neural Networks

@article{Fu2016DeepQF, title={Deep Q-Networks for Accelerating the Training of Deep Neural Networks}, author={Jie Fu and Zichuan Lin and Miao Liu and Nicholas L{\'e}onard and Jiashi Feng and Tat-Seng Chua}, journal={ArXiv}, year={2016}, volume={abs/1606.01467} }

In this paper, we propose a principled deep reinforcement learning (RL) approach that is able to accelerate the convergence rate of general deep neural networks (DNNs. [...] Key Method The state features of the agent are learned from the weight statistics of the optimizee during training. The reward function of this agent is designed to learn policies that minimize the optimizee's training time given a certain performance goal. Expand

#### 11 Citations

Learning to Optimize Neural Nets

- Computer Science, Mathematics
- ArXiv
- 2017

An extension to Learning to Optimize is developed that is suited to learning optimization algorithms in this setting and it is demonstrated that the learned optimization algorithm consistently outperforms other known optimization algorithms even on unseen tasks and is robust to changes in stochasticity of gradients and the neural net architecture. Expand

Online Learning of a Memory for Learning Rates

- Computer Science, Engineering
- 2018 IEEE International Conference on Robotics and Automation (ICRA)
- 2018

A computationally efficient online meta-learning algorithm that builds and optimizes a memory model of the optimal learning rate landscape from previously observed gradient behaviors, which speeds up learning of MNIST classification and a variety of learning control tasks, either in batch or online learning settings. Expand

Introspection: Accelerating Neural Network Training By Learning Weight Evolution

- Computer Science, Mathematics
- ICLR
- 2017

This paper uses a neural network to learn the training pattern from MNIST classification and utilizes it to accelerate training of neural networks used for CIFAR-10 and ImageNet classification, indicating a general trend in the weight evolution during training of Neural networks. Expand

Playing T-Rex Rush with Deep Reinforcement Learning

- Computer Science
- CNIOT
- 2020

Considering the advantages of deep learning in image feature extraction, in order to achieve autonomous control of the T-Rex Rush game, on the basis of the combination of convolutional neural network and reinforcement learning, a deep neural network based on Q-Learning algorithm is proposed. Expand

MQGrad: Reinforcement Learning of Gradient Quantization in Parameter Server

- Computer Science, Mathematics
- ICTIR
- 2018

Experimental results based on a benchmark dataset showed that MQGrad can accelerate the learning of a large scale deep neural network while keeping its prediction accuracies. Expand

Survey of deep learning and architectures for visual captioning—transitioning between media and natural languages

- Computer Science
- Multimedia Tools and Applications
- 2019

This document will provide a detailed description of the computational neuroscience starting from artificial neural network and how researchers retrospected the drawbacks faced by the previous architectures and paved the way for modern deep learning. Expand

Robotic grasp manipulation using evolutionary computing and deep reinforcement learning

- Computer Science, Mathematics
- Intell. Serv. Robotics
- 2021

This paper has developed learning-based pose estimation by decomposing the problem into both position and orientation learning and develops a deep reinforcement learning (DRL) model which is named as grasp deep Q-network (GDQN). Expand

Unsupervised Deep Learning for Data-Driven Reliability and Risk Analysis of Engineered Systems

- Computer Science
- 2017

Autoencoders, a class of deep learning frameworks, are reviewed and it is shown that they can not only classify the states at a good accuracy, but also help to discover the failure mechanism. Expand

Comprehensive Control System for Gathering Pipe Network Operation Based on Reinforcement Learning

- Computer Science
- ICNCC 2018
- 2018

A DQN-based algorithm is proposed that completes the temperature and pressure control in the pipeline and joint optimization of valve opening and heating furnace and pressure pump and shows excellent control effect and robustness. Expand

Web-scale Multimedia Search for Internet Video Content

- Computer Science
- WSDM
- 2016

The proposed method provides a new dimension of looking at content-based video search, from finding a simple concept like "puppy" to searching a complex incident like "a scene in urban area where people running away after an explosion". Expand

#### References

SHOWING 1-10 OF 38 REFERENCES

Using Deep Q-Learning to Control Optimization Hyperparameters

- Mathematics, Computer Science
- ArXiv
- 2016

A novel definition of the reinforcement learning state, actions and reward function that allows a deep Q-network to learn to control an optimization hyperparameter is presented and it is shown that the DQN's q-values associated with optimal action converge and that the Q-gradient descent algorithms outperform gradient descent with an Armijo or nonmonotone line search. Expand

Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning

- Computer Science, Mathematics
- ICLR
- 2016

This work defines a novel method of multitask and transfer learning that enables an autonomous agent to learn how to behave in multiple tasks simultaneously, and then generalize its knowledge to new domains, and uses Atari games as a testing environment to demonstrate these methods. Expand

Recurrent Reinforcement Learning: A Hybrid Approach

- Computer Science
- ArXiv
- 2015

This work investigates a deep-learning approach to learning the representation of states in partially observable tasks, with minimal prior knowledge of the domain, and proposes a new family of hybrid models that combines the strength of both supervised learning and reinforcement learning, trained in a joint fashion. Expand

On the importance of initialization and momentum in deep learning

- Computer Science
- ICML
- 2013

It is shown that when stochastic gradient descent with momentum uses a well-designed random initialization and a particular type of slowly increasing schedule for the momentum parameter, it can train both DNNs and RNNs to levels of performance that were previously achievable only with Hessian-Free optimization. Expand

Why Does Unsupervised Pre-training Help Deep Learning?

- Computer Science
- AISTATS
- 2010

The results suggest that unsupervised pre-training guides the learning towards basins of attraction of minima that support better generalization from the training data set; the evidence from these results supports a regularization explanation for the effect of pre- training. Expand

Speeding Up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves

- Computer Science
- IJCAI
- 2015

This paper mimics the early termination of bad runs using a probabilistic model that extrapolates the performance from the first part of a learning curve, enabling state-of-the-art hyperparameter optimization methods for DNNs to find DNN settings that yield better performance than those chosen by human experts. Expand

Online Batch Selection for Faster Training of Neural Networks

- Computer Science, Mathematics
- ArXiv
- 2015

This work investigates online batch selection strategies for two state-of-the-art methods of stochastic gradient-based optimization, AdaDelta and Adam, and proposes a simple strategy where all datapoints are ranked w.r.t. their latest known loss value and the probability to be selected decays exponentially as a function of rank. Expand

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

- Computer Science
- ICML
- 2015

Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Expand

Weight Features for Predicting Future Model Performance of Deep Neural Networks

- Computer Science
- IJCAI
- 2016

The findings demonstrate that using weight features can help construct prediction models with a smaller number of training samples and terminate underperformance runs at an earlier stage of the learning process of DNNs than the conventional use of learning curve, thus facilitating the speed-up of hyperparameter searches. Expand

Human-level control through deep reinforcement learning

- Computer Science, Medicine
- Nature
- 2015

This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks. Expand