# Deep learning in neural networks: An overview

@article{Schmidhuber2015DeepLI, title={Deep learning in neural networks: An overview}, author={J{\"u}rgen Schmidhuber}, journal={Neural networks : the official journal of the International Neural Network Society}, year={2015}, volume={61}, pages={ 85-117 } }

## 12,788 Citations

### Deep Reinforcement Learning: An Overview

- Computer ScienceIntelliSys
- 2016

This article reviews the recent advances in deep reinforcement learning with focus on the most used deep architectures such as autoencoders, convolutional neural networks and recurrent neural networks which have successfully been come together with the reinforcement learning framework.

### A Shallow Introduction to Deep Neural Networks

- Computer Science
- 2020

This chapter familiarizes the readers with the major classes of deep neural networks that are frequently used, namely CNN (Convolutional neural Network), RNN (Recurrent Neural Network), DBN (Deep Belief Network), Deep autoencoder, GAN (Generative Adversarial Network) and Deep Recursive Network.

### Deep Learning

- Computer ScienceHandbook of Research on Deep Learning Innovations and Trends
- 2019

This chapter covers the basics ofDeep learning, different architectures of deep learning like artificial neural network, feed forward neural network), CNN, CNN, recurrent neuralnetwork, deep Boltzmann machine, and their comparison and summarizes the applications ofdeep learning in different areas.

### A Comprehensive Study of Deep Neural Networks for Unsupervised Deep Learning

- Computer Science
- 2020

This chapter first study difficulties with neural networks while training with backpropagation-algorithms, then different structures, namely, restricted Boltzmann machines (RBMs), Deep Belief Networks (DBNs), nonlinear autoencoders, deep BoltZmann machines are covered.

### An Overview Studying of Deep Learning

- Computer Science
- 2020

The main advantage of Deep Learning is to create an artificial neural network that can learn and make intelligent decisions on its own and to process large numbers of features makes deep learning very powerful when dealing with unstructured data.

### A Tutorial on Deep Neural Networks for Intelligent Systems

- Computer ScienceArXiv
- 2016

A tutorial of Deep Neural Networks (DNNs), and some insights about the origin of the term "deep"; references to deep learning are given.

### Deep learning and Its Development

- Computer Science
- 2021

The current application of deep learning in various fields is summarized, such as artificial intelligence, computer vision and natural language processing applications, and some open problems for future research are also analyzed.

### Recent Advances in Deep Learning: An Overview

- Computer ScienceArXiv
- 2018

This paper is going to briefly discuss about recent advances in Deep Learning for past few years.

## References

SHOWING 1-10 OF 998 REFERENCES

### Unsupervised Learning in LSTM Recurrent Neural Networks

- Computer ScienceICANN
- 2001

Long Short-Term Memory recurrent networks are trained to maximize two information-theoretic objectives for unsupervised learning: Binary Information Gain Optimization (BINGO) and Nonparametric Entropyoptimization (NEO).

### Neural Networks: Tricks of the Trade

- Computer ScienceLecture Notes in Computer Science
- 2002

It is shown how nonlinear semi-supervised embedding algorithms popular for use with â œshallowâ learning techniques such as kernel methods can be easily applied to deep multi-layer architectures.

### Co-evolving recurrent neurons learn deep memory POMDPs

- Computer Science, BiologyGECCO '05
- 2005

A new neuroevolution algorithm called Hierarchical Enforced SubPopulations that simultaneously evolves networks at two levels of granularity: full networks and network components or neurons is introduced.

### Adaptive dropout for training deep neural networks

- Computer ScienceNIPS
- 2013

A method is described called 'standout' in which a binary belief network is overlaid on a neural network and is used to regularize of its hidden units by selectively setting activities to zero, which achieves lower classification error rates than other feature learning methods, including standard dropout, denoising auto-encoders, and restricted Boltzmann machines.

### Unsupervised Learning Procedures for Neural Networks

- Computer ScienceInt. J. Neural Syst.
- 1991

This paper describes the major approaches that have been taken to model unsupervised learning, and gives an in-depth review of several examples of each approach.

### A Learning Algorithm for Continually Running Fully Recurrent Neural Networks

- Computer ScienceNeural Computation
- 1989

The exact form of a gradient-following learning algorithm for completely recurrent networks running in continually sampled time is derived and used as the basis for practical algorithms for temporal…

### Learning long-term dependencies with gradient descent is difficult

- Computer ScienceIEEE Trans. Neural Networks
- 1994

This work shows why gradient based learning algorithms face an increasingly difficult problem as the duration of the dependencies to be captured increases, and exposes a trade-off between efficient learning by gradient descent and latching on information for long periods.

### How to Construct Deep Recurrent Neural Networks

- Computer ScienceICLR
- 2014

Two novel architectures of a deep RNN are proposed which are orthogonal to an earlier attempt of stacking multiple recurrent layers to build aDeep RNN, and an alternative interpretation is provided using a novel framework based on neural operators.

### Greedy Layer-Wise Training of Deep Networks

- Computer ScienceNIPS
- 2006

These experiments confirm the hypothesis that the greedy layer-wise unsupervised training strategy mostly helps the optimization, by initializing weights in a region near a good local minimum, giving rise to internal distributed representations that are high-level abstractions of the input, bringing better generalization.

### Learning Deep Architectures for AI

- Computer ScienceFound. Trends Mach. Learn.
- 2007

The motivations and principles regarding learning algorithms for deep architectures, in particular those exploiting as building blocks unsupervised learning of single-layer modelssuch as Restricted Boltzmann Machines, used to construct deeper models such as Deep Belief Networks are discussed.