• Corpus ID: 7509837

Three Classes of Deep Learning Architectures and Their Applications: A Tutorial Survey

@inproceedings{Deng2012ThreeCO,
  title={Three Classes of Deep Learning Architectures and Their Applications: A Tutorial Survey},
  author={Li Deng},
  year={2012}
}
  • L. Deng
  • Published 2012
  • Computer Science
In this invited paper, my overview material on the same topic as presented in the plenary overview session of APSIPA-2011 and the tutorial material presented in the same conference (Deng, 2011) are expanded and updated to include more recent developments in deep learning. [] Key Method Three representative deep architectures --deep auto-encoder, deep stacking network, and deep neural network (pre-trained with deep belief network) --one in each of the three classes, are presented in more detail. Next…
microsoft.com
115 Citations
Highly Influential Citations
7
Background Citations
40
Methods Citations
21

Figures and Tables from this paper

115 Citations

A Review on Deep Neural Network Design and Their Applications
  • N. Praveena, K. Vivekanandan
  • Computer Science
    2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS)
  • 2021
TLDR
Some commonly used design for deep learning and their practical applications are highlighted and various models of deep neural networks are investigated.
A Study on Deep Learning Architecture and Their Applications
TLDR
An intrinsic investigation of the DL architectures and their applications in the practical world is provided, specifically, the overview of autoencoder, restricted Boltzmann machine, generative adversarial network, and convolutional neural network.
A hybrid Deep Boltzmann Functional Link Network for classification problems
TLDR
A hybrid deep learning algorithm, namely, the Deep Boltzmann Functional Link Network (DBFLN), which is used to classify the images in the TID2013 data set, based on their depth of distortions, and performance studies show the superior classification ability of the DBFLN.
Vanishing Gradient Mitigation with Deep Learning Neural Network Optimization
  • H. Tan, K. Lim
  • Computer Science
    2019 7th International Conference on Smart Computing & Communications (ICSCC)
  • 2019
TLDR
Stochastic diagonal approximate greatest descent (SDAGD) demonstrated no sign of vanishing gradient when training with both saturated and unsaturated activation functions, and when hidden layers are added, SDAGD could obtain reasonable results as compared to Stochastic Gradient Descent.
A Comprehensive Review of Deep Learning Architectures for Computer Vision Applications
TLDR
This article discusses how models have been designed based on the concept of the human brain and the development of a Convolutional Neural Network and its various architectures, which have shown great efficiency and evaluation in object detection, face recognition, image classification, and localization.
Survey on deep learning methods in human action recognition
TLDR
The authors present an analytical framework to classify and to evaluate these methods based on some important functional measures, and a categorisation of the state-of-the-art approaches in deep learning for human action recognition is presented.
Deep Learning in Kernel Machines
TLDR
Three deep kernel learning models are developed that analyze the behavior of arc-cosine kernel and modeled a scalable deep kernel machine by incorporating arc- cosine kernel in core vector machines, and a scalableDeep learning architecture with unsupervised feature extraction with promising results.
An Evaluation of Deep Learning Miniature Concerning in Soft Computing
TLDR
The history of deep learning, Deep Learning Architectures are discussed, and the components of Deep Boltzmann Machines, Deep Stacking Networks, Compound Hierarchical Deep Models, Deep Convolutional Neural Network and Deep Belief Network their learning algorithms are abridge.
ScienceDirect The Third Information Systems International Conference Simulated Annealing Algorithm for Deep Learning
TLDR
Simulated Annealing (SA) is proposed to improve the performance of Convolution Neural Network (CNN), as an alternative approach for optimal DL using modern optimization technique, i.e. metaheuristic algorithm.
A Comprehensive Classification of Deep Learning Libraries
TLDR
The motivation of this article is to create awareness among researchers, especially beginners, regarding the various languages and interfaces that are available to implement deep learning and to provide a simplified ontological basis for selecting between them.
...
...

References

SHOWING 1-10 OF 214 REFERENCES
An Overview of Deep-Structured Learning for Information Processing
TLDR
This paper develops a classificatory scheme to analyze and summarize major work reported in the deep learning literature, and provides a taxonomy-oriented survey on the existing deep architectures, and categorize them into three types: generative, discriminative, and hybrid.
Representation Learning: A Review and New Perspectives
TLDR
Recent work in the area of unsupervised feature learning and deep learning is reviewed, covering advances in probabilistic models, autoencoders, manifold learning, and deep networks.
Use of kernel deep convex networks and end-to-end learning for spoken language understanding
TLDR
Experimental results demonstrating dramatic error reduction achieved by the K-DCN over both the Boosting-based baseline and the DCN on a domain classification task of SLU, especially when a highly correlated set of features extracted from search query click logs are used.
ImageNet classification with deep convolutional neural networks
TLDR
A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.
On optimization methods for deep learning
TLDR
It is shown that more sophisticated off-the-shelf optimization methods such as Limited memory BFGS (L-BFGS) and Conjugate gradient (CG) with line search can significantly simplify and speed up the process of pretraining deep algorithms.
Why Does Unsupervised Pre-training Help Deep Learning?
TLDR
The results suggest that unsupervised pre-training guides the learning towards basins of attraction of minima that support better generalization from the training data set; the evidence from these results supports a regularization explanation for the effect of pre- training.
Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier]
TLDR
An overview of the mainstream deep learning approaches and research directions proposed over the past decade is provided and some perspective into how it may evolve is presented.
Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion
TLDR
This work clearly establishes the value of using a denoising criterion as a tractable unsupervised objective to guide the learning of useful higher level representations.
Scalable stacking and learning for building deep architectures
  • L. Deng, Dong Yu, John C. Platt
  • Computer Science
    2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • 2012
TLDR
The Deep Stacking Network (DSN) is presented, which overcomes the problem of parallelizing learning algorithms for deep architectures and provides a method of stacking simple processing modules in buiding deep architectures, with a convex learning problem in each module.
Deep stacking networks for information retrieval
TLDR
It is demonstrated desirable monotonic correlation between NDCG and classification rate in a wide range of IR quality and the weaker correlation and more flat relationship in the high IR-quality region suggest the need for developing new learning objectives and optimization methods.
...
...