Three Classes of Deep Learning Architectures and Their Applications: A Tutorial Survey
@inproceedings{Deng2012ThreeCO, title={Three Classes of Deep Learning Architectures and Their Applications: A Tutorial Survey}, author={Li Deng}, year={2012} }
In this invited paper, my overview material on the same topic as presented in the plenary overview session of APSIPA-2011 and the tutorial material presented in the same conference (Deng, 2011) are expanded and updated to include more recent developments in deep learning. [] Key Method Three representative deep architectures --deep auto-encoder, deep stacking network, and deep neural network (pre-trained with deep belief network) --one in each of the three classes, are presented in more detail. Next…
Figures and Tables from this paper
115 Citations
A Review on Deep Neural Network Design and Their Applications
- Computer Science2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS)
- 2021
Some commonly used design for deep learning and their practical applications are highlighted and various models of deep neural networks are investigated.
A Study on Deep Learning Architecture and Their Applications
- Computer Science2019 International Conference on Power Electronics, Control and Automation (ICPECA)
- 2019
An intrinsic investigation of the DL architectures and their applications in the practical world is provided, specifically, the overview of autoencoder, restricted Boltzmann machine, generative adversarial network, and convolutional neural network.
A hybrid Deep Boltzmann Functional Link Network for classification problems
- Computer Science2016 IEEE Symposium Series on Computational Intelligence (SSCI)
- 2016
A hybrid deep learning algorithm, namely, the Deep Boltzmann Functional Link Network (DBFLN), which is used to classify the images in the TID2013 data set, based on their depth of distortions, and performance studies show the superior classification ability of the DBFLN.
Vanishing Gradient Mitigation with Deep Learning Neural Network Optimization
- Computer Science2019 7th International Conference on Smart Computing & Communications (ICSCC)
- 2019
Stochastic diagonal approximate greatest descent (SDAGD) demonstrated no sign of vanishing gradient when training with both saturated and unsaturated activation functions, and when hidden layers are added, SDAGD could obtain reasonable results as compared to Stochastic Gradient Descent.
A Comprehensive Review of Deep Learning Architectures for Computer Vision Applications
- Computer Science
- 2021
This article discusses how models have been designed based on the concept of the human brain and the development of a Convolutional Neural Network and its various architectures, which have shown great efficiency and evaluation in object detection, face recognition, image classification, and localization.
Survey on deep learning methods in human action recognition
- Computer ScienceIET Comput. Vis.
- 2017
The authors present an analytical framework to classify and to evaluate these methods based on some important functional measures, and a categorisation of the state-of-the-art approaches in deep learning for human action recognition is presented.
Deep Learning in Kernel Machines
- Computer Science
- 2019
Three deep kernel learning models are developed that analyze the behavior of arc-cosine kernel and modeled a scalable deep kernel machine by incorporating arc- cosine kernel in core vector machines, and a scalableDeep learning architecture with unsupervised feature extraction with promising results.
An Evaluation of Deep Learning Miniature Concerning in Soft Computing
- Computer Science
- 2015
The history of deep learning, Deep Learning Architectures are discussed, and the components of Deep Boltzmann Machines, Deep Stacking Networks, Compound Hierarchical Deep Models, Deep Convolutional Neural Network and Deep Belief Network their learning algorithms are abridge.
ScienceDirect The Third Information Systems International Conference Simulated Annealing Algorithm for Deep Learning
- Computer Science
- 2015
Simulated Annealing (SA) is proposed to improve the performance of Convolution Neural Network (CNN), as an alternative approach for optimal DL using modern optimization technique, i.e. metaheuristic algorithm.
A Comprehensive Classification of Deep Learning Libraries
- Computer ScienceAdvances in Intelligent Systems and Computing
- 2018
The motivation of this article is to create awareness among researchers, especially beginners, regarding the various languages and interfaces that are available to implement deep learning and to provide a simplified ontological basis for selecting between them.
References
SHOWING 1-10 OF 214 REFERENCES
An Overview of Deep-Structured Learning for Information Processing
- Computer Science
- 2011
This paper develops a classificatory scheme to analyze and summarize major work reported in the deep learning literature, and provides a taxonomy-oriented survey on the existing deep architectures, and categorize them into three types: generative, discriminative, and hybrid.
Representation Learning: A Review and New Perspectives
- Computer ScienceIEEE Transactions on Pattern Analysis and Machine Intelligence
- 2013
Recent work in the area of unsupervised feature learning and deep learning is reviewed, covering advances in probabilistic models, autoencoders, manifold learning, and deep networks.
Use of kernel deep convex networks and end-to-end learning for spoken language understanding
- Computer Science2012 IEEE Spoken Language Technology Workshop (SLT)
- 2012
Experimental results demonstrating dramatic error reduction achieved by the K-DCN over both the Boosting-based baseline and the DCN on a domain classification task of SLU, especially when a highly correlated set of features extracted from search query click logs are used.
ImageNet classification with deep convolutional neural networks
- Computer ScienceCommun. ACM
- 2012
A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.
On optimization methods for deep learning
- Computer ScienceICML
- 2011
It is shown that more sophisticated off-the-shelf optimization methods such as Limited memory BFGS (L-BFGS) and Conjugate gradient (CG) with line search can significantly simplify and speed up the process of pretraining deep algorithms.
Why Does Unsupervised Pre-training Help Deep Learning?
- Computer ScienceAISTATS
- 2010
The results suggest that unsupervised pre-training guides the learning towards basins of attraction of minima that support better generalization from the training data set; the evidence from these results supports a regularization explanation for the effect of pre- training.
Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier]
- Computer ScienceIEEE Computational Intelligence Magazine
- 2010
An overview of the mainstream deep learning approaches and research directions proposed over the past decade is provided and some perspective into how it may evolve is presented.
Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion
- Computer ScienceJ. Mach. Learn. Res.
- 2010
This work clearly establishes the value of using a denoising criterion as a tractable unsupervised objective to guide the learning of useful higher level representations.
Scalable stacking and learning for building deep architectures
- Computer Science2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- 2012
The Deep Stacking Network (DSN) is presented, which overcomes the problem of parallelizing learning algorithms for deep architectures and provides a method of stacking simple processing modules in buiding deep architectures, with a convex learning problem in each module.
Deep stacking networks for information retrieval
- Computer Science2013 IEEE International Conference on Acoustics, Speech and Signal Processing
- 2013
It is demonstrated desirable monotonic correlation between NDCG and classification rate in a wide range of IR quality and the weaker correlation and more flat relationship in the high IR-quality region suggest the need for developing new learning objectives and optimization methods.