Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier]

  title={Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier]},
  author={Itamar Arel and Derek C. Rose and Thomas P. Karnowski},
  journal={IEEE Computational Intelligence Magazine},
This article provides an overview of the mainstream deep learning approaches and research directions proposed over the past decade. It is important to emphasize that each approach has strengths and "weaknesses, depending on the application and context in "which it is being used. Thus, this article presents a summary on the current state of the deep machine learning field and some perspective into how it may evolve. Convolutional Neural Networks (CNNs) and Deep Belief Networks (DBNs) (and their… 

Figures and Tables from this paper

A survey on deep learning: one small step toward AI

A history of deep learning is reviewed, summarizing the components of Convolutional Neural Networks and Deep Belief Networks together with their learning algorithms and their performances in different applications.


CNN's three most used topologies and learning methods are described and a conceptual explanation of the network is provided to encourage scholars to pursue this area of research by giving them a thorough understanding of CNN.


This review paper provides a brief overview of deep learning algorithms which are the most popular like Auto-Encoders, Convolutional Neural Network, Deep Belief Networks, Long Short-Term Memory Networks, Restricted Boltzmann Machine and Recurrent Neural Networks.

Deep Learning Network

Deep learning is a particular kind of machine learning that provides flexibility and great power, with its attempts to learn in multiple levels of representation with the operations of multiple layers.

An Extensive Survey on Some Deep-Learning Applications

A review of DL and its applications including the recent development in natural language processing (NLP), which has already influenced the search for speech recognition, automatic navigation systems, parallel computations, image processing, ImageNet, natural languageprocessing, representation learning, Google translate, etc.

An End-to-End Trainable Deep Convolutional Neuro-Fuzzy Classifier

This work reports on the design of a new deep neural network that achieves improved interpretability without sacrificing accuracy, and is a hybrid deep learning algorithm based in part upon fuzzy logic, which performs as accurately as existing convolutional neural networks.

A review:deep learning technique for image classification

The fundamental reason for the work exhibited in this paper, is to break down the idea of a deep learning algorithm to be specific, convolutional neural networks (CNN) in image classification. A

A Review about Building Hidden Layer Methods of Deep Learning

This paper emphatically introduces deep networks for unsupervised and supervised learning model and elaborates typical deep learning models and the corresponding extension models and analyzes both advantage and disadvantage of each model and points out each extension method's inheritance relationship with the corresponding typical model.

An Overview Studying of Deep Learning

The main advantage of Deep Learning is to create an artificial neural network that can learn and make intelligent decisions on its own and to process large numbers of features makes deep learning very powerful when dealing with unstructured data.



An empirical evaluation of deep architectures on problems with many factors of variation

A series of experiments indicate that these models with deep architectures show promise in solving harder learning problems that exhibit many factors of variation.

Learning Deep Architectures for AI

The motivations and principles regarding learning algorithms for deep architectures, in particular those exploiting as building blocks unsupervised learning of single-layer modelssuch as Restricted Boltzmann Machines, used to construct deeper models such as Deep Belief Networks are discussed.

DeSTIN: A Scalable Deep Learning Architecture with Application to High-Dimensional Robust Pattern Recognition

A Deep SpatioTemporal Inference Network (DeSTIN) is presented — a scalable deep learning architecture that relies on a combination of unsupervised learning and Bayesian inference, particularly in the context of high-dimensional signal classification.

A new class of convolutional neural networks (SICoNNets) and their application of face detection

  • F. TiviveA. Bouzerdoum
  • Computer Science
    Proceedings of the International Joint Conference on Neural Networks, 2003.
  • 2003
A new class of convolutional neural networks, namely shunting inhibitory convolutionAL neural networks (SICoNNets), is introduced, and a training algorithm is developed using supervised learning based on resilient backpropagation with momentum.

Unsupervised feature learning for audio classification using convolutional deep belief networks

In recent years, deep learning approaches have gained significant interest as a way of building hierarchical representations from unlabeled data. However, to our knowledge, these deep learning

Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations

The convolutional deep belief network is presented, a hierarchical generative model which scales to realistic image sizes and is translation-invariant and supports efficient bottom-up and top-down probabilistic inference.

Greedy Layer-Wise Training of Deep Networks

These experiments confirm the hypothesis that the greedy layer-wise unsupervised training strategy mostly helps the optimization, by initializing weights in a region near a good local minimum, giving rise to internal distributed representations that are high-level abstractions of the input, bringing better generalization.

A Fast Learning Algorithm for Deep Belief Nets

A fast, greedy algorithm is derived that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory.

Best practices for convolutional neural networks applied to visual document analysis

A set of concrete bestpractices that document analysis researchers can use to get good results with neural networks, including a simple "do-it-yourself" implementation of convolution with a flexible architecture suitable for many visual document problems.

Deep learning from temporal coherence in video

A learning method for deep architectures that takes advantage of sequential data, in particular from the temporal coherence that naturally exists in unlabeled video recordings, and is used to improve the performance on a supervised task of interest.