Deep Learning

  title={Deep Learning},
  author={Ian J. Goodfellow and Yoshua Bengio and Aaron C. Courville},
Machine-learning technology powers many aspects of modern society: from web searches to content filtering on social networks to recommendations on e-commerce websites, and it is increasingly present in consumer products such as cameras and smartphones. [] Key Method Representation learning is a set of methods that allows a machine to be fed with raw data and to automatically discover the representations needed for detection or classification.

Neural Networks for Survey Researchers

This article describes what neural networks are and how they learn, considers their strengths and weaknesses as a machine learning approach, and illustrates how they perform on a classification task predicting survey response from respondents’ (and nonrespondents’) prior known demographics.

Deep Learning: A Primer for Radiologists.

  • G. ChartrandP. Cheng A. Tang
  • Computer Science
    Radiographics : a review publication of the Radiological Society of North America, Inc
  • 2017
The key concepts of deep learning for clinical radiologists are reviewed, technical requirements are discussed, emerging applications in clinical radiology are described, and limitations and future directions in this field are outlined.

Deep learning: a branch of machine learning

A broad writing survey is completed and the utilization of deep learning in different fields is reviewed and how and in what real applications deep learning algorithms have been used are shown.

Image Classification Using Deep Neural Networks: Transfer Learning and the Handling of Unknown Images

This research set out to evaluate the effect of transfer learning on the performance of a Deep Neural Network (DNN) and found that pre-trained AlexNet was selected, modified and retrained for 3 image classification applications with a modest database and gave 99% classification accuracy using transfer learning.

Tricks from Deep Learning

A way to dramatically reduce the size of the tape when performing reverse-mode AD on a (theoretically) time-reversible process like an ODE integrator; and a new mathematical insight that allows for the implementation of a stochastic Newton's method are discussed.

Convolutional neural network and its pretrained models for image classification and object detection: A survey

This paper presents detailed and analytical literature starting from the very elementary level to the recent trends of this trending technology while focusing on the most used DL model, that is, convolutional neural network and its pretrained models for image classification and object detection.

Recent advances in deep learning

Focusing on recent developments in DL architectures and their applications, the articles in this issue are classified into four categories: (1) deep architectures and conventional neural networks, (2) incremental learning, (3) recurrent neural Networks, and (4) generative models and adversarial examples.

Deep learning in the automotive industry: Applications and tools

An end-to-end deep learning application utilizing a mobile app for data collection and process support, and an Amazon-based cloud backend for storage and training, and the effectiveness of the trained classifier in a real world setting during manufacturing process is demonstrated.



Learning Multiple Layers of Features from Tiny Images

It is shown how to train a multi-layer generative model that learns to extract meaningful features which resemble those found in the human visual cortex, using a novel parallelization algorithm to distribute the work among multiple machines connected on a network.

An Analysis of Single-Layer Networks in Unsupervised Feature Learning

The results show that large numbers of hidden nodes and dense feature extraction are critical to achieving high performance—so critical, in fact, that when these parameters are pushed to their limits, they achieve state-of-the-art performance on both CIFAR-10 and NORB using only a single layer of features.

Machine learning - a probabilistic perspective

  • K. Murphy
  • Computer Science
    Adaptive computation and machine learning series
  • 2012
This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach, and is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.

Measuring Invariances in Deep Networks

A number of empirical tests are proposed that directly measure the degree to which these learned features are invariant to different input transformations and find that stacked autoencoders learn modestly increasingly invariant features with depth when trained on natural images and convolutional deep belief networks learn substantially more invariant Features in each layer.

Sparse Feature Learning for Deep Belief Networks

This work proposes a simple criterion to compare and select different unsupervised machines based on the trade-off between the reconstruction error and the information content of the representation, and describes a novel and efficient algorithm to learn sparse representations.

Gradient-based learning applied to document recognition

This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task, and Convolutional neural networks are shown to outperform all other techniques.

Zero-Shot Learning Through Cross-Modal Transfer

This work introduces a model that can recognize objects in images even if no training data is available for the object class, and uses novelty detection methods to differentiate unseen classes from seen classes.

Building high-level features using large scale unsupervised learning

Contrary to what appears to be a widely-held intuition, the experimental results reveal that it is possible to train a face detector without having to label images as containing a face or not.

Beyond simple features: A large-scale feature search approach to unconstrained face recognition

This work demonstrates a large-scale feature search approach to generating new, more powerful feature representations in which a multitude of complex, nonlinear, multilayer neuromorphic feature representations are randomly generated and screened to find those best suited for the task at hand.

How transferable are features in deep neural networks?

This paper quantifies the generality versus specificity of neurons in each layer of a deep convolutional neural network and reports a few surprising results, including that initializing a network with transferred features from almost any number of layers can produce a boost to generalization that lingers even after fine-tuning to the target dataset.