• Corpus ID: 8432978

High-Performance Neural Networks for Visual Object Classification

@article{Ciresan2011HighPerformanceNN,
  title={High-Performance Neural Networks for Visual Object Classification},
  author={Dan C. Ciresan and Ueli Meier and Jonathan Masci and Luca Maria Gambardella and J{\"u}rgen Schmidhuber},
  journal={ArXiv},
  year={2011},
  volume={abs/1102.0183}
}
We present a fast, fully parameterizable GPU implementation of Convolutional Neural Network variants. Our feature extractors are neither carefully designed nor pre-wired, but rather learned in a supervised way. Our deep hierarchical architectures achieve the best published results on benchmarks for object classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple back-propagation perform better… 

Figures and Tables from this paper

Convolutional Neural Network Committees for Handwritten Character Classification

TLDR
This work applies the same architecture to NIST SD 19, a more challenging dataset including lower and upper case letters, and obtains the best results published so far for both NIST digits and NIST letters.

Evaluation of Convolutional Architectures for Offline Handwritten Digit Recognition

TLDR
This paper presents a comparative study of two architectures among those which are implemented for handwriting recognition, Highway CNN and LeNet-5, and demonstrates the advantage of global training and feature mapping on the MNIST dataset.

ImageNet classification with deep convolutional neural networks

TLDR
A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.

Object Recognition with Multi-Scale Pyramidal Pooling Networks

TLDR
A Multi-Scale Pyramidal Pooling Network, featuring a novel pyramidal pooling layer at multiple scales and a novel encoding layer, which improves generalisation performance in comparison to similar neural network architectures, especially when training data is scarce.

An improved faster-RCNN model for handwritten character recognition

TLDR
This work presents an effective and efficient HDR system, introducing a customized faster regional convolutional neural network (Faster-RCNN) and demonstrates that the approach is more competent and able to accurately detect and classify numerals than other recent methods.

ImageNet Classification with Deep Convolutional Neural Networks

TLDR
A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently-developed regularization method called “dropout” that proved to be very effective.

Convolutional-Neural-Network-Based Handwritten Character Recognition: An Approach with Massive Multisource Data

TLDR
Characters of the English alphabet and digit recognition are performed by proposing a custom-tailored CNN model with two different datasets of handwritten images, i.e., Kaggle and MNIST, respectively, which are lightweight but achieve higher accuracies than state-of-the-art models.

A framework for machine vision based on neuro-mimetic front end processing and clustering

TLDR
This paper investigates a framework that combines a front end based on the known neuroscientific findings about the visual pathway, together with unsupervised feature extraction based on clustering, and obtains competitive classification results on standard image databases.

Striving for Simplicity: The All Convolutional Net

TLDR
It is found that max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks.

Deep Learning using Linear Support Vector Machines

TLDR
The results using L2-SVMs show that by simply replacing softmax with linear SVMs gives significant gains on popular deep learning datasets MNIST, CIFAR-10, and the ICML 2013 Representation Learning Workshop's face expression recognition challenge.
...

References

SHOWING 1-10 OF 23 REFERENCES

Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition

TLDR
The aim is to gain insight into different functions by directly comparing them on a fixed architecture for several common object recognition tasks, and empirical results show that a maximum pooling operation significantly outperforms subsampling operations.

3D Object Recognition with Deep Belief Nets

TLDR
A new type of top-level model for Deep Belief Nets is introduced, a third-order Boltzmann machine, trained using a hybrid algorithm that combines both generative and discriminative gradients that substantially outperforms shallow models such as SVMs.

High Performance Convolutional Neural Networks for Document Processing

TLDR
Three novel approaches to speeding up CNNs are presented: a) unrolling convolution, b) using BLAS (basic linear algebra subroutines), and c) using GPUs (graphic processing units).

An Analysis of Single-Layer Networks in Unsupervised Feature Learning

TLDR
The results show that large numbers of hidden nodes and dense feature extraction are critical to achieving high performance—so critical, in fact, that when these parameters are pushed to their limits, they achieve state-of-the-art performance on both CIFAR-10 and NORB using only a single layer of features.

Deep, Big, Simple Neural Nets for Handwritten Digit Recognition

Good old online backpropagation for plain multilayer perceptrons yields a very low 0.35 error rate on the MNIST handwritten digits benchmark. All we need to achieve this best result so far are many

Learning Multiple Layers of Features from Tiny Images

TLDR
It is shown how to train a multi-layer generative model that learns to extract meaningful features which resemble those found in the human visual cortex, using a novel parallelization algorithm to distribute the work among multiple machines connected on a network.

Neocognitron for handwritten digit recognition

Gradient-based learning applied to document recognition

TLDR
This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task, and Convolutional neural networks are shown to outperform all other techniques.

A High-Throughput Screening Approach to Discovering Good Forms of Biologically Inspired Visual Representation

TLDR
This approach can yield significant, reproducible gains in performance across an array of basic object recognition tasks, consistently outperforming a variety of state-of-the-art purpose-built vision systems from the literature.

Best practices for convolutional neural networks applied to visual document analysis

TLDR
A set of concrete bestpractices that document analysis researchers can use to get good results with neural networks, including a simple "do-it-yourself" implementation of convolution with a flexible architecture suitable for many visual document problems.