Deep, Big, Simple Neural Nets for Handwritten Digit Recognition

@article{Ciresan2010DeepBS,
  title={Deep, Big, Simple Neural Nets for Handwritten Digit Recognition},
  author={Dan C. Ciresan and Ueli Meier and Luca Maria Gambardella and J{\"u}rgen Schmidhuber},
  journal={Neural Computation},
  year={2010},
  volume={22},
  pages={3207-3220}
}
Good old online backpropagation for plain multilayer perceptrons yields a very low 0.35 error rate on the MNIST handwritten digits benchmark. All we need to achieve this best result so far are many hidden layers, many neurons per layer, numerous deformed training images to avoid overfitting, and graphics cards to greatly speed up learning. 
Deep Big Multilayer Perceptrons for Digit Recognition
TLDR
All you need to achieve this until 2011 best result are many hidden layers, many neurons per layer, numerous deformed training images to avoid overfitting, and graphics cards to greatly speed up learning. Expand
Handwritten Digit Recognition with a Committee of Deep Neural Nets on GPUs
TLDR
Another substantial improvement is reported: 0.31% obtained using a committee of MLPs using simple but deep MLPs, outperforming all the previous more complex methods. Expand
Input Transformation and Output Combination for Improved Handwritten Digit Recognition
TLDR
A specific set of input pattern transformations is presented that achieves good results with modestly sized Neural Networks using some heuristics for the construction of an ensemble allows reaching low error rates. Expand
Handwritten Digit Recognition with Pattern Transformations and Neural Network Averaging
TLDR
A relatively modest sized Neural Network trained with standard Back Propagation and combined with a set of input pattern transformations is proposed, giving an encouraging error rate of 0.34% measured on the MNIST dataset. Expand
Fast Handwritten Digit Recognition with Multilayer Ensemble Extreme Learning Machine
TLDR
A novel classifier based on Extreme Learning Machine is proposed that achieves competitive accuracy results while keeping training times low and is called multilayer ensemble Extreme learning Machine. Expand
Combining additive input noise annealing and pattern transformations for improved handwritten character recognition
TLDR
Using a modest sized ANN, the proposed combination of input data transformations enables the achievement of a test error as low as 0.43%, which is up to standard compared to other more complex neural architectures like Convolutional or Deep Neural Networks. Expand
Multi-column deep neural networks for image classification
TLDR
On the very competitive MNIST handwriting benchmark, this method is the first to achieve near-human performance and improves the state-of-the-art on a plethora of common image classification benchmarks. Expand
Digit Recognition Using Convolution Neural Network
TLDR
The main objective of this work is to obtain highest accuracy 99.15% by using convolution neural network (CNN) to recognize the digit without doing too much pre-processing of dataset. Expand
Better Digit Recognition with a Committee of Simple Neural Nets
TLDR
A new method to train the members of a committee of one-hidden-layer neural nets is presented, which obtains a recognition error rate on the MNIST digit recognition benchmark set of 0.39%, on par with state-of-the-art recognition rates of more complicated systems. Expand
Efficient Handwritten Digit Recognition Using Normalized Cross-Correlation
The aim of handwriting recognition is to recognize series of handwritten characters by machines. It is a popular field in computer vision and has several commercial applications which requiresExpand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 34 REFERENCES
A trainable feature extractor for handwritten digit recognition
TLDR
A trainable feature extractor based on the LeNet5 convolutional neural network architecture is introduced to solve the first problem in a black box scheme without prior knowledge on the data and the results show that the system can outperform both SVMs and Le net5 while providing performances comparable to the best performance on this database. Expand
Accelerating Large-Scale Convolutional Neural Networks with Parallel Graphics Multiprocessors
TLDR
This work has adapted the inherent multi-level parallelism of CNNs for Nvidia's CUDA GPU architecture to accelerate the training by two orders of magnitude, allowing to apply CNN architectures to pattern recognition tasks on datasets with high-resolution natural images. Expand
Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure
We show how to pretrain and fine-tune a multilayer neural network to learn a nonlinear transformation from the input space to a lowdimensional feature space in which K-nearest neighbourExpand
Gradient-based learning applied to document recognition
TLDR
This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task, and Convolutional neural networks are shown to outperform all other techniques. Expand
High Performance Convolutional Neural Networks for Document Processing
TLDR
Three novel approaches to speeding up CNNs are presented: a) unrolling convolution, b) using BLAS (basic linear algebra subroutines), and c) using GPUs (graphic processing units). Expand
Reducing the Dimensionality of Data with Neural Networks
TLDR
This work describes an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data. Expand
Training Invariant Support Vector Machines
TLDR
This work reports the recent achievement of the lowest reported test error on the well-known MNIST digit recognition benchmark task, with SVM training times that are also significantly faster than previous SVM methods. Expand
Best practices for convolutional neural networks applied to visual document analysis
TLDR
A set of concrete bestpractices that document analysis researchers can use to get good results with neural networks, including a simple "do-it-yourself" implementation of convolution with a flexible architecture suitable for many visual document problems. Expand
Using GPUs for machine learning algorithms
TLDR
This work proposes a generic 2-layer fully connected neural network GPU implementation which yields over 3/spl times/ speedup for both training and testing with respect to a 3 GHz P4 CPU. Expand
To recognize shapes, first learn to generate images.
TLDR
This chapter describes several of the proposed algorithms and shows how they can be combined to produce hybrid methods that work efficiently in networks with many layers and millions of adaptive connections. Expand
...
1
2
3
4
...