Gradient-based learning applied to document recognition

  title={Gradient-based learning applied to document recognition},
  author={Yann LeCun and L{\'e}on Bottou and Yoshua Bengio and Patrick Haffner},
  journal={Proc. IEEE},
Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. [] Key Method Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph…

Figures from this paper

Neural Network Based Handwritten Digits Recognition- An Experiment and Analysis

Different network structures and their corresponding recognition rates are compared in this paper to find the optimal parameters of the Neural Network for this application.

Neural Networks for Handwriting Recognition

In this chapter a novel kind of Recurrent Neural Networks (RNNs) is described, specifically designed for sequence labeling tasks where the data is hard to segment and contains long-range, bidirectional or multidirectional interdependencies.

Ada-boosting Extreme learning machines for handwritten digit and digit strings recognition

  • Raid Saabni
  • Computer Science
    2015 Fifth International Conference on Digital Information Processing and Communications (ICDIPC)
  • 2015
Very high accurate results in terms of very low error rates while keeping efficient time of online training were achieved by the presented approach, which enables on demand time/precision tradeoff.

Object Recognition with Gradient-based Learning 1 Learning the Right Features

This paper attempts to show that for recognizing simple objects with high shape variability such as handwritten characters, it is possible, and even advantageous, to feed the system directly with minimally processed images and to rely on learning to extract the right set of features.

A Deep Convolutional Neural Network-Based Approach for Handwritten Recognition System

This paper has made an attempt to build a recognition system for multi-scripts such as Odia and Bangla scripts with convolutional neural network (CNN) and defined the data-driven learning mechanism of CNN along with deriving the discriminate features of handwritten images.

Handwritten Digits Recognition Using SVM, KNN, RF and Deep Learning Neural Networks

Several classification algorithms of recognizing numbers from photographic images or with manual input, namely: support vector machine (SVM), Knearest neighbors (KNN), random forest (RF) and several variants of neural networks are discussed.

Artificial neural networks for document analysis and recognition

This paper surveys the most significant problems in the area of offline document image processing, where connectionist-based approaches have been applied and depicts the most promising research guidelines in the field.


A controlled unsupervised learning system was created to allow the networks to continue to learn from experience efficiently after the initial training phase, and a system was included where the weights of the connections, as well as the outputs of the neurons, were slightly mutated during training in order to simulate real world neuron failure, and noise, to allow for improved generality.

Multi-modular architecture based on convolutional neural networks for online handwritten character recognition

In this paper, several convolutional neural network architectures are investigated for online isolated handwritten character recognition (Latin alphabet) and an hybrid architecture called SDTDNN has been derived, it allows the combination of on-line and off-line recognisers.

Neural network based off-line handwritten text recognition system

This dissertation introduces a new system for handwritten text recognition based on an improved neural network design that utilizes the mean quartic error function, where the third and fourth derivatives are non-zero.

Off Line Recognition of Handwritten Postal Words Using Neural Networks

A method, “Shortest Path Segmentation” (SPS), which combines dynamic programming and a neural net recognizer for segmenting and recognizing character strings is described, and applications of some of these ideas are described.

Handwritten Digit Recognition Using K Nearest-Neighbor, Radial-Basis Function, and Backpropagation Neural Networks

These results on a large, high input dimensional problem demonstrate that practical constraints including training time, memory usage, and classification time often constrain classifier selection more strongly than small differences in overall error rate.

Comparing different neural network architectures for classifying handwritten digits

The authors propose a novel way of organizing the network architectures by training several small networks so as to deal separately with subsets of the problem, and then combining the results.

Connectionist architectural learning for high performance character and speech recognition

  • U. BodenhausenS. Manke
  • Computer Science
    1993 IEEE International Conference on Acoustics, Speech, and Signal Processing
  • 1993
The authors applied an automatic structure optimization (ASO) algorithm to the optimization of multistate time-delay neural networks (MSTDNNs), an extension of the TDNN, which was applied successfully to speech recognition and handwritten character recognition tasks with varying amounts of training data.

A system for the off-line recognition of handwritten text

  • T. Breuel
  • Computer Science
    Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5)
  • 1994
A new system for the recognition of handwritten text that goes from raw, binary scanned images of census forms to ASCII transcriptions of the fields contained within the forms, using two different statistical language models.

Distortion Invariant Object Recognition in the Dynamic Link Architecture

An object recognition system based on the dynamic link architecture, an extension to classical artificial neural networks (ANNs), is presented and the implementation on a transputer network achieved recognition of human faces and office objects from gray-level camera images.

Face recognition: a convolutional neural-network approach

A hybrid neural-network for human face recognition which compares favourably with other methods and analyzes the computational complexity and discusses how new classes could be added to the trained recognizer.

A connectionist recognizer for on-line cursive handwriting recognition

  • S. MankeU. Bodenhausen
  • Computer Science
    Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing
  • 1994
The MS-TDNN integrates the high accuracy single character recognition capabilities of a TDNN with a non-linear time alignment procedure (dynamic time warping algorithm) for finding stroke and character boundaries in isolated, handwritten characters and words.

Neural networks for speech and sequence recognition

This paper presents post-processors based on dynamic programming ANN/DP hybrids ANN/HMM Hybrids, and experiments on phoneme recognition with RBFs and online handwriting recognition experiments.

Multi-Digit Recognition Using a Space Displacement Neural Network

A feed-forward network architecture for recognizing an unconstrained handwritten multi-digit string with segmentation done on the feature maps developed in the Space Displacement Neural Network rather than the input (pixel) space.