Gradient-based learning applied to document recognition

  title={Gradient-based learning applied to document recognition},
  author={Yann LeCun and L{\'e}on Bottou and Yoshua Bengio and Patrick Haffner},
  journal={Proc. IEEE},
Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. [] Key Method Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph…

Figures from this paper

Neural Network Based Handwritten Digits Recognition- An Experiment and Analysis

Different network structures and their corresponding recognition rates are compared in this paper to find the optimal parameters of the Neural Network for this application.

Ada-boosting Extreme learning machines for handwritten digit and digit strings recognition

  • Raid Saabni
  • Computer Science
    2015 Fifth International Conference on Digital Information Processing and Communications (ICDIPC)
  • 2015
Very high accurate results in terms of very low error rates while keeping efficient time of online training were achieved by the presented approach, which enables on demand time/precision tradeoff.

Object Recognition with Gradient-based Learning 1 Learning the Right Features

This paper attempts to show that for recognizing simple objects with high shape variability such as handwritten characters, it is possible, and even advantageous, to feed the system directly with minimally processed images and to rely on learning to extract the right set of features.

A Deep Convolutional Neural Network-Based Approach for Handwritten Recognition System

This paper has made an attempt to build a recognition system for multi-scripts such as Odia and Bangla scripts with convolutional neural network (CNN) and defined the data-driven learning mechanism of CNN along with deriving the discriminate features of handwritten images.

Handwritten Digits Recognition Using SVM, KNN, RF and Deep Learning Neural Networks

Several classification algorithms of recognizing numbers from photographic images or with manual input, namely: support vector machine (SVM), Knearest neighbors (KNN), random forest (RF) and several variants of neural networks are discussed.

Artificial neural networks for document analysis and recognition

This paper surveys the most significant problems in the area of offline document image processing, where connectionist-based approaches have been applied and depicts the most promising research guidelines in the field.


A controlled unsupervised learning system was created to allow the networks to continue to learn from experience efficiently after the initial training phase, and a system was included where the weights of the connections, as well as the outputs of the neurons, were slightly mutated during training in order to simulate real world neuron failure, and noise, to allow for improved generality.

Multi-modular architecture based on convolutional neural networks for online handwritten character recognition

In this paper, several convolutional neural network architectures are investigated for online isolated handwritten character recognition (Latin alphabet) and an hybrid architecture called SDTDNN has been derived, it allows the combination of on-line and off-line recognisers.

Neural network based off-line handwritten text recognition system

This dissertation introduces a new system for handwritten text recognition based on an improved neural network design that utilizes the mean quartic error function, where the third and fourth derivatives are non-zero.

A Lightweight Handwriting Recognition System Based on an Improved Convolutional Neural Network

A new improved handwritten character recognition method of convolutional neural network is proposed, which adds Batch Normalization Layer and Residual Network Structure to the general convolutionAL neural network, and adopts multi-scale prediction method.



Off Line Recognition of Handwritten Postal Words Using Neural Networks

A method, “Shortest Path Segmentation” (SPS), which combines dynamic programming and a neural net recognizer for segmenting and recognizing character strings is described, and applications of some of these ideas are described.

Handwritten Digit Recognition Using K Nearest-Neighbor, Radial-Basis Function, and Backpropagation Neural Networks

These results on a large, high input dimensional problem demonstrate that practical constraints including training time, memory usage, and classification time often constrain classifier selection more strongly than small differences in overall error rate.

Connectionist architectural learning for high performance character and speech recognition

  • U. BodenhausenS. Manke
  • Computer Science
    1993 IEEE International Conference on Acoustics, Speech, and Signal Processing
  • 1993
The authors applied an automatic structure optimization (ASO) algorithm to the optimization of multistate time-delay neural networks (MSTDNNs), an extension of the TDNN, which was applied successfully to speech recognition and handwritten character recognition tasks with varying amounts of training data.

A system for the off-line recognition of handwritten text

  • T. Breuel
  • Computer Science
    Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5)
  • 1994
A new system for the recognition of handwritten text that goes from raw, binary scanned images of census forms to ASCII transcriptions of the fields contained within the forms, using two different statistical language models.

Face recognition: a convolutional neural-network approach

A hybrid neural-network for human face recognition which compares favourably with other methods and analyzes the computational complexity and discusses how new classes could be added to the trained recognizer.

A connectionist recognizer for on-line cursive handwriting recognition

  • S. MankeU. Bodenhausen
  • Computer Science
    Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing
  • 1994
The MS-TDNN integrates the high accuracy single character recognition capabilities of a TDNN with a non-linear time alignment procedure (dynamic time warping algorithm) for finding stroke and character boundaries in isolated, handwritten characters and words.

Neural networks for speech and sequence recognition

This paper presents post-processors based on dynamic programming ANN/DP hybrids ANN/HMM Hybrids, and experiments on phoneme recognition with RBFs and online handwriting recognition experiments.

Multi-Digit Recognition Using a Space Displacement Neural Network

A feed-forward network architecture for recognizing an unconstrained handwritten multi-digit string with segmentation done on the feature maps developed in the Space Displacement Neural Network rather than the input (pixel) space.

Shortest path segmentation: a method for training a neural network to recognize character strings

  • C. BurgesO. Matan J. Ben
  • Computer Science
    [Proceedings 1992] IJCNN International Joint Conference on Neural Networks
  • 1992
A method which combines dynamic programming and a neural network recognizer for segmenting and recognizing character strings and has achieved a per-zip-code raw recognition rate of 81% on a 2368 handwritten zip-code test set.

Integrated Segmentation and Recognition of Hand-Printed Numerals

A neural network algorithm that simultaneously segments and recognizes in an integrated system that uses a supervised learning algorithm (backpropagation), but is able to take position-independent information as targets and self-organize the activities of the units in a competitive fashion to infer the positional information.