Learning representations by back-propagating errors

@article{Rumelhart1986LearningRB,
  title={Learning representations by back-propagating errors},
  author={D. Rumelhart and Geoffrey E. Hinton and Ronald J. Williams},
  journal={Nature},
  year={1986},
  volume={323},
  pages={533-536}
}
We describe a new learning procedure, back-propagation, for networks of neurone-like units. The procedure repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector. As a result of the weight adjustments, internal ‘hidden’ units which are not part of the input or output come to represent important features of the task domain, and the regularities in the task are captured… Expand
View on Springer
cs.utoronto.ca
18,330 Citations
Highly Influential Citations
632
Background Citations
2,921
Methods Citations
4,235
Results Citations
28

18,330 Citations

Citation Type

Citation Type

Distributed bottlenecks for improved generalization in back-propagation networks
TLDR
The primary goal of any adaptive system that learns by example is to generalize from the training examples to novel inputs, and a method for dynamically producing them, concurrent with back-propagation learning, is explained. Expand
Learning sets of filters using back-propagation
TLDR
Further research is described on back-propagation for layered networks of deterministic, neuron-like units and an example in which a network learns a set of filters that enable it to discriminate formant-like patterns in the presence of noise. Expand
Improving generalization in backpropagation networks with distributed bottlenecks
  • J. Kruschke
  • Computer Science
  • International 1989 Joint Conference on Neural Networks
  • 1989
TLDR
A method for dynamically creating hidden-layer bottlenecks, concurrent with backpropagation learning, that compresses the dimensionality of the space spanned by the hidden-unit weight vectors and forms clusters of weight vectors in the low-dimensional space. Expand
Connectionist Learning Procedures
TLDR
These relatively simple, gradient-descent learning procedures work well for small tasks and the new challenge is to find ways of improving their convergence rate and their generalization abilities so that they can be applied to larger, more realistic tasks. Expand
20 – CONNECTIONIST LEARNING PROCEDURES1
TLDR
These relatively simple, gradient-descent learning procedures work well for small tasks, and the new challenge is to find ways of improving their convergence rate and their generalization abilities so that they can be applied to larger, more realistic tasks. Expand
Crossprop: Learning Representations by Stochastic Meta-Gradient Descent in Neural Networks
TLDR
This paper introduces a new incremental learning algorithm called crossprop, which learns incoming weights of hidden units based on the meta-gradient descent approach, that was previously introduced by Sutton (1992) and Schraudolph (1999) for learning step-sizes. Expand
Back propagation neural networks.
  • M. Buscema
  • Computer Science, Medicine
  • Substance use & misuse
  • 1998
TLDR
The BP are networks, whose learning’s function tends to “distribute itself” on the connections, just for the specific correction algorithm of the weights that is utilized. Expand
A structural learning by adding independent noises to hidden units
TLDR
It is shown that a skeletal structure of a network emerges when independent noises are added to the inputs of the hidden units of multilayer perceptron during the learning by error backpropagation. Expand
Generalized Back Propagation for Training Pattern Derivatives
We consider a multi-layered neural network trained with back propagation. This neural network model realizes a vector function. Sometimes we know the shape of a required function in some area (e.g.… Expand
Improved generalization of neural classifiers with enforced internal representation
TLDR
This paper proposes an efficient framework to enforce a transparent internal knowledge representation in BP-networks during training that will be forced to group around three possible values, namely 1, 0 and 0.5. Expand
...
1
2
3
4
5
...

References

SHOWING 1-3 OF 3 REFERENCES
Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations
The fundamental principles, basic mechanisms, and formal analyses involved in the development of parallel distributed processing (PDP) systems are presented in individual chapters contributed by… Expand
Principles of Neurodynamics {Spartan
  • 1961
Principles of neurodynamics