Learning representations by back-propagating errors
@article{Rumelhart1986LearningRB,
title={Learning representations by back-propagating errors},
author={David E. Rumelhart and Geoffrey E. Hinton and Ronald J. Williams},
journal={Nature},
year={1986},
volume={323},
pages={533-536}
}We describe a new learning procedure, back-propagation, for networks of neurone-like units. The procedure repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector. As a result of the weight adjustments, internal ‘hidden’ units which are not part of the input or output come to represent important features of the task domain, and the regularities in the task are captured…
21,103 Citations
Distributed bottlenecks for improved generalization in back-propagation networks
- Computer Science
- 1989
The primary goal of any adaptive system that learns by example is to generalize from the training examples to novel inputs, and a method for dynamically producing them, concurrent with back-propagation learning, is explained.
Improving generalization in backpropagation networks with distributed bottlenecks
- Computer ScienceInternational 1989 Joint Conference on Neural Networks
- 1989
A method for dynamically creating hidden-layer bottlenecks, concurrent with backpropagation learning, that compresses the dimensionality of the space spanned by the hidden-unit weight vectors and forms clusters of weight vectors in the low-dimensional space.
Crossprop: Learning Representations by Stochastic Meta-Gradient Descent in Neural Networks
- Computer ScienceECML/PKDD
- 2017
This paper introduces a new incremental learning algorithm called crossprop, which learns incoming weights of hidden units based on the meta-gradient descent approach, that was previously introduced by Sutton (1992) and Schraudolph (1999) for learning step-sizes.
Back propagation neural networks.
- Computer ScienceSubstance use & misuse
- 1998
The BP are networks, whose learning’s function tends to “distribute itself” on the connections, just for the specific correction algorithm of the weights that is utilized.
Reverse Back Propagation to Make Full Use of Derivative
- Computer ScienceArXiv
- 2022
This work provides an approach that conducts the back-propagation again to reverse the traditional back Propagation process to optimize the input loss at the input end of a neural network for better effects without extra costs during the inference time.
A structural learning by adding independent noises to hidden units
- Computer ScienceProceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94)
- 1994
It is shown that a skeletal structure of a network emerges when independent noises are added to the inputs of the hidden units of multilayer perceptron during the learning by error backpropagation.
Generalized Back Propagation for Training Pattern Derivatives
- Computer Science
- 1996
This work generalizes the back propagation learning algorithm for training pattern derivatives and proposes a compromise solution based on the derivatives of training patterns that can improve the network's learning ability in case the authors know the explicit shape of a required diierentiable network function in some area.
References
SHOWING 1-2 OF 2 REFERENCES
Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations
- Computer Science
- 1986
The fundamental principles, basic mechanisms, and formal analyses involved in the development of parallel distributed processing (PDP) systems are presented in individual chapters contributed by…