Learning representations by back-propagating errors

  title={Learning representations by back-propagating errors},
  author={David E. Rumelhart and Geoffrey E. Hinton and Ronald J. Williams},
We describe a new learning procedure, back-propagation, for networks of neurone-like units. The procedure repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector. As a result of the weight adjustments, internal ‘hidden’ units which are not part of the input or output come to represent important features of the task domain, and the regularities in the task are captured… 

Distributed bottlenecks for improved generalization in back-propagation networks

The primary goal of any adaptive system that learns by example is to generalize from the training examples to novel inputs, and a method for dynamically producing them, concurrent with back-propagation learning, is explained.

Improving generalization in backpropagation networks with distributed bottlenecks

  • J. Kruschke
  • Computer Science
    International 1989 Joint Conference on Neural Networks
  • 1989
A method for dynamically creating hidden-layer bottlenecks, concurrent with backpropagation learning, that compresses the dimensionality of the space spanned by the hidden-unit weight vectors and forms clusters of weight vectors in the low-dimensional space.

Connectionist Learning Procedures

Crossprop: Learning Representations by Stochastic Meta-Gradient Descent in Neural Networks

This paper introduces a new incremental learning algorithm called crossprop, which learns incoming weights of hidden units based on the meta-gradient descent approach, that was previously introduced by Sutton (1992) and Schraudolph (1999) for learning step-sizes.

Back propagation neural networks.

  • M. Buscema
  • Computer Science
    Substance use & misuse
  • 1998
The BP are networks, whose learning’s function tends to “distribute itself” on the connections, just for the specific correction algorithm of the weights that is utilized.

Reverse Back Propagation to Make Full Use of Derivative

This work provides an approach that conducts the back-propagation again to reverse the traditional back Propagation process to optimize the input loss at the input end of a neural network for better effects without extra costs during the inference time.

A structural learning by adding independent noises to hidden units

It is shown that a skeletal structure of a network emerges when independent noises are added to the inputs of the hidden units of multilayer perceptron during the learning by error backpropagation.

Generalized Back Propagation for Training Pattern Derivatives

This work generalizes the back propagation learning algorithm for training pattern derivatives and proposes a compromise solution based on the derivatives of training patterns that can improve the network's learning ability in case the authors know the explicit shape of a required diierentiable network function in some area.



Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations

The fundamental principles, basic mechanisms, and formal analyses involved in the development of parallel distributed processing (PDP) systems are presented in individual chapters contributed by