Corpus ID: 14814125

Bayesian Back-Propagation

@article{Buntine1991BayesianB,
  title={Bayesian Back-Propagation},
  author={Wray L. Buntine and Andreas S. Weigend},
  journal={Complex Syst.},
  year={1991},
  volume={5}
}
Connectionist feed-forward networks, t rained with backpropagat ion, can be used both for nonlinear regression and for (discrete one-of-C ) classification. This paper presents approximate Bayesian meth ods to statistical components of back-propagat ion: choosing a cost funct ion and penalty term (interpreted as a form of prior probability), pruning insignifican t weights, est imat ing the uncertainty of weights, predict ing for new pat terns ("out -of-sample") , est imating the uncertainty in… Expand
Bayesian learning in neural networks for sequence processing
The Bayesian modelli ng framework, first applied to neural networks in [Buntine and Weigend, 1991], [MacKay, 1992] and [Neal, 1992], is appealing for several reasons. First, instead of obtaining anExpand
Posterior Simulation for FeedForward Neural Network
1 Motivation We are interested in Bayesian inference and prediction with feed-forward neural network models (FFNN's), speciically those with one hidden layer with M hidden nodes, p input nodes, 1Expand
Bayesian Inference In Neural Networks
TLDR
In the present article, some aspects of the Bayesian method will be illustrated in a 2-group classification problem and then applied at a very rudimentary level to the development of a neural network for the prediction of tornados. Expand
Bayesian training of backpropagation networks by the hybrid Monte-Carlo method
TLDR
It is shown that Bayesian training of backpropagation neural networks can feasibly be performed by the Hybrid Monte Carlo method, and the method has been applied to a test problem, demonstrating that it can produce good predictions, as well as an indication of the uncertainty of these predictions. Expand
Two papers on feed-forward networks
TLDR
The first paper works through Bayesian methods to complement back-propagation in the training of feed-forward networks, and the second addresses a problem raised by the first: how to efficiently calculate second derivatives on feed- forward networks. Expand
Bayesian Backprop in Action: Pruning, Committees, Error Bars and an Application to Spectroscopy
TLDR
The Bayesian framework for backpropagation is extended to pruned nets, leading to an Ockham Factor for "tuning the architecture to the data". Expand
Bayesian Recurrent Neural Networks
TLDR
This work shows that a simple adaptation of truncated backpropagation through time can yield good quality uncertainty estimates and superior regularisation at only a small extra computational cost during training, and demonstrates how a novel kind of posterior approximation yields further improvements to the performance of Bayesian RNNs. Expand
Bayesian Methods for Backpropagation Networks
TLDR
This chapter describes numerical techniques based on Gaussian approximations for implementation of powerful and practical methods for controlling, comparing, and using adaptive network models. Expand
On the Use of Evidence in Neural Networks
TLDR
It turns out that the evidence procedure's MAP estimate for neural nets is, in toto, approximation error, and the exact result neither has to be re-calculated for every new data set, nor requires the running of computer code. Expand
Bayesian Backpropagation Over I-O Functions Rather Than Weights
TLDR
The conventional Bayesian justification for backprop is that it finds the MAP weight vector, but to find the MAP i-o function instead, one must add a correction term to backprop, which biases one towards i-O functions with small description lengths. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 73 REFERENCES
Experiments on Learning by Back Propagation.
TLDR
The learning procedure can discover appropriate weights in their kind of network, as well as determine an optimal schedule for varying the nonlinearity of the units during a search. Expand
A statistical approach to learning and generalization in layered neural networks
TLDR
This paper presents a general statistical description of the problem of learning from examples, which is posed as an optimization problem: a search in the network parameter space for a network that minimizes an additive error function of the statistically independent examples. Expand
Learning curves in large neural networks
TLDR
Two model perceptrons, with weights that are constrained to be discrete, that exhibit sudden learning are discussed, and a general classification of generalization curves in models of realizable rules is proposed. Expand
Predicting the Future: a Connectionist Approach
TLDR
Since the ultimate goal is accuracy in the prediction, it is found that sigmoid networks trained with the weight-elimination algorithm outperform traditional nonlinear statistical approaches. Expand
A statistical approach to learning and generalization in layered neural networks
TLDR
The proposed formalism is applied to the problems of selecting an optimal architecture and the prediction of learning curves and the Gibbs distribution on the ensemble of networks with a fixed architecture is derived. Expand
What Size Net Gives Valid Generalization?
TLDR
It is shown that if m O(W/ ∊ log N/∊) random examples can be loaded on a feedforward network of linear threshold functions with N nodes and W weights, so that at least a fraction 1 ∊/2 of the examples are correctly classified, then one has confidence approaching certainty that the network will correctly classify a fraction 2 ∊ of future test examples drawn from the same distribution. Expand
Supervised Learning of Probability Distributions by Neural Networks
We propose that the back propagation algorithm for supervised learning can be generalized, put on a satisfactory conceptual footing, and very likely made more efficient by defining the values of theExpand
A structural learning algorithm with forgetting of link weights
  • M. Ishikawa
  • Computer Science
  • International 1989 Joint Conference on Neural Networks
  • 1989
TLDR
A novel learning algorithm is proposed, called structural learning algorithm, which generates a skeletal structure of a network: a network in which Minimum number of links and a minimum number of hidden units are actually used, which solves the first difficulty of trial and error. Expand
Refinement of approximate domain theories by knowledge-based neural networks
TLDR
The KBANN system relaxes this constraint through the use of empirical learning methods to refine approximately correct knowledge, used to determine the structure of an artificial neural network and the weights on its links, thereby making the knowledge accessible for modification by neural learning. Expand
A new error criterion for posterior probability estimation with neural nets
TLDR
An error criterion for training is introduced which improves the performance of neural nets as posterior probability estimators, as compared to using least squares, and is similar to the Kullback-Leibler information measure. Expand
...
1
2
3
4
5
...