Interpreting a Penalty as the Influence of a Bayesian Prior
@article{Wolinski2020InterpretingAP, title={Interpreting a Penalty as the Influence of a Bayesian Prior}, author={Pierre Wolinski and G. Charpiat and Y. Ollivier}, journal={ArXiv}, year={2020}, volume={abs/2002.00178} }
In machine learning, it is common to optimize the parameters of a probabilistic model, modulated by a somewhat ad hoc regularization term that penalizes some values of the parameters. Regularization terms appear naturally in Variational Inference (VI), a tractable way to approximate Bayesian posteriors: the loss to optimize contains a Kullback--Leibler divergence term between the approximate posterior and a Bayesian prior. We fully characterize which regularizers can arise this way, and provide… Expand
References
SHOWING 1-10 OF 30 REFERENCES
Probable networks and plausible predictions - a review of practical Bayesian methods for supervised neural networks
- Computer Science
- 1995
- 830
An Introduction to Variational Methods for Graphical Models
- Mathematics, Computer Science
- Machine Learning
- 2004
- 2,359
- PDF
A Practical Bayesian Framework for Backpropagation Networks
- Mathematics, Computer Science
- Neural Computation
- 1992
- 2,227
- Highly Influential
- PDF
Group sparse regularization for deep neural networks
- Mathematics, Computer Science
- Neurocomputing
- 2017
- 235
- PDF