Corpus ID: 195700200

'In-Between' Uncertainty in Bayesian Neural Networks

@article{Foong2019InBetweenUI,
  title={'In-Between' Uncertainty in Bayesian Neural Networks},
  author={Andrew Y. K. Foong and Yingzhen Li and Jos{\'e} Miguel Hern{\'a}ndez-Lobato and R. Turner},
  journal={ArXiv},
  year={2019},
  volume={abs/1906.11537}
}
  • Andrew Y. K. Foong, Yingzhen Li, +1 author R. Turner
  • Published 2019
  • Mathematics, Computer Science
  • ArXiv
  • We describe a limitation in the expressiveness of the predictive uncertainty estimate given by mean-field variational inference (MFVI), a popular approximate inference method for Bayesian neural networks. In particular, MFVI fails to give calibrated uncertainty estimates in between separated regions of observations. This can lead to catastrophically overconfident predictions when testing on out-of-distribution data. Avoiding such overconfidence is critical for active learning, Bayesian… CONTINUE READING
    Uncertainty in Neural Networks: Approximately Bayesian Ensembling
    7
    Improving predictions of Bayesian neural networks via local linearization
    1
    Bayesian Perceptron: Towards fully Bayesian Neural Networks
    Subspace Inference for Bayesian Deep Learning
    17
    Benchmarking the Neural Linear Model for Regression
    2
    Global inducing point variational posteriors for Bayesian neural networks and deep Gaussian processes
    1
    Estimating Model Uncertainty of Neural Networks in Sparse Information Form
    1

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 27 REFERENCES
    Ensemble learning in Bayesian neural networks
    126
    A Practical Bayesian Framework for Backpropagation Networks
    1958
    Overpruning in Variational Bayesian Neural Networks
    25
    Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
    418
    Weight Uncertainty in Neural Networks
    777
    Weight Uncertainty in Neural Network
    330
    Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling
    86
    Bayesian learning for neural networks
    3082
    Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
    2115