Corpus ID: 174802962

Practical Deep Learning with Bayesian Principles

@article{Osawa2019PracticalDL,
  title={Practical Deep Learning with Bayesian Principles},
  author={Kazuki Osawa and Siddharth Swaroop and Anirudh Jain and Runa Eschenhagen and Richard E. Turner and Rio Yokota and Mohammad Emtiyaz Khan},
  journal={ArXiv},
  year={2019},
  volume={abs/1906.02506}
}
  • Kazuki Osawa, Siddharth Swaroop, +4 authors Mohammad Emtiyaz Khan
  • Published 2019
  • Mathematics, Computer Science
  • ArXiv
  • Bayesian methods promise to fix many shortcomings of deep learning, but they are impractical and rarely match the performance of standard methods, let alone improve them. In this paper, we demonstrate practical training of deep networks with natural-gradient variational inference. By applying techniques such as batch normalisation, data augmentation, and distributed training, we achieve similar performance in about the same number of epochs as the Adam optimiser, even on large datasets such as… CONTINUE READING

    Citations

    Publications citing this paper.
    SHOWING 1-10 OF 32 CITATIONS

    Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors

    VIEW 1 EXCERPT
    CITES BACKGROUND

    Scalable and Practical Natural Gradient for Large-Scale Deep Learning

    VIEW 2 EXCERPTS
    CITES METHODS

    Continual Deep Learning by Functional Regularisation of Memorable Past

    VIEW 2 EXCERPTS
    CITES BACKGROUND

    FILTER CITATIONS BY YEAR

    2019
    2020

    CITATION STATISTICS

    • 2 Highly Influenced Citations

    • Averaged 16 Citations per year from 2019 through 2020

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 59 REFERENCES

    Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Models

    VIEW 1 EXCERPT