Corpus ID: 88516005

A Statistical Theory of Deep Learning via Proximal Splitting

@article{Polson2015AST,
  title={A Statistical Theory of Deep Learning via Proximal Splitting},
  author={Nicholas G. Polson and Brandon T. Willard and Massoud Heidari},
  journal={arXiv: Machine Learning},
  year={2015}
}
  • Nicholas G. Polson, Brandon T. Willard, Massoud Heidari
  • Published 2015
  • Mathematics
  • arXiv: Machine Learning
  • In this paper we develop a statistical theory and an implementation of deep learning models. We show that an elegant variable splitting scheme for the alternating direction method of multipliers optimises a deep learning objective. We allow for non-smooth non-convex regularisation penalties to induce sparsity in parameter weights. We provide a link between traditional shallow layer statistical models such as principal component and sliced inverse regression and deep layer models. We also define… CONTINUE READING

    Figures and Tables from this paper.

    Deep Learning: A Bayesian Perspective
    50
    Deep learning for epileptic intracranial EEG data
    55
    Arabic Handwritten Alphanumeric Character Recognition Using Very Deep Neural Network
    10

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 54 REFERENCES
    Proximal Algorithms in Statistics and Machine Learning
    59
    A Fast Learning Algorithm for Deep Belief Nets
    10801
    Breaking the Curse of Dimensionality with Convex Neural Networks
    248
    On optimization methods for deep learning
    733
    Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers
    10975
    Data augmentation for support vector machines
    158