In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning

Abstract

We present experiments demonstrating that some other form of capacity control, different from network size, plays a central role in learning multi-layer feedforward networks. We argue, partially through analogy to matrix factorization, that this is an inductive bias that can help shed light on deep learning. 

2 Figures and Tables

Topics

Statistics

0204060201520162017
Citations per Year

Citation Velocity: 18

Averaging 18 citations per year over the last 3 years.

Learn more about how we calculate this metric in our FAQ.
  • Blog articles referencing this paper

  • Presentations referencing similar topics