Theory IIIb: Generalization in Deep Networks

  title={Theory IIIb: Generalization in Deep Networks},
  author={Tomaso A. Poggio and Qianli Liao and Brando Miranda and Andrzej Banburski and Xavier Boix and Jack Hidary},
A main puzzle of deep neural networks (DNNs) revolves around the apparent absence of "overfitting", defined in this paper as follows: the expected error does not get worse when increasing the number of neurons or of iterations of gradient descent. This is surprising because of the large capacity demonstrated by DNNs to fit randomly labeled data and the absence of explicit regularization. Recent results by Srebro et al. provide a satisfying solution of the puzzle for linear networks used in… CONTINUE READING
Twitter Mentions