Corpus ID: 195069235

# Disentangling feature and lazy learning in deep neural networks: an empirical study

@article{Geiger2019DisentanglingFA,
title={Disentangling feature and lazy learning in deep neural networks: an empirical study},
author={M. Geiger and S. Spigler and Arthur Jacot and M. Wyart},
journal={ArXiv},
year={2019},
volume={abs/1906.08034}
}
Two distinct limits for deep learning as the net width $h\to\infty$ have been proposed, depending on how the weights of the last layer scale with $h$. In the "lazy-learning" regime, the dynamics becomes linear in the weights and is described by a Neural Tangent Kernel $\Theta$. By contrast, in the "feature-learning" regime, the dynamics can be expressed in terms of the density distribution of the weights. Understanding which regime describes accurately practical architectures and which one… Expand