Corpus ID: 225103395

Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth

@article{Nguyen2020DoWA,
  title={Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth},
  author={Thao Nguyen and M. Raghu and Simon Kornblith},
  journal={ArXiv},
  year={2020},
  volume={abs/2010.15327}
}

Thao Nguyen, M. Raghu, Simon Kornblith

Published 2020

Computer Science

ArXiv

A key factor in the success of deep neural networks is the ability to scale models to improve performance by varying the architecture depth and width. This simple property of neural network design has resulted in highly effective architectures for a variety of tasks. Nevertheless, there is limited understanding of effects of depth and width on the learned representations. In this paper, we study this fundamental question. We begin by investigating how varying depth and width affects model… CONTINUE READING

View PDF on arXiv

Save to Library

Create Alert

Cite

Launch Research Feed