#### Filter Results:

- Full text PDF available (32)

#### Publication Year

2009

2019

- This year (4)
- Last 5 years (51)
- Last 10 years (59)

#### Supplemental Content

#### Publication Type

#### Co-author

#### Journals and Conferences

Learn More

Supervised learning on molecules has incredible potential to be useful in chemistry, drug discovery, and materials science. Luckily, several promising and closely related neural network models… (More)

It is well known that weight initialization in deep networks can have a dramatic impact on learning speed. For example, ensuring the mean squared singular value of a network's input-output Jacobian… (More)

We study the behavior of untrained neural networks whose weights and biases are randomly distributed using mean field theory. We show the existence of depth scales that naturally limit the maximum… (More)

A deep fully-connected neural network with an i.i.d. prior over its parameters is equivalent to a Gaussian process (GP) in the limit of infinite network width. This correspondence enables exact… (More)

In recent years, state-of-the-art methods in computer vision have utilized increasingly deep convolutional neural network architectures (CNNs), with some of the most successful models employing… (More)

Recurrent neural networks have gained widespread use in modeling sequence data across various domains. While many successful recurrent architectures employ a notion of gating, the exact mechanism… (More)

- Greg Yang, Samuel S. Schoenholz
- NIPS
- 2017

We study randomly initialized residual networks using mean field theory and the theory of difference equations. Classical feedforward neural networks, such as those with tanh activations, exhibit… (More)

- Justin Gilmer, Luke Metz, +4 authors Ian J. Goodfellow
- ICLR
- 2018

State of the art computer vision models have been shown to be vulnerable to small adversarial perturbations of the input. In other words, most images in the data distribution are both correctly… (More)

A longstanding goal in deep learning research has been to precisely characterize training and generalization. However, the often complex loss landscapes of neural networks have made a theory of… (More)

- Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli
- AISTATS
- 2018

Recent work has shown that tight concentration of the entire spectrum of singular values of a deep network's input-output Jacobian around one at initialization can speed up learning by orders of… (More)