#### Filter Results:

- Full text PDF available (54)

#### Publication Year

1996

2018

- This year (12)
- Last 5 years (41)
- Last 10 years (50)

#### Publication Type

#### Co-author

#### Journals and Conferences

Learn More

A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent or quasi-Newton methods areâ€¦ (More)

- Andrew M. Saxe, James L. McClelland, Surya Ganguli
- ArXiv
- 2013

Despite the widespread practical success of deep learning methods, our theoretical understanding of the dynamics of learning in deep neural networks remains quite sparse. We attempt to bridge the gapâ€¦ (More)

- Surya Ganguli, Dongsung Huh, Haim Sompolinsky
- Proceedings of the National Academy of Sciencesâ€¦
- 2008

To perform nontrivial, real-time computations on a sensory input stream, biological systems must retain a short-term memory trace of their recent inputs. It has been proposed that genericâ€¦ (More)

- Chris Piech, Jonathan C. Spencer, +4 authors Jascha Sohl-Dickstein
- NIPS
- 2015

Knowledge tracingâ€”where a machine models the knowledge of a student as they interact with courseworkâ€”is a well established problem in computer supported education. Though effectively modeling studentâ€¦ (More)

We combine Riemannian geometry with the mean field theory of high dimensional chaos to study the nature of signal propagation in generic, deep neural networks with random weights. Our results revealâ€¦ (More)

We propose a new approach to the problem of neural network expressivity, which seeks to characterize how structural properties of a neural network family affect the functions it is able to compute.â€¦ (More)

- Friedemann Zenke, Ben Poole, Surya Ganguli
- ICML
- 2017

While deep learning has led to remarkable advances across diverse applications, it struggles in domains where the data distribution changes over the course of learning. In stark contrast, biologicalâ€¦ (More)

We study the behavior of untrained neural networks whose weights and biases are randomly distributed using mean field theory. We show the existence of depth scales that naturally limit the maximumâ€¦ (More)

It is well known that weight initialization in deep networks can have a dramatic impact on learning speed. For example, ensuring the mean squared singular value of a networkâ€™s input-output Jacobianâ€¦ (More)

- Edward K. Boyda, Surya Ganguli, Petr HÇ’rava, Uday Varadarajan
- 2003

We analyze the structure of supersymmetric GÃ¶del-like cosmological solutions of string theory. Just as the original four-dimensional GÃ¶del universe, these solutions represent rotating, topologicallyâ€¦ (More)