• Corpus ID: 56350682

# Provable limitations of deep learning

@article{Abbe2018ProvableLO,
title={Provable limitations of deep learning},
author={Emmanuel Abbe and Colin Sandon},
journal={ArXiv},
year={2018},
volume={abs/1812.06369}
}
• Published 16 December 2018
• Computer Science
• ArXiv
As the success of deep learning reaches more grounds, one would like to also envision the potential limits of deep learning. This paper gives a first set of results proving that certain deep learning algorithms fail at learning certain efficiently learnable functions. The results put forward a notion of cross-predictability that characterizes when such failures take place. Parity functions provide an extreme example with a cross-predictability that decays exponentially, while a mere super…

## Figures from this paper

### Poly-time universality and limitations of deep learning

• Computer Science
ArXiv
• 2020
SGD is universal even with some poly-noise while full GD or SQ algorithms are not (e.g., parities); this also gives a separation between SGD-based deep learning and statistical query algorithms.

### When Hardness of Approximation Meets Hardness of Learning

• Computer Science
ArXiv
• 2020

### Failures of Gradient-Based Deep Learning

• Computer Science
ICML
• 2017
This work describes four types of simple problems, for which the gradient-based algorithms commonly used in deep learning either fail or suffer from significant difficulties.

### Extractor-based time-space lower bounds for learning

• Computer Science, Mathematics
Electron. Colloquium Comput. Complex.
• 2017
This work shows that for a large class of learning problems, any learning algorithm requires either a memory of size at least Ω(k · l ), or at least 2Ω(r) samples, or an exponential number of samples, achieving a tight Ω((log|X|) · (log|A|)) lower bound on the size of the memory.

### A Time-Space Lower Bound for a Large Class of Learning Problems

• R. Raz
• Mathematics, Computer Science
2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS)
• 2017
We prove a general time-space lower bound that applies for a large class of learning problems and shows that for every problem in that class, any learning algorithm requires either a memory of

### Efficient noise-tolerant learning from statistical queries

This paper formalizes a new but related model of learning from statistical queries, and demonstrates the generality of the statistical query model, showing that practically every class learnable in Valiant’s model and its variants can also be learned in the new model (and thus can be learning in the presence of noise).

### Weakly learning DNF and characterizing statistical query learning using Fourier analysis

• Computer Science
STOC '94
• 1994
It is proved that an algorithm due to Kushilevitz and Mansour can be used to weakly learn DNF using membership queries in polynomial time, with respect to the uniform distribution on the inputs, and it is obtained that DNF expressions and decision trees are not evenWeakly learnable with any unproven assumptions.