"Zero-Shot" Super-Resolution Using Deep Internal Learning
- Assaf Shocher, Nadav Cohen, M. Irani
- Computer ScienceComputer Vision and Pattern Recognition
- 17 December 2017
This paper exploits the internal recurrence of information inside a single image, and train a small image-specific CNN at test time, on examples extracted solely from the input image itself, which is the first unsupervised CNN-based SR method.
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization
- Sanjeev Arora, Nadav Cohen, Elad Hazan
- Computer ScienceInternational Conference on Machine Learning
- 19 February 2018
This paper suggests that, sometimes, increasing depth can speed up optimization and proves that it is mathematically impossible to obtain the acceleration effect of overparametrization via gradients of any regularizer.
Implicit Regularization in Deep Matrix Factorization
- Sanjeev Arora, Nadav Cohen, Wei Hu, Yuping Luo
- Computer ScienceNeural Information Processing Systems
- 31 May 2019
This work studies the implicit regularization of gradient descent over deep linear neural networks for matrix completion and sensing, a model referred to as deep matrix factorization, and finds that adding depth to a matrix factorizations enhances an implicit tendency towards low-rank solutions.
A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks
- Sanjeev Arora, Nadav Cohen, Noah Golowich, Wei Hu
- Computer ScienceInternational Conference on Learning…
- 27 September 2018
The speed of convergence to global optimum for gradient descent training a deep linear neural network is analyzed by minimizing the $\ell_2$ loss over whitened data by maximizing the initial loss of any rank-deficient solution.
On the Expressive Power of Deep Learning: A Tensor Analysis
- Nadav Cohen, Or Sharir, A. Shashua
- Computer ScienceAnnual Conference Computational Learning Theory
- 16 September 2015
It is proved that besides a negligible set, all functions that can be implemented by a deep network of polynomial size, require exponential size in order to be realized (or even approximated) by a shallow network.
Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Design
- Yoav Levine, David Yakira, Nadav Cohen, A. Shashua
- Computer ScienceInternational Conference on Learning…
- 5 April 2017
This work establishes a fundamental connection between the fields of quantum physics and deep learning, and shows an equivalence between the function realized by a deep convolutional arithmetic circuit (ConvAC) and a quantum many-body wave function, which relies on their common underlying tensorial structure.
Convolutional Rectifier Networks as Generalized Tensor Decompositions
- Nadav Cohen, A. Shashua
- Computer ScienceInternational Conference on Machine Learning
- 1 March 2016
Developing effective methods for training convolutional arithmetic circuits may give rise to a deep learning architecture that is provably superior to Convolutional rectifier networks, which has so far been overlooked by practitioners.
Tensorial Mixture Models
- Or Sharir, Ronen Tamari, Nadav Cohen, A. Shashua
- Computer ScienceArXiv
- 24 April 2017
The effectiveness of the Tensorial Mixture Models model when tackling the problem of classification with missing data is demonstrated, leveraging TMMs unique ability of tractable marginalization which leads to optimal classifiers regardless of the missingness distribution.
Continuous vs. Discrete Optimization of Deep Neural Networks
- Omer Elkabetz, Nadav Cohen
- Computer ScienceNeural Information Processing Systems
- 14 July 2021
It is hypothesized that the theory of gradient flows will unravel mysteries behind deep learning, and it is shown that over deep neural networks with homogeneous activations, gradient flow trajectories enjoy favorable curvature, suggesting they are well approximated by gradient descent.
Implicit Regularization in Deep Learning May Not Be Explainable by Norms
- Noam Razin, Nadav Cohen
- Computer ScienceNeural Information Processing Systems
- 13 May 2020
The results suggest that, rather than perceiving the implicit regularization via norms, a potentially more useful interpretation is minimization of rank, and it is demonstrated empirically that this interpretation extends to a certain class of non-linear neural networks, and hypothesize that it may be key to explaining generalization in deep learning.
...
...