• Publications
  • Influence
On the Number of Linear Regions of Deep Neural Networks
We study the complexity of functions computable by deep feedforward neural networks with piecewise linear activations in terms of the symmetries and the number of linear regions that they have. DeepExpand
On the number of response regions of deep feed forward networks with piece-wise linear activations
TLDR
This paper offers a framework for comparing deep and shallow models that belong to the family of piecewise linear functions based on computational geometry, and looks at a deep rectifier multi-layer perceptron with linear outputs units and compares it with a single layer version of the model. Expand
On the number of inference regions of deep feed forward networks with piece-wise linear activations
TLDR
This paper offers a framework for comparing deep and shallow models that belong to the family of piecewise linear functions based on computational geometry and looks at a deep rectifier multi-layer perceptron (MLP) with linear outputs units and compares it with a single layer version of the model. Expand
Refinements of Universal Approximation Results for Deep Belief Networks and Restricted Boltzmann Machines
TLDR
It is shown that any distribution on the set of binary vectors of length can be arbitrarily well approximated by an RBM with hidden units, and this confirms a conjecture presented in Le Roux and Bengio (2010). Expand
Haar Graph Pooling
TLDR
A new graph pooling operation based on compressive Haar transforms -- HaarPooling is proposed, which synthesizes the features of any given input graph into a feature vector of uniform size. Expand
Restricted Boltzmann Machines: Introduction and Review
TLDR
An introduction to the mathematical analysis of restricted Boltzmann machines is given, recent results on the geometry of the sets of probability distributions representable by these models are reviewed, and a few directions for further investigation are suggested. Expand
Geometry and expressive power of conditional restricted Boltzmann machines
TLDR
This work addresses the representational power of conditional restricted Boltzmann machines, proving their ability to represent conditional Markov random fields and conditional distributions with restricted supports, the minimal size of universal approximators, the maximal model approximation errors, and on the dimension of the set of representable conditional distributions. Expand
Kernelized Wasserstein Natural Gradient
TLDR
This work proposes a general framework to approximate the natural gradient for the Wasserstein metric, by leveraging a dual formulation of the metric restricted to a Reproducing Kernel Hilbert Space, and leads to an estimator for gradient direction that can trade-off accuracy and computational cost, with theoretical guarantees. Expand
How Well Do WGANs Estimate the Wasserstein Metric?
TLDR
This work studies how well the methods, that are used in generative adversarial networks to approximate the Wasserstein metric, perform, and considers, in particular, the $c-transform formulation, which eliminates the need to enforce the constraints explicitly. Expand
Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks
TLDR
Tight bounds on the smallest eigenvalue of NTK matrices for deep ReLU nets are provided, both in the limiting case of infinite widths and for finite widths. Expand
...
1
2
3
4
5
...