Skip to search formSkip to main contentSkip to account menu

The Singular Values of Convolutional Layers

It is shown that this is an effective regularizer; for example, it improves the test error of a deep residual network using batch normalization on CIFAR-10 from 6.2\% to 5.3\%.
View PDF on arXiv (opens in a new tab)

What is being transferred in transfer learning?

Through a series of analyses on transferring to block-shuffled images, the effect of feature reuse from learning low-level statistics of data is separated and it is shown that some benefit of transfer learning comes from the latter.
View PDF on arXiv (opens in a new tab)

Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods

This work proposes a computationally efficient method with guaranteed risk bounds for training neural networks with one hidden layer based on tensor decomposition, which provably converges to the global optimum, under a set of mild non-degeneracy conditions.

Score Function Features for Discriminative Learning: Matrix and Tensor Framework

This paper considers a novel class of matrix and tensor-valued features, which can be pre- trained using unlabeled samples, and presents efficient algorithms for extracting discriminative information, given these pre-trained features and labeled samples for any related task.
View PDF on arXiv (opens in a new tab)

Provable Tensor Methods for Learning Mixtures of Generalized Linear Models

This work considers the problem of learning mixtures of generalized linear models (GLM) which arise in classification and regression problems and presents a tensor decomposition method which is guaranteed to correctly recover the parameters.
View Paper (opens in a new tab)

Leveraging Unlabeled Data to Predict Out-of-Distribution Performance

Average Thresholded Confidence (ATC) is proposed, a practical method that learns a threshold on the model’s confidence, predicting accuracy as the fraction of unlabeled examples for which model confidence exceeds that threshold.
View PDF on arXiv (opens in a new tab)

Generalization bounds for deep convolutional neural networks

Borders on the generalization error of convolutional networks are proved in terms of the training loss, the number of parameters, the Lipschitz constant of the loss and the distance from the weights to the initial weights.
View PDF on arXiv (opens in a new tab)

The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks

If the permutation invariance of neural networks is taken into account, SGD solutions will likely have no barrier in the linear interpolation between them, which has implications for lottery ticket hypothesis, distributed training and ensemble methods.
View PDF on arXiv (opens in a new tab)

The Deep Bootstrap Framework: Good Online Learners are Good Offline Generalizers

View Paper (opens in a new tab)

Size-free generalization bounds for convolutional neural networks

Borders on the generalization error of convolutional networks are proved in terms of the training loss, the number of parameters, the Lipschitz constant of the loss and the distance from the weights to the initial weights.
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy (opens in a new tab), Terms of Service (opens in a new tab), and Dataset License (opens in a new tab)