Resource-Efficient Invariant Networks: Exponential Gains by Unrolled Optimization

  title={Resource-Efficient Invariant Networks: Exponential Gains by Unrolled Optimization},
  author={Sam Buchanan and Jingkai Yan and Ellie Haber and John N. Wright},
2022 Abstract Achieving invariance to nuisance transformations is a fundamental challenge in the construction of robust and reliable vision systems. Existing approaches to invariance scale exponentially with the dimension of the family of transformations, making them unable to cope with natural variabilities in visual data such as changes in pose and perspective. We identify a common limitation of these approaches—they rely on sampling to traverse the high-dimensional space of transformations… 

Figures from this paper


Manitest: Are classifiers really invariant?
The Manitest method is proposed, built on the efficient Fast Marching algorithm to compute the invariance of classifiers, which quantifies in particular the importance of data augmentation for learning invariance from data, and the increased invariances of convolutional neural networks with depth.
Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales
A formalism for analysing the covariance and invariance properties of scale-channel networks, including exploring their relations to scale-space theory, is developed and a new type of foveated scale- channel architecture is proposed, where the scale channels process increasingly larger parts of the image with decreasing resolution.
Geometric Robustness of Deep Networks: Analysis and Improvement
This work proposes ManiFool as a simple yet scalable algorithm to measure the invariance of deep networks and builds on it to propose a new adversarial training scheme and show its effectiveness on improving the invariances properties of deep neural networks.
Exploring the Landscape of Spatial Robustness
This work thoroughly investigate the vulnerability of neural network--based classifiers to rotations and translations and finds that, in contrast to the p-norm case, first-order methods cannot reliably find worst-case perturbations.
Scaling the Scattering Transform: Deep Hybrid Networks
We use the scattering network as a generic and fixed initialization of the first layers of a supervised hybrid deep network. We show that early layers do not necessarily need to be learned, providing
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
A 'geometric unification' endeavour that provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNN's, GNNs, and Transformers, and gives a constructive procedure to incorporate prior physical knowledge into neural architectures and provide principled way to build future architectures yet to be invented.
Spatial Transformer Networks
This work introduces a new learnable module, the Spatial Transformer, which explicitly allows the spatial manipulation of data within the network, and can be inserted into existing convolutional architectures, giving neural networks the ability to actively spatially transform feature maps.
Harmonic Networks: Deep Translation and Rotation Equivariance
H-Nets are presented, a CNN exhibiting equivariance to patch-wise translation and 360-rotation, and it is demonstrated that their layers are general enough to be used in conjunction with the latest architectures and techniques, such as deep supervision and batch normalization.
Why do deep convolutional networks generalize so poorly to small image transformations?
The results indicate that the problem of insuring invariance to small image transformations in neural networks while preserving high accuracy remains unsolved.