Equivariant maps from invariant functions

@article{BlumSmith2022EquivariantMF,
  title={Equivariant maps from invariant functions},
  author={Ben Blum-Smith and Soledad Villar},
  journal={ArXiv},
  year={2022},
  volume={abs/2209.14991}
}
In equivariant machine learning the idea is to restrict the learning to a hypothesis class where all the functions are equivariant with respect to some group action. Irreducible representations or invariant theory are typically used to parameterize the space of such functions. In this note, we explicate a general procedure, attributed to Malgrange, to express all polynomial maps between linear spaces that are equivariant with respect to the action of a group G , given a characterization of the… 

Dimensionless machine learning: Imposing exact units equivariance

The approach can be used to impose units equivariance across a broad range of machine learning methods which are equivariant to rotations and other groups and discusses the in-sample and out-of-sample prediction accuracy gains one can obtain in contexts like symbolic regression and emulation, where symmetry is important.

The passive symmetries of machine learning

It is argued that the implementation of passive symmetries is particularly valuable when the goal of the learning problem is to generalize out of sample, and can have a impact on helping machine learning make the transition that took place for modern physics in the first half of the Twentieth century.

References

SHOWING 1-10 OF 56 REFERENCES

Provably Strict Generalisation Benefit for Equivariant Models

This paper provides the first provably non-zero improvement in generalisation for invariant/equivariant models when the target distribution is invariant /Equivariant with respect to a compact group.

Dimensionless machine learning: Imposing exact units equivariance

The approach can be used to impose units equivariance across a broad range of machine learning methods which are equivariant to rotations and other groups and discusses the in-sample and out-of-sample prediction accuracy gains one can obtain in contexts like symbolic regression and emulation, where symmetry is important.

Coordinate Independent Convolutional Networks - Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds

The generality of the differential geometric formulation of convolutional networks is demonstrated by an extensive literature review which explains a large number of Euclidean CNNs, spherical CNNs and CNNs on general surfaces as specific instances of coordinate independent convolutions.

E(n) Equivariant Graph Neural Networks

This paper introduces a new model to learn graph neural networks equivariant to rotations, translations, reflections and permutations called E( n ) Equivariant Graph Neural Networks (EGNNs), which does not require computationally expensive higher-order representations in intermediate layers while it still achieves competitive or better performance.

A simple equivariant machine learning method for dynamics based on scalars

This work implements a principled model based on invariant scalars, shows that the Scalars method outperforms state-of-the-art approaches for learning the properties of physical systems with symmetries, both in terms of accuracy and speed, and applies it to a simple chaotic dynamical system.

Invariant and Equivariant Graph Networks

This paper provides a characterization of all permutation invariant and equivariant linear layers for (hyper-)graph data, and shows that their dimension, in case of edge-value graph data, is 2 and 15, respectively.

Data Augmentation vs. Equivariant Networks: A Theory of Generalization on Dynamics Forecasting

This work derives the generalization bounds for data augmentation and equivariant networks, characterizing their effect on learning in a unified framework and focuses on non-stationary dynamics forecasting with complex temporal dependen-cies.

Learning with invariances in random features and kernel models

This work characterize the test error of invariant methods in a high-dimensional regime in which the sample size and number of hidden units scale as polynomials in the dimension, and shows that exploiting invariance in the architecture saves a d factor to achieve the same test error as for unstructured architectures.

Invariant polynomials and machine learning

This work obtains two types of generators of the Lorentzand permutationinvariant polynomials in particle momenta; minimal algebra generators and Hironaka decompositions and discusses and proves some approximation theorems to make use of these invariant generators in machine learning algorithms in general and in neural networks specifically.

On the Sample Complexity of Learning with Geometric Stability

This work provides non-parametric rates of convergence for kernel methods, and shows improvements in sample complexity by a factor equal to the size of the group when using an invariant kernel over the group, compared to the corresponding non-invariant kernel.
...