• Corpus ID: 247450503

Simple GNN Regularisation for 3D Molecular Property Prediction and Beyond

@inproceedings{Godwin2021SimpleGR,
  title={Simple GNN Regularisation for 3D Molecular Property Prediction and Beyond},
  author={Jonathan Godwin and Michael Schaarschmidt and Alex Gaunt and Alvaro Sanchez-Gonzalez and Yulia Rubanova and Petar Velivckovi'c and James Kirkpatrick and Peter W. Battaglia},
  booktitle={International Conference on Learning Representations},
  year={2021}
}
In this paper we show that simple noisy regularisation can be an effective way to address GNN oversmoothing. First we argue that regularisers addressing oversmoothing should both penalise node latent similarity and encourage meaningful node representations. From this observation we derive “Noisy Nodes”, a simple technique in which we corrupt the input graph with noise, and add a noise correcting node-level loss. The diverse node level loss encourages latent node diversity, and the denoising… 
[PDF] Semantic Reader
35 Citations
Highly Influential Citations
4
Background Citations
18
Methods Citations
15
Results Citations
1

35 Citations

Pre-training via Denoising for Molecular Property Prediction

A pre-training technique based on denoising that achieves a new state-of-the-art in molecular property prediction by utilizing large datasets of 3D molecular structures at equilibrium to learn meaningful representations for downstream tasks.

Robust Optimization as Data Augmentation for Large-scale Graphs

  • Kezhi KongG. Li T. Goldstein
  • Computer Science
    2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2022
This work proposes FLAG (Free Large-scale Adversarial Augmentation on Graphs), which iteratively augments node features with gradient-based adversarial perturbations during training, which universally works in node classification, link prediction, and graph classification tasks.

ComENet: Towards Complete and Efficient Message Passing for 3D Molecular Graphs

This work proposes a novel message passing scheme that operates within 1-hop neighborhood that guarantees full completeness of 3D information on 3D graphs by achieving global and local completeness, and provides rigorous proof of completeness and analysis of time complexity for the methods.

Spherical Message Passing for 3D Molecular Graphs

This work proposes the spherical message passing (SMP) as a novel and powerful scheme for 3D molecular learning that dramatically reduces training complexity, enabling it to perform efficiently on large-scale molecules and demonstrates the advantages of SphereNet in terms of capability, efficiency, and scalability.

Benchmarking Graphormer on Large-Scale Molecular Modeling Datasets

The recent updates of Graphormer, including architecture design modifications, and the adaption to 3D molecules are described, which show that with a global receptive and an adaptive aggregation strategy, graphormer is more powerful than classic message-passing-based GNNs.

Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs

To better adapt Transformers to 3D graphs, a novel equivariant graph attention is proposed, which considers both content and geometric information such as relative position contained in irreps features.

Polymer informatics at-scale with multitask graph neural networks

This work demonstrates that directly “machine-learning” important features from a polymer repeat unit is a cheap and viable alternative to extracting expensive features by hand, and speeds up feature extraction by one to two orders of magnitude relative to presently adopted handcrafted methods.

Affinity-Aware Graph Networks

This paper explores the use of affinity measures as features in graph neural networks, in particular measures arising from random walks, including effective resistance, hitting and commute times, and proposes message passing networks based on these features.

GPS++: An Optimised Hybrid MPNN/Transformer for Molecular Property Prediction

This technical report presents GPS++, the first-place solution to the Open Graph Benchmark Large-Scale Challenge (OGB-LSC 2022) for the PCQM4Mv2 molecular property prediction task, a hybrid MPNN/Transformer model that incorporates 3D atom positions and an auxiliary denoising task.

Spherical Message Passing for 3D Graph Networks

This work proposes the spherical message passing (SMP) as a novel and specific scheme for realizing the 3DGN framework in the spherical coordinate system (SCS), and derives physically-based representations of geometric information and proposes the SphereNet for learning representations of 3D graphs.

References

SHOWING 1-10 OF 73 REFERENCES

PairNorm: Tackling Oversmoothing in GNNs

PairNorm is a novel normalization layer that is based on a careful analysis of the graph convolution operator, which prevents all node embeddings from becoming too similar and significantly boosts performance for a new problem setting that benefits from deeper GNNs.

Revisiting "Over-smoothing" in Deep GCNs

This work interprets a standard GCN architecture as layerwise integration of a Multi-layer Perceptron (MLP) and graph regularization and concludes that before training, the final representation of a deep GCN does over-smooth, however, it learns anti-oversmoothing during training.

Effective Training Strategies for Deep Graph Neural Networks

The proposed NodeNorm regularizes deep GCNs by discouraging feature-wise correlation of hidden embeddings and increasing model smoothness with respect to input node features, and thus effectively reduces overfitting, enabling deep GNNs to compete with and even outperform shallow ones.

Relational Pooling for Graph Representations

This work generalizes graph neural networks (GNNs) beyond those based on the Weisfeiler-Lehman (WL) algorithm, graph Laplacians, and diffusions to provide a framework with maximal representation power for graphs.

DeeperGCN: All You Need to Train Deeper GCNs

Extensive experiments on Open Graph Benchmark show DeeperGCN significantly boosts performance over the state-of-the-art on the large scale graph learning tasks of node property prediction and graph property prediction.

A Note on Over-Smoothing for Graph Neural Networks

It is shown that when the weight matrix satisfies the conditions determined by the spectrum of augmented normalized Laplacian, the Dirichlet energy of embeddings will converge to zero, resulting in the loss of discriminative power.

Neural Message Passing for Quantum Chemistry

Using MPNNs, state of the art results on an important molecular property prediction benchmark are demonstrated and it is believed future work should focus on datasets with larger molecules or more accurate ground truth labels.

Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning

It is shown that the graph convolution of the GCN model is actually a special form of Laplacian smoothing, which is the key reason why GCNs work, but it also brings potential concerns of over-smoothing with many convolutional layers.

FLAG: Adversarial Data Augmentation for Graph Neural Networks

This work proposes a simple but effective solution, FLAG (Free Large-scale Adversarial Augmentation on Graphs), which iteratively augments node features with gradient-based adversarial perturbations during training, and boosts performance at test time.

Graph Contrastive Learning with Augmentations

The results show that, even without tuning augmentation extents nor using sophisticated GNN architectures, the GraphCL framework can produce graph representations of similar or better generalizability, transferrability, and robustness compared to state-of-the-art methods.
...