• Corpus ID: 236912752

PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equations

@inproceedings{Eliasof2021PDEGCNNA,
title={PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equations},
author={Moshe Eliasof and Eldad Haber and Eran Treister},
booktitle={NeurIPS},
year={2021}
}
• Published in NeurIPS 4 August 2021
• Computer Science
Graph neural networks are increasingly becoming the go-to approach in various fields such as computer vision, computational biology and chemistry, where data are naturally explained by graphs. However, unlike traditional convolutional neural networks, deep graph networks do not necessarily yield better performance than shallow graph networks. This behavior usually stems from the over-smoothing phenomenon. In this work, we propose a family of architectures to control this behavior by design. Our…

Figures and Tables from this paper

Quantized convolutional neural networks through the lens of partial differential equations

• Computer Science
Research in the Mathematical Sciences
• 2022
It is demonstrated through several experiments that the property of forward sta- bility preserves the action of a network under diﬀerent quantization rates, and that at times, stability even aids in improving accuracy.

Graph Kernel Neural Networks

• Computer Science
ArXiv
• 2021
This paper proposes to use graph kernels, i.e., kernel functions that compute an inner product on graphs, to extend the standard convolution operator to the graph domain, and proposes an entirely structural model that does not require computing the embedding of the input graph.

Graph-Coupled Oscillator Networks

• Computer Science
ICML
• 2022
It is proved that GraphCON mitigates the exploding and vanishing gradients problem to facilitate training of deep multi-layer GNNs and offers competitive performance with respect to the state-of-the-art on a variety of graph-based learning tasks.

pathGCN: Learning General Graph Spatial Operators from Paths

• Computer Science
ICML
• 2022
By properly learning both the spatial and point-wise convolutions, phenomena like over-smoothing can be inherently avoided, and new state-of-the-art performance is achieved.

Graph Neural Networks as Gradient Flows

• Computer Science
ArXiv
• 2022
This approach allows to analyse the GNN evolution from a multi-particle perspective as learning attractive and repulsive forces in feature space via the positive and negative eigenvalues of a symmetric ‘channel-mixing’ matrix.

Predicting parametric spatiotemporal dynamics by multi-resolution PDE structure-preserved deep learning

• Computer Science
ArXiv
• 2022
A novel PDE-preserved neural network (PPNN) for rapidly predicting parametric spatiotemporal dynamics, given the governing PDEs are (partially) known, and this physics-inspired learning architecture design endows PPNN with excellent generalizability and long-term prediction accuracy compared to the state-of-the-art blackbox ConvResNet baseline.

Optimization-Induced Graph Implicit Nonlinear Diffusion

• Computer Science
ICML
• 2022
It is shown that the learned representation can be formalized as the minimizer of an explicit convex optimization objective, and can embed prior properties to the equilibrium, as well as introducing skip connections to promote training stability.

Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited

• Computer Science
ArXiv
• 2022
CebNetII is proposed, a new GNN model based on Chebyshev interpolation, which enhances the original Chebyshv polynomial approximation while reducing the Runge phenomenon, and is scaled to a billion graph papers100M, showing that spectral-based GNNs have superior performance.

ACMP: Allen-Cahn Message Passing for Graph Neural Networks with Particle Phase Transition

• Computer Science
ArXiv
• 2022
Experiments for various real node classiﬁcation datasets show the GNNs with ACMP can achieve state of the art performance with no decay of Dirichlet energy, and provides a deep model ofGNNs which circumvents the common GNN problem of oversmoothing.

GRAND++: G RAPH N EURAL D IFFUSION WITH A S OURCE T ERM

• Computer Science
• 2022
The proposed GRAph Neural Diffusion with a source term (GRAND++) can provide accurate classification even when the model is trained with a very limited number of labeled training data, showing a significant improvement over many existing graph neural networks.

References

SHOWING 1-10 OF 63 REFERENCES

Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations

• Computer Science
ICML
• 2018
It is shown that many effective networks, such as ResNet, PolyNet, FractalNet and RevNet, can be interpreted as different numerical discretizations of differential equations and established a connection between stochastic control and noise injection in the training process which helps to improve generalization of the networks.

GRAND: Graph Neural Diffusion

• Computer Science
ICML
• 2021
We present Graph Neural Diffusion (GRAND) that approaches deep learning on graphs as a continuous diffusion process and treats Graph Neural Networks (GNNs) as discretisations of an underlying PDE. In

Simple and Deep Graph Convolutional Networks

• Computer Science
ICML
• 2020
The GCNII is proposed, an extension of the vanilla GCN model with two simple yet effective techniques: {\em Initial residual} and {\em Identity mapping} that effectively relieves the problem of over-smoothing.

Deep Neural Networks Motivated by Partial Differential Equations

• Computer Science
Journal of Mathematical Imaging and Vision
• 2019
A new PDE interpretation of a class of deep convolutional neural networks (CNN) that are commonly used to learn from speech, image, and video data is established and three new ResNet architectures are derived that fall into two new classes: parabolic and hyperbolic CNNs.

Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs

• Computer Science
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
• 2017
This work generalizes the convolution operator from regular grids to arbitrary graphs while avoiding the spectral domain, which allows us to handle graphs of varying size and connectivity.

DiffGCN: Graph Convolutional Networks via Differential Operators and Algebraic Multigrid Pooling

• Computer Science
NeurIPS
• 2020
This work proposed novel approaches for graph convolution, pooling and unpooling, taking inspiration from finite-elements and algebraic multigrid frameworks, form a parameterized convolution kernel based on discretized differential operators, leveraging the graph mass, gradient and Laplacian.

Stable Architectures for Deep Neural Networks

• Computer Science
ArXiv
• 2017
This paper relates the exploding and vanishing gradient phenomenon to the stability of the discrete ODE and presents several strategies for stabilizing deep learning for very deep networks.

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

• Computer Science
NIPS
• 2016
This work presents a formulation of CNNs in the context of spectral graph theory, which provides the necessary mathematical background and efficient numerical schemes to design fast localized convolutional filters on graphs.

Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs

• Computer Science
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
• 2017
This paper proposes a unified framework allowing to generalize CNN architectures to non-Euclidean domains (graphs and manifolds) and learn local, stationary, and compositional task-specific features and test the proposed method on standard tasks from the realms of image-, graph-and 3D shape analysis and show that it consistently outperforms previous approaches.