Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspective

@article{Wei2021DesigningTT,
  title={Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspective},
  author={Lanning Wei and Huan Zhao and Zhiqiang He},
  journal={Proceedings of the ACM Web Conference 2022},
  year={2021}
}
In recent years, Graph Neural Networks (GNNs) have shown superior performance on diverse real-world applications. To improve the model capacity, besides designing aggregation operations, GNN topology design is also very important. In general, there are two mainstream GNN topology design manners. The first one is to stack aggregation operations to obtain the higher-level features but easily got performance drop as the network goes deeper. Secondly, the multiple aggregation operations are… 
View on ACM
[PDF] Semantic Reader
13 Citations
Highly Influential Citations
2
Background Citations
5
Methods Citations
7
Results Citations
1

13 Citations

Neural Architecture Search for GNN-based Graph Classification

Experimental results in this paper demonstrate the effectiveness and efficiency of the proposed PAS in designing the pooling architectures for graph classification and the Top-1 performance on two Open Graph Benchmark datasets further indicates the utility of PAS when facing diverse realistic data.

Search to Capture Long-range Dependency with Stacking GNNs for Graph Classification

This paper evoke the importance of stacking-based GNNs and then employ them to capture the long-range dependencies without modifying the original graph structure, and proposes a novel approach with the help of neural architecture search (NAS), dubbed LRGNN (Long-Range Graph Neural Networks).

Enhancing Intra-class Information Extraction for Heterophilous Graphs: One Neural Architecture Search Approach

A unified framework is proposed based on the literature, in which the intra-class information from the node itself and neighbors can be extracted based on seven carefully designed blocks, and the help of neural architecture search is proposed to provide an architecture predictor to design GNNs for each node.

Graph Property Prediction on Open Graph Benchmark: A Winning Solution by Graph Neural Architecture Search

It is proved that the NAS method has high generalization ability for multiple tasks and the advantage of the method in processing graph property prediction tasks.

Gradient Gating for Deep Multi-Rate Learning on Graphs

A novel framework based on gating the output of GNN layers with a mechanism for multi-rate flow of message passing information across nodes of the underlying graph that alleviates the oversmoothing problem and allows the design of deep GNNs.

GBK-GNN: Gated Bi-Kernel Graph Neural Networks for Modeling Both Homophily and Heterophily

A novel GNN model based on a bi-kernel feature transformation and a selection gate is proposed, where two kernels capture homophily and heterophily information respectively, and the gate is introduced to select which kernel the authors should use for the given node pairs.

Bridging the Gap of AutoGraph Between Academia and Industry: Analyzing AutoGraph Challenge at KDD Cup 2020

The gaps between academia and industry on modeling scope, effectiveness, and efficiency are quantified, and it is shown that academic AutoML for Graph solutions focus on GNN architecture search while industrial solutions, especially the winning ones in the KDD Cup, tend to obtain an overall solution with only neural architecture search.

Bi-level Multi-objective Evolutionary Learning: A Case Study on Multi-task Graph Neural Topology Search

A bi-level multi-objective learning framework (BLMOL) that can outperform some state-of-the-art algorithms and generate well-representative UL solutions and LL weights is proposed, and a novel case study on multi-task graph neural topology search is considered.

Graph Neural Networks for Double-Strand DNA Breaks Prediction

A graph neural network based method to predict DSBs (GraphDSB), using DNA sequence features and chromosome structure information, and introduces Jumping Knowledge architecture and several effective structural encoding methods to improve the expression ability of the model.

Differentiable Meta Multigraph Search with Partial Message Propagation on Heterogeneous Information Networks

.

References

SHOWING 1-10 OF 75 REFERENCES

Auto-GNN: Neural architecture search of graph neural networks

An automated graph neural networks (AGNN) framework is proposed, which aims to find the optimal GNN architecture efficiently and achieves the best performance and search efficiency, comparing with existing human-invented models and the traditional search methods.

Design Space for Graph Neural Networks

This work defines and systematically study the architectural design space for GNNs which consists of 315,000 different designs over 32 different predictive tasks, and offers a principled and scalable approach to transition from studying individual GNN designs for specific tasks, to systematically studying the GNN design space and the task space.

Search to aggregate neighborhood for graph neural network

This work proposes a framework, which tries to Search to Aggregate NEighborhood (SANE), to automatically design data-specific GNN architectures, and proposes a differentiable search algorithm, which is more efficient than previous reinforcement learning based methods.

Enhance Information Propagation for Graph Neural Network by Heterogeneous Aggregations

It is argued that as richer information are propagated from shallow to deep layers, the discriminative capability of features formulated by GNN can benefit from it, and a new generic GNN layer formulation and upon this a new GNN variant referred as HAG-Net is proposed.

Simplifying Architecture Search for Graph Neural Network

The SNAG framework (Simplified Neural Architecture search for Graph neural networks), consisting of a novel search space and a reinforcement learning based search algorithm, is proposed, demonstrating the effectiveness of the SNAg framework compared to human-designed GNNs and NAS methods, including GraphNAS and Auto-GNN.

Graph Differentiable Architecture Search with Structure Learning

A Graph differentiable Architecture Search model with Structure Optimization (GASSO), which allows differentiable search of the architecture with gradient descent and is able to discover graph neural architectures with better performance through employing graph structure learning as a denoising process in the search procedure.

Geom-GCN: Geometric Graph Convolutional Networks

The proposed aggregation scheme is permutation-invariant and consists of three modules, node embedding, structural neighborhood, and bi-level aggregation, and an implementation of the scheme in graph convolutional networks, termed Geom-GCN, to perform transductive learning on graphs.

Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View

Two methods to alleviate the over-smoothing issue of GNNs are proposed: MADReg which adds a MADGap-based regularizer to the training objective; AdaEdge which optimizes the graph topology based on the model predictions.

Towards Deeper Graph Neural Networks

This work provides a systematical analysis and theoretical analysis of the over-smoothing issue and proposes Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields to learn graph node representations from larger receptive fields.

One-shot Graph Neural Architecture Search with Dynamic Search Space

This work proposes a novel dynamic one-shot search space for multi-branch neural architectures of GNNs and demonstrates that this method outperforms the current state-of-the-art manually designed architectures and reaches competitive performance to existing GNN NAS approaches with up to 10 times of speedup.
...