# Graph Attention Networks

@article{Velickovic2018GraphAN, title={Graph Attention Networks}, author={Petar Velickovic and Guillem Cucurull and Arantxa Casanova and Adriana Romero and Pietro Lio’ and Yoshua Bengio}, journal={ArXiv}, year={2018}, volume={abs/1710.10903} }

We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations. [...] Key Result Our GAT models have achieved or matched state-of-the-art results across four established transductive and inductive graph benchmarks: the Cora, Citeseer and Pubmed citation network datasets, as well as a protein-protein interaction dataset… Expand

#### 3,895 Citations

Graph Representation Learning Network via Adaptive Sampling

- Computer Science, Mathematics
- ArXiv
- 2020

A new architecture to address issues of scalability and efficiency with GAT and GraphSAGE is proposed that is more efficient and is capable of incorporating different edge type information. Expand

DAGCN: Dual Attention Graph Convolutional Networks

- Computer Science, Mathematics
- 2019 International Joint Conference on Neural Networks (IJCNN)
- 2019

DAGCN automatically learns the importance of neighbors at different hops using a novel attention graph convolution layer, and then employs a second attention component, a self-attention pooling layer, to generalize the graph representation from the various aspects of a matrix graph embedding. Expand

Improving Graph Attention Networks with Large Margin-based Constraints

- Computer Science, Mathematics
- ArXiv
- 2019

This work first theoretically demonstrate the over-smoothing behavior of GATs and then develops an approach using constraint on the attention weights according to the class boundary and feature aggregation pattern, which leads to significant improvements over the previous state-of-the-art graph attention methods on all datasets. Expand

Graph Attention Networks with Positional Embeddings

- Computer Science
- PAKDD
- 2021

This work proposes a framework, termed Graph Attentional Networks with Positional Embeddings (GAT-POS), to enhance GATs with positional embeddings which capture structural and positional information of the nodes in the graph. Expand

Understanding Attention and Generalization in Graph Neural Networks

- Computer Science, Mathematics
- NeurIPS
- 2019

This work proposes an alternative recipe and train attention in a weakly-supervised fashion that approaches the performance of supervised models, and, compared to unsupervised models, improves results on several synthetic as well as real datasets. Expand

Graphs, Entities, and Step Mixture

- Computer Science, Mathematics
- ArXiv
- 2020

A new graph neural network that considers both edge-based neighborhood relationships and node-based entity features, i.e. Graph Entities with Step Mixture via random walk (GESM), which achieves state-of-the-art or comparable performances on eight benchmark graph datasets comprising transductive and inductive learning tasks. Expand

GraphMix: Regularized Training of Graph Neural Networks for Semi-Supervised Learning

- Computer Science, Mathematics
- ArXiv
- 2019

This work proposes a unified approach in which a fully-connected network is trained jointly with the graph neural network via parameter sharing, interpolation-based regularization, and self-predicted-targets. Expand

Node Masking: Making Graph Neural Networks Generalize and Scale Better

- Computer Science, Mathematics
- ArXiv
- 2020

This paper discusses some theoretical tools to better visualize the operations performed by state of the art spatial GNNs and introduces a simple concept, node masking, that allows them to generalize and scale better. Expand

Graph-CAT: Graph Co-Attention Networks via local and global attribute augmentations

- Computer Science
- Future Gener. Comput. Syst.
- 2021

This paper proposes a novel Graph Co-ATtention Network (Graph-CAT), which performs both the local and global attribute augmentations based on two different yet complementary attention schemes and demonstrates the superiority of the Graph-C AT compared to the state-of-the-art methods. Expand

Learnt Sparsification for Interpretable Graph Neural Networks

- Computer Science
- ArXiv
- 2021

This paper proposes a novel method called KEdge for explicitly sparsification using the Hard Kumaraswamy distribution that can be used in conjugation with any GNN model and effectively counters the over-smoothing phenomena in deep GNNs by maintaining good task performance with increasing GNN layers. Expand

#### References

SHOWING 1-10 OF 47 REFERENCES

Gated Graph Sequence Neural Networks

- Computer Science, Mathematics
- ICLR
- 2016

This work studies feature learning techniques for graph-structured inputs and achieves state-of-the-art performance on a problem from program verification, in which subgraphs need to be matched to abstract data structures. Expand

Deep Convolutional Networks on Graph-Structured Data

- Computer Science
- ArXiv
- 2015

This paper develops an extension of Spectral Networks which incorporates a Graph Estimation procedure, that is test on large-scale classification problems, matching or improving over Dropout Networks with far less parameters to estimate. Expand

Inductive Representation Learning on Large Graphs

- Computer Science, Mathematics
- NIPS
- 2017

GraphSAGE is presented, a general, inductive framework that leverages node feature information (e.g., text attributes) to efficiently generate node embeddings for previously unseen data and outperforms strong baselines on three inductive node-classification benchmarks. Expand

Semi-Supervised Classification with Graph Convolutional Networks

- Computer Science, Mathematics
- ICLR
- 2017

A scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs which outperforms related methods by a significant margin. Expand

Learning Convolutional Neural Networks for Graphs

- Computer Science, Mathematics
- ICML
- 2016

This work proposes a framework for learning convolutional neural networks for arbitrary graphs that operate on locally connected regions of the input and demonstrates that the learned feature representations are competitive with state of the art graph kernels and that their computation is highly efficient. Expand

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

- Computer Science, Mathematics
- NIPS
- 2016

This work presents a formulation of CNNs in the context of spectral graph theory, which provides the necessary mathematical background and efficient numerical schemes to design fast localized convolutional filters on graphs. Expand

Diffusion-Convolutional Neural Networks

- Computer Science, Mathematics
- NIPS
- 2016

Through the introduction of a diffusion-convolution operation, it is shown how diffusion-based representations can be learned from graph-structured data and used as an effective basis for node classification. Expand

Spectral Networks and Locally Connected Networks on Graphs

- Computer Science, Mathematics
- ICLR
- 2014

This paper considers possible generalizations of CNNs to signals defined on more general domains without the action of a translation group, and proposes two constructions, one based upon a hierarchical clustering of the domain, and another based on the spectrum of the graph Laplacian. Expand

Revisiting Semi-Supervised Learning with Graph Embeddings

- Computer Science, Mathematics
- ICML
- 2016

On a large and diverse set of benchmark tasks, including text classification, distantly supervised entity extraction, and entity classification, the proposed semi-supervised learning framework shows improved performance over many of the existing models. Expand

Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs

- Computer Science
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- 2017

This paper proposes a unified framework allowing to generalize CNN architectures to non-Euclidean domains (graphs and manifolds) and learn local, stationary, and compositional task-specific features and test the proposed method on standard tasks from the realms of image-, graph-and 3D shape analysis and show that it consistently outperforms previous approaches. Expand