Graph Attention Networks

@article{Velickovic2018GraphAN,
  title={Graph Attention Networks},
  author={Petar Velickovic and Guillem Cucurull and Arantxa Casanova and Adriana Romero and Pietro Lio’ and Yoshua Bengio},
  journal={ArXiv},
  year={2018},
  volume={abs/1710.10903}
}
We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations. [...] Key Result Our GAT models have achieved or matched state-of-the-art results across four established transductive and inductive graph benchmarks: the Cora, Citeseer and Pubmed citation network datasets, as well as a protein-protein interaction dataset…Expand
Graph Representation Learning Network via Adaptive Sampling
TLDR
A new architecture to address issues of scalability and efficiency with GAT and GraphSAGE is proposed that is more efficient and is capable of incorporating different edge type information. Expand
DAGCN: Dual Attention Graph Convolutional Networks
TLDR
DAGCN automatically learns the importance of neighbors at different hops using a novel attention graph convolution layer, and then employs a second attention component, a self-attention pooling layer, to generalize the graph representation from the various aspects of a matrix graph embedding. Expand
Improving Graph Attention Networks with Large Margin-based Constraints
TLDR
This work first theoretically demonstrate the over-smoothing behavior of GATs and then develops an approach using constraint on the attention weights according to the class boundary and feature aggregation pattern, which leads to significant improvements over the previous state-of-the-art graph attention methods on all datasets. Expand
Graph Attention Networks with Positional Embeddings
TLDR
This work proposes a framework, termed Graph Attentional Networks with Positional Embeddings (GAT-POS), to enhance GATs with positional embeddings which capture structural and positional information of the nodes in the graph. Expand
Understanding Attention and Generalization in Graph Neural Networks
TLDR
This work proposes an alternative recipe and train attention in a weakly-supervised fashion that approaches the performance of supervised models, and, compared to unsupervised models, improves results on several synthetic as well as real datasets. Expand
Graphs, Entities, and Step Mixture
TLDR
A new graph neural network that considers both edge-based neighborhood relationships and node-based entity features, i.e. Graph Entities with Step Mixture via random walk (GESM), which achieves state-of-the-art or comparable performances on eight benchmark graph datasets comprising transductive and inductive learning tasks. Expand
GraphMix: Regularized Training of Graph Neural Networks for Semi-Supervised Learning
TLDR
This work proposes a unified approach in which a fully-connected network is trained jointly with the graph neural network via parameter sharing, interpolation-based regularization, and self-predicted-targets. Expand
Node Masking: Making Graph Neural Networks Generalize and Scale Better
TLDR
This paper discusses some theoretical tools to better visualize the operations performed by state of the art spatial GNNs and introduces a simple concept, node masking, that allows them to generalize and scale better. Expand
Graph-CAT: Graph Co-Attention Networks via local and global attribute augmentations
TLDR
This paper proposes a novel Graph Co-ATtention Network (Graph-CAT), which performs both the local and global attribute augmentations based on two different yet complementary attention schemes and demonstrates the superiority of the Graph-C AT compared to the state-of-the-art methods. Expand
Learnt Sparsification for Interpretable Graph Neural Networks
TLDR
This paper proposes a novel method called KEdge for explicitly sparsification using the Hard Kumaraswamy distribution that can be used in conjugation with any GNN model and effectively counters the over-smoothing phenomena in deep GNNs by maintaining good task performance with increasing GNN layers. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 47 REFERENCES
Gated Graph Sequence Neural Networks
TLDR
This work studies feature learning techniques for graph-structured inputs and achieves state-of-the-art performance on a problem from program verification, in which subgraphs need to be matched to abstract data structures. Expand
Deep Convolutional Networks on Graph-Structured Data
TLDR
This paper develops an extension of Spectral Networks which incorporates a Graph Estimation procedure, that is test on large-scale classification problems, matching or improving over Dropout Networks with far less parameters to estimate. Expand
Inductive Representation Learning on Large Graphs
TLDR
GraphSAGE is presented, a general, inductive framework that leverages node feature information (e.g., text attributes) to efficiently generate node embeddings for previously unseen data and outperforms strong baselines on three inductive node-classification benchmarks. Expand
Semi-Supervised Classification with Graph Convolutional Networks
TLDR
A scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs which outperforms related methods by a significant margin. Expand
Learning Convolutional Neural Networks for Graphs
TLDR
This work proposes a framework for learning convolutional neural networks for arbitrary graphs that operate on locally connected regions of the input and demonstrates that the learned feature representations are competitive with state of the art graph kernels and that their computation is highly efficient. Expand
Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
TLDR
This work presents a formulation of CNNs in the context of spectral graph theory, which provides the necessary mathematical background and efficient numerical schemes to design fast localized convolutional filters on graphs. Expand
Diffusion-Convolutional Neural Networks
TLDR
Through the introduction of a diffusion-convolution operation, it is shown how diffusion-based representations can be learned from graph-structured data and used as an effective basis for node classification. Expand
Spectral Networks and Locally Connected Networks on Graphs
TLDR
This paper considers possible generalizations of CNNs to signals defined on more general domains without the action of a translation group, and proposes two constructions, one based upon a hierarchical clustering of the domain, and another based on the spectrum of the graph Laplacian. Expand
Revisiting Semi-Supervised Learning with Graph Embeddings
TLDR
On a large and diverse set of benchmark tasks, including text classification, distantly supervised entity extraction, and entity classification, the proposed semi-supervised learning framework shows improved performance over many of the existing models. Expand
Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs
TLDR
This paper proposes a unified framework allowing to generalize CNN architectures to non-Euclidean domains (graphs and manifolds) and learn local, stationary, and compositional task-specific features and test the proposed method on standard tasks from the realms of image-, graph-and 3D shape analysis and show that it consistently outperforms previous approaches. Expand
...
1
2
3
4
5
...