• Corpus ID: 239015937

A Heterogeneous Graph Based Framework for Multimodal Neuroimaging Fusion Learning

  title={A Heterogeneous Graph Based Framework for Multimodal Neuroimaging Fusion Learning},
  author={Gen Shi and Yifan Zhu and Wenjin Liu and Xuesong Li},
Here, we present a Heterogeneous Graph neural network for Multimodal neuroimaging fusion learning (HGM). Traditional GNN-based models usually assume the brain network is a homogeneous graph with single type of nodes and edges. However, vast literatures have shown the heterogeneity of the human brain especially between the two hemispheres. Homogeneous brain network is insufficient to model the complicated brain state. Therefore, in this work we firstly model the brain network as heterogeneous… 

Figures and Tables from this paper


Attention-Diffusion-Bilinear Neural Network for Brain Network Analysis
An Attention-Diffusion-Bilinear Neural Network (ADB-NN) framework for brain network analysis, which is trained in an end-to-end manner and generates a joint representation of FC and SC for diagnosis.
Deep Representation Learning For Multimodal Brain Networks
This work proposes a novel end-to-end deep graph representation learning (Deep Multimodal Brain Networks - DMBN) to fuse multimodal brain networks and decipher the cross-modality relationship through a graph encoding and decoding process.
Semi-Supervised Classification with Graph Convolutional Networks
A scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs which outperforms related methods by a significant margin.
Deep Graph Infomax
Deep Graph Infomax (DGI) is presented, a general approach for learning node representations within graph-structured data in an unsupervised manner that is readily applicable to both transductive and inductive learning setups.
The Human Brainnetome Atlas: A New Brain Atlas Based on Connectional Architecture
A connectivity-based parcellation framework is designed that identifies the subdivisions of the entire human brain, revealing the in vivo connectivity architecture and provides a fine-grained, cross-validated atlas and contains information on both anatomical and functional connections.
Graph Attention Networks
We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior
DeepWalk: online learning of social representations
DeepWalk is an online learning algorithm which builds useful incremental results, and is trivially parallelizable, which make it suitable for a broad class of real world applications such as network classification, and anomaly detection.
M-GCN: A Multimodal Graph Convolutional Network to Integrate Functional and Structural Connectomics Data to Predict Multidimensional Phenotypic Characterizations
This work demonstrates that the M-GCN outperforms several state-of-the-art baselines in a five-fold cross validated setting and extracts predictive biomarkers from both healthy and autistic populations.
Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning
This paper proposes a novel co-contrastive learning mechanism for HGNNs, named HeCo, which differs from traditional contrastive learning which only focuses on contrasting positive and negative samples, and employs cross-view contrastive mechanism.