• Corpus ID: 239015937

A Heterogeneous Graph Based Framework for Multimodal Neuroimaging Fusion Learning

  title={A Heterogeneous Graph Based Framework for Multimodal Neuroimaging Fusion Learning},
  author={Gen Shi and Yifan Zhu and Wenjin Liu and Xuesong Li},
Here, we present a Heterogeneous Graph neural network for Multimodal neuroimaging fusion learning (HGM). Traditional GNN-based models usually assume the brain network is a homogeneous graph with single type of nodes and edges. However, vast literatures have shown the heterogeneity of the human brain especially between the two hemispheres. Homogeneous brain network is insufficient to model the complicated brain state. Therefore, in this work we firstly model the brain network as heterogeneous… 

Figures and Tables from this paper


Deep Representation Learning For Multimodal Brain Networks
This work proposes a novel end-to-end deep graph representation learning (Deep Multimodal Brain Networks - DMBN) to fuse multimodal brain networks and decipher the cross-modality relationship through a graph encoding and decoding process.
Integrating Heterogeneous Brain Networks for Predicting Brain Disease Conditions
A graph convolutional network (GCN) framework is developed to integrate heterogeneous brain networks and achieves state-of-the-art prediction results, and validates the advantages of the utilization of multi-modal brain networks.
Attention-Diffusion-Bilinear Neural Network for Brain Network Analysis
An Attention-Diffusion-Bilinear Neural Network (ADB-NN) framework for brain network analysis, which is trained in an end-to-end manner and generates a joint representation of FC and SC for diagnosis.
Understanding Graph Isomorphism Network for rs-fMRI Functional Connectivity Analysis
This paper develops a framework for analyzing the fMRI data using the Graph Isomorphism Network (GIN), which was recently proposed as a powerful GNN for graph classification, and validate the proposed framework using large-scale resting-state fMRI (rs-fMRI) data for classifying the sex of the subject based on the graph structure of the brain.
Hi-GCN: A hierarchical graph convolution network for graph embedding learning of brain network and brain disorders prediction
The proposed hi-GCN method performs the graph embedding learning from a hierarchical perspective while considering the structure in individual brain network and the subject's correlation in the global population network, which can capture the most essential embedding features to improve the classification performance of disease diagnosis.
Unified Brain Network with Functional and Structural Data
A unified brain network construction algorithm that is jointly learned from both functional and structural data is proposed, and it is applied to epilepsy diagnosis and experimental results show that the method is effective and can achieve a promising performance in diagnosis of epilepsy.
Functional network estimation using multigraph learning with application to brain maturation study
A multigraph learning model is proposed that can extract brain connectivity network more accurately by considering the latent common structures between different fMRI paradigms, which are significant for both understanding brain development and recognizing population groups of different ages.
Metric learning with spectral graph convolutions on brain connectivity networks
The proposed framework takes into consideration the graph structure for the evaluation of similarity between a pair of graphs, by employing spectral graph convolutions that allow the generalisation of traditional convolutions to irregular graphs and operates in the graph spectral domain.
Persistent Feature Analysis of Multimodal Brain Networks Using Generalized Fused Lasso for EMCI Identification
A novel multimodal brain network modeling method that employs the structural connectivity based on diffusion tensor imaging (DTI), as a constraint, to guide the regression of BOLD time series from resting state functional magnetic resonance imaging (rs-fMRI).
A graph neural network framework for causal inference in brain networks
The proposed multi-modal GNN framework can provide a novel perspective on the structure-function relationship in the brain and appears to be promising for the characterization of the information flow in brain networks.