• Corpus ID: 240419872

Evaluating deep transfer learning for whole-brain cognitive decoding

  title={Evaluating deep transfer learning for whole-brain cognitive decoding},
  author={Armin W. Thomas and Ulman Lindenberger and Wojciech Samek and Klaus-Robert M{\"u}ller},
Research in many fields has shown that transfer learning (TL) is well-suited to improve the performance of deep learning (DL) models in datasets with small numbers of samples. This empirical success has triggered interest in the application of TL to cognitive decoding analyses with functional neuroimaging data. Here, we systematically evaluate TL for the application of DL models to the decoding of cognitive states (e.g., viewing images of faces or houses) from whole-brain functional Magnetic… 
Self-Supervised Learning Of Brain Dynamics From Broad Neuroimaging Data
A set of novel self-supervised learning frameworks for neuroimaging data based on prominent learning frameworks in NLP show that the pre-trained models transfer well, outperforming baseline models when adapted to the data of only a few individuals, while models pre- trained in a learning framework based on causal language modeling clearly outperform the others.
Comparing interpretation methods in mental state decoding analyses with deep learning models
A comparison of the explanations of prominent interpretation methods for the mental state decoding decisions of DL models trained on three functional Magnetic Resonance Imaging (fMRI) datasets finds that interpretation methods that capture the model’s decision process well, by producing faithful explanations, generally produce explanations that are less in line with the results of standard analyses of the fMRI data.


Deep Transfer Learning For Whole-Brain fMRI Analyses
This work shows that a DL model, which has been previously trained on a large openly available fMRI dataset of the Human Connectome Project, outperforms a model variant with the same architecture, but which is trained from scratch, when both are applied to the data of a new, unrelated fMRI task.
Transfer learning of deep neural network representations for fMRI decoding
Deep learning of fMRI big data: a novel approach to subject-transfer decoding
This decoder is the first successful DNN-based subject-transfer decoder and when applied to a large-scale functional magnetic resonance imaging database, it achieved higher decoding accuracy than other baseline methods, including support vector machine (SVM).
Decoding and mapping task states of the human brain via deep learning
Without incurring the burden of handcrafting the features, the proposed deep decoding method can classify brain task states highly accurately, and is a powerful tool for fMRI researchers.
Analyzing Neuroimaging Data Through Recurrent Deep Learning Models
DeepLight outperforms conventional approaches of uni- and multivariate fMRI analysis in decoding the cognitive states and in identifying the physiologically appropriate brain regions associated with these states, and is demonstrated to have the versatility to apply to a large fMRI dataset of the Human Connectome Project.
Deep learning encodes robust discriminative neuroimaging representations to outperform standard machine learning
A large-scale systematic comparison of deep learning methods in multiple classification and regression tasks on structural MRI images shows that if trained following prevalent DL practices, DL methods substantially improve compared to SML methods by encoding robust discriminative brain representations.
Functional Annotation of Human Cognitive States using Deep Graph Convolution
Extracting representations of cognition across neuroimaging studies improves brain decoding
A new methodology to analyze brain responses across tasks without a joint model of the psychological processes is introduced, which boosts statistical power in small studies with specific cognitive focus by analyzing them jointly with large studies that probe less focal mental processes.
Attend and Decode: 4D fMRI Task State Decoding Using Attention Models
A novel architecture called Brain Attend and Decode (BAnD), that uses residual convolutional neural networks for spatial feature extraction and self-attention mechanisms for temporal modeling, and achieves significant performance gain compared to previous works on a 7-task benchmark from the large-scale Human Connectome Project (HCP) dataset.
Challenges for cognitive decoding using deep learning methods
This work proposes to approach cognitive decoding challenges by leveraging recent advances in explainable artificial intelligence and transfer learning, while also providing specific recommendations on how to improve the reproducibility and robustness of DL modeling results.