• Corpus ID: 237091496

Challenges for cognitive decoding using deep learning methods

  title={Challenges for cognitive decoding using deep learning methods},
  author={Armin W. Thomas and Christopher R{\'e} and Russell A. Poldrack},
In cognitive decoding, researchers aim to characterize a brain region's representations by identifying the cognitive states (e.g., accepting/rejecting a gamble) that can be identified from the region's activity. Deep learning (DL) methods are highly promising for cognitive decoding, with their unmatched ability to learn versatile representations of complex data. Yet, their widespread application in cognitive decoding is hindered by their general lack of interpretability as well as difficulties… 

Figures from this paper

Evaluating deep transfer learning for whole-brain cognitive decoding

This work systematically evaluates TL for the application of DL models to the decoding of cognitive states from whole-brain functional Magnetic Resonance Imaging data and demonstrates that benefits arise from the ability of the pre-trained models to reuse many of their learned features when training with new data, providing deeper insights into the mechanisms giving rise to the benefits of pre-training.

Comparing interpretation methods in mental state decoding analyses with deep learning models

A gradient between two key characteristics of an explanation in mental state decoding, namely, its biological plausibility and faithfulness are demonstrated: interpretation methods with high explanation faithfulness, which capture the model’s decision process well, generally provide explanations that are biologically less plausible than the explanations of interpretations methods with less explanationfaithfulness.

On the benefits of self-taught learning for brain decoding

The results suggest that pre-training can be bene⬁cial when studying difficult classiflcation problems such as those with few training samples or complex classi fication tasks.

Self-Supervised Learning of Brain Dynamics from Broad Neuroimaging Data

A set of novel self-supervised learning frameworks for neuroimaging data based on prominent learning frameworks in NLP show that the pre-trained models transfer well, outperforming baseline models when adapted to the data of only a few individuals, while models pre- trained in a learning framework based on causal language modeling clearly outperform the others.

Interpreting Brain Biomarkers: Challenges and solutions in interpreting machine learning-based predictive neuroimaging

Methods and applications for interpreting brain signatures derived from predictive neuroimaging, based on a survey of 326 research articles, are reviewed to recommend exhaustive validation of the reliability and interpretability of biomarkers across multiple data sets and contexts.

Title : Interpretable brain decoding from sensations to cognition to 1 action : graph neural networks reveal the representational hierarchy of 2 human cognition 3 Running Title : Interpretable cognitive modeling using GNN 4

  • 2022



Analyzing Neuroimaging Data Through Recurrent Deep Learning Models

DeepLight outperforms conventional approaches of uni- and multivariate fMRI analysis in decoding the cognitive states and in identifying the physiologically appropriate brain regions associated with these states, and is demonstrated to have the versatility to apply to a large fMRI dataset of the Human Connectome Project.

Deep Transfer Learning For Whole-Brain fMRI Analyses

This work shows that a DL model, which has been previously trained on a large openly available fMRI dataset of the Human Connectome Project, outperforms a model variant with the same architecture, but which is trained from scratch, when both are applied to the data of a new, unrelated fMRI task.

Deep learning of fMRI big data: a novel approach to subject-transfer decoding

This decoder is the first successful DNN-based subject-transfer decoder and when applied to a large-scale functional magnetic resonance imaging database, it achieved higher decoding accuracy than other baseline methods, including support vector machine (SVM).

Decoding and mapping task states of the human brain via deep learning

Without incurring the burden of handcrafting the features, the proposed deep decoding method can classify brain task states highly accurately, and is a powerful tool for fMRI researchers.

Improving Whole-Brain Neural Decoding of fMRI with Domain Adaptation

A domain adaptation framework for whole- brain fMRI (DawfMRI) to improve whole-brain neural decoding on target data leveraging pre-existing source data is proposed and a plausible relationship between psychological similarity and adaptation effectiveness is discovered.

Attend and Decode: 4D fMRI Task State Decoding Using Attention Models

A novel architecture called Brain Attend and Decode (BAnD), that uses residual convolutional neural networks for spatial feature extraction and self-attention mechanisms for temporal modeling, and achieves significant performance gain compared to previous works on a 7-task benchmark from the large-scale Human Connectome Project (HCP) dataset.

Extracting representations of cognition across neuroimaging studies improves brain decoding

A new methodology to analyze brain responses across tasks without a joint model of the psychological processes is introduced, which boosts statistical power in small studies with specific cognitive focus by analyzing them jointly with large studies that probe less focal mental processes.

Learning Important Features Through Propagating Activation Differences

DeepLIFT (Deep Learning Important FeaTures), a method for decomposing the output prediction of a neural network on a specific input by backpropagating the contributions of all neurons in the network to every feature of the input, is presented.