Generalizing Brain Decoding Across Subjects with Deep Learning

  title={Generalizing Brain Decoding Across Subjects with Deep Learning},
  author={Richard Csaky and Mats W. J. van Es and Oiwi Parker Jones and Mark W. Woolrich},
Decoding experimental variables from brain imaging data is gaining popularity, with applications in brain-computer interfaces and the study of neural representations. Decoding is typically subject-specific and does not generalise well over subjects. Here, we investigate ways to achieve cross-subject decoding. We used magnetoencephalography (MEG) data where 15 subjects viewed 118 different images, with 30 examples per image. Training on the entire 1s window following the presentation of each… 



MEG decoding across subjects

This work formally describes the problem of decoding across subjects for magnetoen-cephalographic (MEG) experiments and proposes the use of ensemble learning, and specifically of stacked generalization, to address the variability across subjects within train data, with the aim of producing more stable classifiers.

Sequence Transfer Learning for Neural Decoding

This work demonstrates that LSTMS are a versatile model that can accurately capture temporal patterns in neural data and can provide a foundation for transfer learning in neural decoding.

Inter-Subject MEG Decoding for Visual Information with Hybrid Gated Recurrent Network

A hybrid gated recurrent network (HGRN) is proposed for inter-subject visual MEG decoding and can be considered as a new tool for decoding and analyzing brain MEG signal, which is significant for visual cognitive research in neuroscience.

Optimizing Layers Improves CNN Generalization and Transfer Learning for Imagined Speech Decoding from EEG

Two distinct TL methodologies are employed to classify EEG data corresponding to imagined speech production of vowels, using a deep convolutional neural network (CNN), and fine-tuning of the input layers resulted in the highest overall accuracy.

Across-subject offline decoding of motor imagery from MEG and EEG

MTL and training with other subject’s MI is efficient for inter-subject decoding of MI, and passive movements of other subjects are likely suboptimal for training the MI classifiers.

Classification of imagined spoken Word-Pairs using Convolutional Neural Networks

The potential of deep learning is considered as a possible alternative to traditional BCI methodologies in relation to imagined speech EEG decoding, and two different convolutional neural networks were trained on multiple imagined speech word-pairs.

Intra- and Inter-subject Variability in EEG-Based Sensorimotor Brain Computer Interface: A Review

  • S. SahaM. Baumert
  • Psychology, Computer Science
    Frontiers in Computational Neuroscience
  • 2019
The importance of measuring inter-session/subject performance predictors for generalized BCI frameworks for both normal and motor-impaired people is highlighted, reducing the necessity for tedious and annoying calibration sessions and BCI training.

Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence

It was shown that the DNN captured the stages of human visual processing in both time and space from early visual areas towards the dorsal and ventral streams and provided an algorithmically informed view on the spatio-temporal dynamics of visual object recognition in the human visual brain.