Multiscale Voxel Based Decoding For Enhanced Natural Image Reconstruction From Brain Activity

@article{Halac2022MultiscaleVB,
  title={Multiscale Voxel Based Decoding For Enhanced Natural Image Reconstruction From Brain Activity},
  author={Mali Halac and Murat Isik and Hasan Ayaz and Anup Das},
  journal={2022 International Joint Conference on Neural Networks (IJCNN)},
  year={2022},
  pages={1-7}
}
  • Mali HalacMurat Isik Anup Das
  • Published 27 May 2022
  • Computer Science
  • 2022 International Joint Conference on Neural Networks (IJCNN)
Reconstructing perceived images from human brain activity monitored by functional magnetic resonance imaging (fMRI) is hard, especially for natural images. Existing methods often result in blurry and unintelligible reconstructions with low fidelity. In this study, we present a novel approach for enhanced image reconstruction, in which existing methods for object decoding and image reconstruction are merged together. This is achieved by conditioning the reconstructed image to its decoded image… 
[PDF] Semantic Reader

Figures from this paper

References

SHOWING 1-10 OF 25 REFERENCES

Generic decoding of seen and imagined objects using hierarchical visual features

It is shown that visual features, including those derived from a deep convolutional neural network, can be predicted from fMRI patterns, and that greater accuracy is achieved for low-/high-level features with lower-/higher-level visual areas, respectively.

Generative Adversarial Nets

We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a…

Large Scale GAN Training for High Fidelity Natural Image Synthesis

It is found that applying orthogonal regularization to the generator renders it amenable to a simple "truncation trick," allowing fine control over the trade-off between sample fidelity and variety by reducing the variance of the Generator's input.

Perceived Image Decoding From Brain Activity Using Shared Information of Multi-Subject fMRI Data

Experimental results obtained by using two public fMRI datasets showed that the proposed method can more accurately decode categories of perceived images from fMRI activity than previous approaches using a single subject’s fMRI data.

Reconstructing seen image from brain activity by visually-guided cognitive representation and adversarial learning

Generative Adversarial Networks and Its Applications in Biomedical Informatics

The origin, specific working principle, and development history of GAN are introduced, various applications of GAn in digital image processing, Cycle-GAN, and its application in medical imaging analysis, as well as the latest applications in medical informatics and bioinformatics are introduced.

BigGAN-based Bayesian Reconstruction of Natural Images from Human Brain Activity

From voxels to pixels and back: Self-supervision in natural-image reconstruction from fMRI

This work presents a novel approach which allows to train fMRI-to-image reconstruction networks also on "unlabeled" data, and adapts the reconstruction network to the new input test-data, despite its deviation from the statistics of the scarce training data.

End-to-end deep image reconstruction from human brain activity

This work presents an attempt to directly train a DNN model with fMRI data and the corresponding stimulus images to build an end-to-end reconstruction model, and suggests a potential for the end- to-end framework to learn a direct mapping between brain activity and perception given even larger datasets.

Category Decoding of Visual Stimuli From Human Brain Activity Using a Bidirectional Recurrent Neural Network to Simulate Bidirectional Information Flows in Human Visual Cortices

  • Kai QiaoJian Chen Bin Yan
  • Computer Science
    Front. Neurosci.
  • 2019
Comparative analysis validated and revealed that correlative representations of categories were included in visual cortices because of the bidirectional information flows, in addition to the hierarchical, distributed, and complementary representations that accorded with previous studies.