Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders

@article{Miyawaki2008VisualIR,
  title={Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders},
  author={Yoichi Miyawaki and Hajime Uchida and Okito Yamashita and Masa-aki Sato and Yusuke Morito and Hiroki C. Tanabe and Norihiro Sadato and Yukiyasu Kamitani},
  journal={Neuron},
  year={2008},
  volume={60},
  pages={915-929}
}

Figures from this paper

Visual image reconstruction from human brain activity: A modular decoding approach

TLDR
This study demonstrates constraint-free reconstruction of visual images perceived by a subject, from the brain activity pattern, by combining local image bases with multiple scales, whose contrasts were independently decoded from fMRI activity by automatically selecting relevant voxels and exploiting their correlated patterns.

Visual Image Reconstruction from fMRI Activation Using Multi-scale Support Vector Machine Decoders

TLDR
Support vector machine (SVM) was employed to model the rela-tionship between contrast of local image and fMRI; second, additional 3-pixel image bases were considered and demonstrated that the time consumption in modeling the local image decoder was reduced to 1% by SVM compared to SLR.

Bayesian reconstruction of multiscale local contrast images from brain activity

Attentionally modulated subjective images reconstructed from brain activity

TLDR
The results demonstrate that top-down attention counters stimulus-induced responses and modulate neural representations to render reconstructions in accordance with subjective appearance, opening a new possibility of brain-based communication and creation.

Estimating image bases for visual image reconstruction from human brain activity

TLDR
A probabilistic model that relates the fMRI activity space to the visual image space via a set of latent variables is constructed and it is found that spatially localized, multi-scale image bases were estimated near the fovea, and that the model using the estimated image bases was able to accurately reconstruct novel visual images.

Reconstructing Perceived Images From Human Brain Activities With Bayesian Deep Multiview Learning

TLDR
A novel deep generative multiview model for the accurate visual image reconstruction from the human brain activities measured by functional magnetic resonance imaging (fMRI) by using two view-specific generators with a shared latent space is proposed.

Bayesian Reconstruction of Natural Images from Human Brain Activity

Deep image reconstruction from human brain activity

TLDR
A novel image reconstruction method, in which the pixel values of an image are optimized to make its DNN features similar to those decoded from human brain activity at multiple layers, which found that the generated images resembled the stimulus images and the subjective visual content during imagery.
...

References

SHOWING 1-10 OF 42 REFERENCES

Identifying natural images from human brain activity

TLDR
A decoding method based on quantitative receptive-field models that characterize the relationship between visual stimuli and fMRI activity in early visual areas is developed and it is suggested that it may soon be possible to reconstruct a picture of a person’s visual experience from measurements of brain activity alone.

Inverse retinotopy: Inferring the visual content of images from brain activation patterns

Predicting the orientation of invisible stimuli from activity in human primary visual cortex

TLDR
It is shown that even at conventional resolutions it is possible to use fMRI to obtain a direct measure of orientation-selective processing in V1 and to successfully predict which one of two oriented stimuli a participant was viewing, even when masking rendered that stimulus invisible.

Topographical representations of mental images in primary visual cortex

TLDR
Findings resolve a debate in the literature about whether imagery activates early visual cortex and indicate that visual mental imagery involves 'depictive' representations, not solely language-like descriptions12–14.

Reconstruction of Natural Scenes from Ensemble Responses in the Lateral Geniculate Nucleus

TLDR
A linear decoding technique is used to reconstruct spatiotemporal visual inputs from ensemble responses in the lateral geniculate nucleus of the cat to provide a basis for understanding ensemble coding in the early visual pathway.

Optimal decoding of correlated neural population responses in the primate visual cortex

TLDR
It is found that small visual targets elicit widespread responses in V1, and that response variability at distant sites is highly correlated, which suggests an unexpected role for inhibitory mechanisms in efficient decoding of neural population responses.

Retinotopic organization in human visual cortex and the spatial precision of functional MRI.

TLDR
This work identified the borders between several retinotopically organized visual areas in the posterior occipital lobe and estimated the spatial resolution of the fMRI signal and found that signal amplitude falls to 60% at a spatial frequency of 1 cycle per 9 mm of visual cortex.

Decoding Seen and Attended Motion Directions from Activity in the Human Visual Cortex

Estimating receptive field size from fMRI data in human striate and extrastriate visual cortex.

TLDR
FMRI was used to estimate the average receptive field sizes of neurons in each of several striate and extrastriate visual areas of the human cerebral cortex, and results are qualitatively in line with those obtained by others in macaque monkeys using neurophysiological methods.