Using goal-driven deep learning models to understand sensory cortex

@article{Yamins2016UsingGD,
  title={Using goal-driven deep learning models to understand sensory cortex},
  author={Daniel Yamins and James J. DiCarlo},
  journal={Nature Neuroscience},
  year={2016},
  volume={19},
  pages={356-365}
}
Fueled by innovation in the computer vision and artificial intelligence communities, recent developments in computational neuroscience have used goal-driven hierarchical convolutional neural networks (HCNNs) to make strides in modeling neural single-unit and population responses in higher visual cortical areas. In this Perspective, we review the recent progress in a broader modeling context and describe some of the key technical innovations that have supported it. We then outline how the goal… Expand
How can deep learning advance computational modeling of sensory information processing?
TLDR
It is discussed how DNNs are amenable to new model comparison techniques that allow for stronger conclusions to be made about the computational mechanisms underlying sensory information processing. Expand
A Residual Neural-Network Model to Predict Visual Cortex Measurements
TLDR
The asset of using a residual neural network of only 20 layers for this task is that earlier stages of the network can be more easily trained, which allows us to add more layers at the earlier stage. Expand
A shallow residual neural network to predict the visual cortex response
TLDR
The asset of using a shallow residual neural network for this task is that earlier stages of the network can be accurately trained, which allows us to add more layers at the earlier stage and the prediction of the visual brain activity improves. Expand
Deep Reinforcement Learning Models Predict Visual Responses in the Brain: A Preliminary Result
TLDR
This work uses reinforcement learning to train neural network models to play a 3D computer game and finds that these reinforcement learning models achieve neural response prediction accuracy scores in the early visual areas in the levels that are comparable to those accomplished by the supervised neural network model. Expand
Deep neural network models of sensory systems: windows onto the role of task constraints
TLDR
This work considers how task-optimized networks can generate hypotheses about neural representations and functional organization in ways that are analogous to traditional ideal observer models, and highlights the potential uses of deep neural networks to reveal how task performance may constrain neural systems and behavior. Expand
Towards more biologically plausible deep learning and visual processing
TLDR
This thesis discusses several aspects of modern ANNs that can be made more consistent with biology and proposes biologically-plausible modifications of the ANN models to make them more implementable by the brain while maintaining (or even improving) their performance. Expand
Generalization in data-driven models of primary visual cortex
TLDR
This model with its novel readout sets a new state-of-the-art for neural response prediction in mouse visual cortex from natural images, generalizes between animals, and captures better characteristic cortical features than current task-driven pre-training approaches such as VGG16. Expand
Deep convolutional models improve predictions of macaque V1 responses to natural images
TLDR
Multi-layer convolutional neural networks (CNNs) set the new state of the art for predicting neural responses to natural images in primate V1 and deep features learned for object recognition are better explanations for V1 computation than all previous filter bank theories. Expand
Neural Population Control via Deep Image Synthesis
TLDR
These results show how the knowledge embedded in today’s ANN models might be used to non-invasively set desired internal brain states at neuron-level resolution, and suggest that more accurate ANN models would produce even more accurate control. Expand
Deep convolutional models improve predictions of macaque V1 responses to natural images
TLDR
Multi-layer convolutional neural networks (CNNs) set the new state of the art for predicting neural responses to natural images in primate V1 and deep features learned for object recognition are better explanations for V1 computation than all previous filter bank theories. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 78 REFERENCES
Human-level control through deep reinforcement learning
TLDR
This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks. Expand
Performance-optimized hierarchical models predict neural responses in higher visual cortex
TLDR
This work uses computational techniques to identify a high-performing neural network model that matches human performance on challenging object categorization tasks and shows that performance optimization—applied in a biologically appropriate model class—can be used to build quantitative predictive models of neural processing. Expand
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream
TLDR
It is quantitatively shown that there indeed exists an explicit gradient for feature complexity in the ventral pathway of the human brain, and this provides strong support for the hypothesis that object categorization is a guiding principle in the functional organization of the primate ventral stream. Expand
Hierarchical Modular Optimization of Convolutional Networks Achieves Representations Similar to Macaque IT and Human Ventral Stream
TLDR
This work construct models of the ventral stream using a novel optimization procedure for category-level object recognition problems, and produce RDMs resembling both macaque IT and human ventralstream, which develops a long-standing functional hypothesis that the Ventral visual stream is a hierarchically arranged series of processing stages optimized for visual object recognition. Expand
Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
TLDR
These evaluations show that, unlike previous bio-inspired models, the latest DNNs rival the representational performance of IT cortex on this visual object recognition task and propose an extension of “kernel analysis” that measures the generalization accuracy as a function of representational complexity. Expand
Hierarchical models of object recognition in cortex
TLDR
A new hierarchical model consistent with physiological data from inferotemporal cortex that accounts for this complex visual task and makes testable predictions is described. Expand
Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation
TLDR
The results suggest that explaining IT requires computational features trained through supervised learning to emphasize the behaviorally important categorical divisions prominently reflected in IT. Expand
Going deeper with convolutions
We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual RecognitionExpand
Context-dependent computation by recurrent dynamics in prefrontal cortex
TLDR
This work studies prefrontal cortex activity in macaque monkeys trained to flexibly select and integrate noisy sensory inputs towards a choice, and finds that the observed complexity and functional roles of single neurons are readily understood in the framework of a dynamical process unfolding at the level of the population. Expand
Metamers of the ventral stream
TLDR
A population model for mid-ventral processing is developed, in which nonlinear combinations of V1 responses are averaged in receptive fields that grow with eccentricity, providing a quantitative framework for assessing the capabilities and limitations of everyday vision. Expand
...
1
2
3
4
5
...