Emergence of simple-cell receptive field properties by learning a sparse code for natural images

@article{Olshausen1996EmergenceOS,
  title={Emergence of simple-cell receptive field properties by learning a sparse code for natural images},
  author={Bruno A. Olshausen and David J. Field},
  journal={Nature},
  year={1996},
  volume={381},
  pages={607-609}
}
THE receptive fields of simple cells in mammalian primary visual cortex can be characterized as being spatially localized, oriented1–4 and bandpass (selective to structure at different spatial scales), comparable to the basis functions of wavelet transforms5,6. [...] Key Result The resulting sparse image code provides a more efficient representation for later stages of processing because it possesses a higher degree of statistical independence among its outputs.Expand
Development of localized oriented receptive fields by learning a translation-invariant code for natural images.
TLDR
It is shown that a strategy for transformation-invariant coding of images based on a first-order Taylor series expansion of an image also causes localized, oriented receptive fields to be learned from natural image inputs. Expand
Are sparse-coding simple cell receptive field models physiologically plausible?
  • P. Watters
  • Computer Science, Medicine
  • Journal of integrative neuroscience
  • 2006
TLDR
Across all image themes, basis function sizes, number of basis functions, sparseness factors and learning rates, the spatial-frequency tuning did not closely resemble that of primate area 17 -- the model results more closely resembled the unclassified cat neurones of area 19 with a single exception, and not area 17 as predicted. Expand
Receptive fields of simple cells from a taxonomic study of natural images and suppression of scale redundancy
TLDR
Two principles emerging from the study of image statistics are: first, there exists scale redundancy and this can be eliminated from the code; second, images are constructed by a combination (partly linear and partly non-linear) of some simple patterns of contrast (edges, bars, and composites of these two). Expand
Emergence of complex cell properties by learning to generalize in natural scenes
TLDR
A model in which neural activity encodes the probability distribution most consistent with a given image is presented, which provides a new functional explanation for nonlinear effects in complex cells and offers insight into coding strategies in primary visual cortex (V1) and higher visual areas. Expand
Efficient Encoding of Natural Time Varying Images Produces Oriented Space-Time Receptive Fields
TLDR
An artificial neural network is described that attempts to accurately reconstruct its spatiotemporal input data while simultaneously reducing the statistical dependencies between its outputs. Expand
Finding the optimal sparse, overcomplete model for natural images by model selection
TLDR
The goal of this work is to investigate the claim that sparse, overcomplete codes might lend some computational advantage in the processing of visual information. Expand
Sparse coding with an overcomplete basis set: A strategy employed by V1?
TLDR
These deviations from linearity provide a potential explanation for the weak forms of non-linearity observed in the response properties of cortical simple cells, and they further make predictions about the expected interactions among units in response to naturalistic stimuli. Expand
Hierarchical Learning from Natural Images
In this paper, we apply unsupervised learning methods to construct response functions for V1 simple cells, V1 complex cells, and V2 simple cells from a set of natural images. To support this, weExpand
Localized Receptive Fields May Mediate Transformation-Invariant Recognition in the Visual Cortex
TLDR
It is shown that a relatively simple neural solution to the problem of transformation-invariant visual recognition also causes localized, oriented receptive fields to be learned from natural images. Expand
A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images
TLDR
A highly parsimonious model is obtained of how the properties of the visual cortex are adapted to the characteristics of the natural input by maximizing the sparseness of locally pooled energies. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 19 REFERENCES
Relations between the statistics of natural images and the response properties of cortical cells.
  • D. Field
  • Computer Science, Medicine
  • Journal of the Optical Society of America. A, Optics and image science
  • 1987
TLDR
The results obtained with six natural images suggest that the orientation and the spatial-frequency tuning of mammalian simple cells are well suited for coding the information in such images if the goal of the code is to convert higher-order redundancy into first- order redundancy. Expand
Entropy reduction and decorrelation in visual coding by oriented neural receptive fields
  • J. Daugman
  • Computer Science, Medicine
  • IEEE Transactions on Biomedical Engineering
  • 1989
TLDR
The present image coding simulations, based on quantitative neurobiological data about the code primitives, provide measures of the bit-rate efficiency of such oriented, quadrature, neural codes. Expand
What Is the Goal of Sensory Coding?
  • D. Field
  • Mathematics, Computer Science
  • Neural Computation
  • 1994
TLDR
It is proposed that compact coding schemes are insufficient to account for the receptive field properties of cells in the mammalian visual pathway and suggested that natural scenes, to a first approximation, can be considered as a sum of self-similar local functions (the inverse of a wavelet). Expand
An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex.
TLDR
It seems that an optimal strategy has evolved for sampling images simultaneously in the 2D spatial and spatial frequency domains and the Gabor function provides a useful and reasonably accurate description of most spatial aspects of simple receptive fields. Expand
Finding compact and sparse-distributed representations of visual images
Some recent work has investigated the dichotomy between compact coding using dimensionality reduction and sparse-distributed coding in the context of understanding biological information processing.Expand
Spatial frequency selectivity of cells in macaque visual cortex
TLDR
Among other things, it is shown that many stirate cells have quite narrow spatial bandwidths and at a given retinal eccentricity, the distribution of peak frequency covers a wide range of frequencies; these findings support the basic multiple channel notion. Expand
Formation of receptive fields in realistic visual environments according to the Bienenstock, Cooper, and Munro (BCM) theory.
  • C. C. Law, L. Cooper
  • Physics, Medicine
  • Proceedings of the National Academy of Sciences of the United States of America
  • 1994
The Bienenstock, Cooper, and Munro (BCM) theory of synaptic plasticity has successfully reproduced the development of orientation selectivity and ocular dominance in kitten visual cortex in normal,Expand
Self-organization in a perceptual network
TLDR
It is shown that even a single developing cell of a layered network exhibits a remarkable set of optimization properties that are closely related to issues in statistics, theoretical physics, adaptive signal processing, the formation of knowledge representation in artificial intelligence, and information theory. Expand
Real and optimal neural images in early vision
TLDR
Compared neural images obtained by scanning an image while recording from a second-order neuron in the fly visual system are compared with theoretical calculations based on maximizing information, taking into account the statistical structure of natural images. Expand
Two-dimensional spatial structure of receptive fields in monkey striate cortex.
  • A. Parker, M. Hawken
  • Physics, Medicine
  • Journal of the Optical Society of America. A, Optics and image science
  • 1988
Measurements of the spatial contrast sensitivity function and orientation selectivity of visual neurons in the foveal striate cortex (V1) of primates were interpreted within the context of a model ofExpand
...
1
2
...