Sparse coding with an overcomplete basis set: A strategy employed by V1?

@article{Olshausen1997SparseCW,
  title={Sparse coding with an overcomplete basis set: A strategy employed by V1?},
  author={Bruno A. Olshausen and David J. Field},
  journal={Vision Research},
  year={1997},
  volume={37},
  pages={3311-3325}
}
The spatial receptive fields of simple cells in mammalian striate cortex have been reasonably well described physiologically and can be characterized as being localized, oriented, and bandpass, comparable with the basis functions of wavelet transforms. Previously, we have shown that these receptive field properties may be accounted for in terms of a strategy for producing a sparse distribution of output activity in response to natural images. Here, in addition to describing this work in a more… Expand
Finding the optimal sparse, overcomplete model for natural images by model selection
TLDR
The goal of this work is to investigate the claim that sparse, overcomplete codes might lend some computational advantage in the processing of visual information. Expand
Are sparse-coding simple cell receptive field models physiologically plausible?
  • P. Watters
  • Computer Science, Medicine
  • Journal of integrative neuroscience
  • 2006
TLDR
Across all image themes, basis function sizes, number of basis functions, sparseness factors and learning rates, the spatial-frequency tuning did not closely resemble that of primate area 17 -- the model results more closely resembled the unclassified cat neurones of area 19 with a single exception, and not area 17 as predicted. Expand
Learning Overcomplete Representations
TLDR
It is shown that overcomplete bases can yield a better approximation of the underlying statistical distribution of the data and can thus lead to greater coding efficiency and provide a method for Bayesian reconstruction of signals in the presence of noise and for blind source separation when there are more sources than mixtures. Expand
On the Choice of a Sparse Prior
TLDR
The constraint of unit variance of neuronal activity is included into the objective functions of the generative model, used in most studies, and it is shown that the effective objective functions are largely dominated by the constraint, and are therefore very similar. Expand
Sparse Coding of Natural Images Using an Overcomplete Set of Limited Capacity Units
TLDR
A model that best balances redundancy reduction and redundant representation is proposed, which accounts for the localized and oriented structure of simple cells, but it also predicts a different organization for the population. Expand
New Evidences for Sparse Coding Strategy Employed in Visual Neurons: from the Image Processing and Nonlinear Approximation Viewpoint
TLDR
It is shown that several newly proposed systems in the area of image processing and nonlinear approximation provide new evidences for the ‘sparse coding’ strategy along a contrary line. Expand
Explicit Object Representation by Sparse Neural Codes
TLDR
This thesis discusses the issue of how best to quantify sparseness, particularly in very sparse systems where biases are significant, and shows results obtained by applying an existing model of sparse coding both to unsupervised category discovery in images and to differentiation between images of different individuals. Expand
Efficient Sparse Coding in Early Sensory Processing: Lessons from Signal Recovery
TLDR
It is argued that higher level overcompleteness becomes computationally tractable by imposing sparsity on synaptic activity and it is shown that such structural sparsity can be facilitated by statistics based decomposition of the stimuli into typical and atypical parts prior to sparse coding. Expand
Development of localized oriented receptive fields by learning a translation-invariant code for natural images.
TLDR
It is shown that a strategy for transformation-invariant coding of images based on a first-order Taylor series expansion of an image also causes localized, oriented receptive fields to be learned from natural image inputs. Expand
PROBABILISTIC FRAMEWORK FOR THE ADAPTATION AND COMPARISON OF IMAGE CODES
TLDR
The learned bases are shown to have better coding efficiency than traditional Fourier and wavelet bases and to provide a Bayesian solution to the problems of image denoising and filling in of missing pixels. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 71 REFERENCES
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
TLDR
It is shown that a learning algorithm that attempts to find sparse linear codes for natural scenes will develop a complete family of localized, oriented, bandpass receptive fields, similar to those found in the primary visual cortex. Expand
What Is the Goal of Sensory Coding?
  • D. Field
  • Mathematics, Computer Science
  • Neural Computation
  • 1994
TLDR
It is proposed that compact coding schemes are insufficient to account for the receptive field properties of cells in the mammalian visual pathway and suggested that natural scenes, to a first approximation, can be considered as a sum of self-similar local functions (the inverse of a wavelet). Expand
Entropy reduction and decorrelation in visual coding by oriented neural receptive fields
  • J. Daugman
  • Computer Science, Medicine
  • IEEE Transactions on Biomedical Engineering
  • 1989
TLDR
The present image coding simulations, based on quantitative neurobiological data about the code primitives, provide measures of the bit-rate efficiency of such oriented, quadrature, neural codes. Expand
Natural image statistics and efficient coding.
TLDR
It is suggested that a good objective for an efficient coding of natural Scenes is to maximize the sparseness of the representation, and it is shown that a network that learns sparse codes of natural scenes succeeds in developing localized, oriented, bandpass receptive fields similar to those in the mammalian striate cortex. Expand
Natural Image Statistics and Eecient Coding
Natural images contain characteristic statistical regularities that set them apart from purely random images. Understanding what these regularities are can enable natural images to be coded moreExpand
Relations between the statistics of natural images and the response properties of cortical cells.
  • D. Field
  • Computer Science, Medicine
  • Journal of the Optical Society of America. A, Optics and image science
  • 1987
TLDR
The results obtained with six natural images suggest that the orientation and the spatial-frequency tuning of mammalian simple cells are well suited for coding the information in such images if the goal of the code is to convert higher-order redundancy into first- order redundancy. Expand
Finding compact and sparse-distributed representations of visual images
Some recent work has investigated the dichotomy between compact coding using dimensionality reduction and sparse-distributed coding in the context of understanding biological information processing.Expand
Efficient Coding of Natural Scenes in the Lateral Geniculate Nucleus: Experimental Test of a Computational Theory
TLDR
The results suggest that the early visual pathway is well adapted for efficient coding of information in the natural visual environment, in agreement with the prediction of the computational theory. Expand
Dynamic Model of Visual Recognition Predicts Neural Response Properties in the Visual Cortex
TLDR
A hierarchical network model of visual recognition that explains experimental observations regarding neural responses in both free viewing and fixating conditions by using a form of the extended Kalman filter as given by the minimum description length (MDL) principle is described. Expand
What Does the Retina Know about Natural Scenes?
TLDR
It is argued that the retinal goal is to transform the visual input as much as possible into a statistically independent basis as the first step in creating a redundancy reduced representation in the cortex, as suggested by Barlow. Expand
...
1
2
3
4
5
...