Sparse coding with an overcomplete basis set: A strategy employed by V1?

  title={Sparse coding with an overcomplete basis set: A strategy employed by V1?},
  author={Bruno A. Olshausen and David J. Field},
  journal={Vision Research},

Finding the optimal sparse, overcomplete model for natural images by model selection

The goal of this work is to investigate the claim that sparse, overcomplete codes might lend some computational advantage in the processing of visual information.

Are sparse-coding simple cell receptive field models physiologically plausible?

  • P. Watters
  • Biology
    Journal of integrative neuroscience
  • 2006
Across all image themes, basis function sizes, number of basis functions, sparseness factors and learning rates, the spatial-frequency tuning did not closely resemble that of primate area 17 -- the model results more closely resembled the unclassified cat neurones of area 19 with a single exception, and not area 17 as predicted.

Sparse coding models predict a spectral bias in the development of primary visual cortex (V1) receptive fields

This work trains an overcomplete sparse coding model (Sparsenet) on natural images and finds that there is indeed order in the development of its basis functions, with basis functions tuned to lower spatial frequencies emerging earlier and higher spatial frequency basis functions emerging later.

Learning Overcomplete Representations

It is shown that overcomplete bases can yield a better approximation of the underlying statistical distribution of the data and can thus lead to greater coding efficiency and provide a method for Bayesian reconstruction of signals in the presence of noise and for blind source separation when there are more sources than mixtures.

On the Choice of a Sparse Prior

The constraint of unit variance of neuronal activity is included into the objective functions of the generative model, used in most studies, and it is shown that the effective objective functions are largely dominated by the constraint, and are therefore very similar.

Sparse Coding of Natural Images Using an Overcomplete Set of Limited Capacity Units

A model that best balances redundancy reduction and redundant representation is proposed, which accounts for the localized and oriented structure of simple cells, but it also predicts a different organization for the population.

New Evidences for Sparse Coding Strategy Employed in Visual Neurons: from the Image Processing and Nonlinear Approximation Viewpoint

It is shown that several newly proposed systems in the area of image processing and nonlinear approximation provide new evidences for the ‘sparse coding’ strategy along a contrary line.

Explicit Object Representation by Sparse Neural Codes

This thesis discusses the issue of how best to quantify sparseness, particularly in very sparse systems where biases are significant, and shows results obtained by applying an existing model of sparse coding both to unsupervised category discovery in images and to differentiation between images of different individuals.

Efficient Sparse Coding in Early Sensory Processing: Lessons from Signal Recovery

It is argued that higher level overcompleteness becomes computationally tractable by imposing sparsity on synaptic activity and it is shown that such structural sparsity can be facilitated by statistics based decomposition of the stimuli into typical and atypical parts prior to sparse coding.

Development of localized oriented receptive fields by learning a translation-invariant code for natural images.

It is shown that a strategy for transformation-invariant coding of images based on a first-order Taylor series expansion of an image also causes localized, oriented receptive fields to be learned from natural image inputs.



Emergence of simple-cell receptive field properties by learning a sparse code for natural images

It is shown that a learning algorithm that attempts to find sparse linear codes for natural scenes will develop a complete family of localized, oriented, bandpass receptive fields, similar to those found in the primary visual cortex.

What Is the Goal of Sensory Coding?

  • D. Field
  • Computer Science
    Neural Computation
  • 1994
It is proposed that compact coding schemes are insufficient to account for the receptive field properties of cells in the mammalian visual pathway and suggested that natural scenes, to a first approximation, can be considered as a sum of self-similar local functions (the inverse of a wavelet).

Entropy reduction and decorrelation in visual coding by oriented neural receptive fields

  • J. Daugman
  • Computer Science, Biology
    IEEE Transactions on Biomedical Engineering
  • 1989
The present image coding simulations, based on quantitative neurobiological data about the code primitives, provide measures of the bit-rate efficiency of such oriented, quadrature, neural codes.

Natural Image Statistics and Eecient Coding

It is suggested that a good objective for an eecient coding of natural Scenes is to maximize the sparseness of the representation, and it is shown that a network that learns sparse codes of natural scenes succeeds in developing localized, oriented, bandpass receptive elds similar to those in the primate striate cortex.

Relations between the statistics of natural images and the response properties of cortical cells.

  • D. Field
  • Computer Science
    Journal of the Optical Society of America. A, Optics and image science
  • 1987
The results obtained with six natural images suggest that the orientation and the spatial-frequency tuning of mammalian simple cells are well suited for coding the information in such images if the goal of the code is to convert higher-order redundancy into first- order redundancy.

Finding compact and sparse-distributed representations of visual images

An artificial neural network which self-organizes on the basis of simple Hebbian learning and negative feedback of activation is introduced and it is shown that it is capable both of forming compact codings of data distributions and of identifying filters most sensitive to sparse-distributed codes.

Efficient Coding of Natural Scenes in the Lateral Geniculate Nucleus: Experimental Test of a Computational Theory

The results suggest that the early visual pathway is well adapted for efficient coding of information in the natural visual environment, in agreement with the prediction of the computational theory.

Dynamic Model of Visual Recognition Predicts Neural Response Properties in the Visual Cortex

A hierarchical network model of visual recognition that explains experimental observations regarding neural responses in both free viewing and fixating conditions by using a form of the extended Kalman filter as given by the minimum description length (MDL) principle is described.

What Does the Retina Know about Natural Scenes?

It is argued that the retinal goal is to transform the visual input as much as possible into a statistically independent basis as the first step in creating a redundancy reduced representation in the cortex, as suggested by Barlow.

A Model of the Spatial-Frequency Organization in Primate Striate Cortex

Here, a model of the spatial frequency organization that would be required in primate V1 in order to form a complete representation of spatial information provided by the optic nerve is presented.