Sparse coding of sensory inputs

@article{Olshausen2004SparseCO,
  title={Sparse coding of sensory inputs},
  author={Bruno A. Olshausen and David J. Field},
  journal={Current Opinion in Neurobiology},
  year={2004},
  volume={14},
  pages={481-487}
}
  • B. Olshausen, D. Field
  • Published 1 August 2004
  • Computer Science, Biology
  • Current Opinion in Neurobiology

Figures from this paper

2 Sparse Coding in the Neocortex

TLDR
It is argued that larger brains have evolved strategies that offer a useful solution to the problem of representing natural data because such a scheme allows the system to take advantage of the sparse structure of the sensory environment.

Sparse Coding in the Neocortex

A Simple Connectivity Scheme for Sparse Coding in an Olfactory System

TLDR
Analysis of the statistics of connectivity in an olfactory network that contributes to the generation of sparse codes indicates that such connectivity indeed maximizes the difference between input vectors to KCs and helps to explain their high specificity.

Questioning the role of sparse coding in the brain

Synaptic Learning Rules and Sparse Coding in a Model Sensory System

TLDR
Activity-dependent plasticity drove the observed specificity, reliability, and expected persistence of odor representations, suggesting a role for plasticity in information processing and making a testable prediction about synaptic plasticity at AL-MB synapses.

Neurally plausible sparse coding via local competition and thresholding

TLDR
A class of neurally plausible locally competitive algorithms that correspond to a collection of interesting sparse coding principles that attempt to minimize a weighted combination of MSE and a coefficient cost function are presented.

Brain at work: time, sparseness and superposition principles.

TLDR
It is shown that synchronization of spikes is a fast and efficient mode to encode the representation of objects based on feature bindings and presents an interesting venue in probing brain encoding mechanisms as it allows the functional establishment of multi-layered and time-conditioned neuronal networks or multislice networks.

Efficient computation via sparse coding in electrosensory neural networks

The neuronal encoding of information in the brain

Efficient transformation of an auditory population code in a small sensory system

TLDR
It is concluded that optimal coding principles do apply to the early auditory system of the grasshopper, despite its size constraints, and the inputs are not encoded in a systematic, map-like fashion as in many larger sensory systems.
...

References

SHOWING 1-10 OF 70 REFERENCES

Sparse coding in the primate cortex

TLDR
There are now strong theoretical reasons and experimental evidence suggesting that the brain adopts a compromise between these extremes which is often referred to as sparse coding.

What Is the Goal of Sensory Coding?

  • D. Field
  • Computer Science
    Neural Computation
  • 1994
TLDR
It is proposed that compact coding schemes are insufficient to account for the receptive field properties of cells in the mammalian visual pathway and suggested that natural scenes, to a first approximation, can be considered as a sum of self-similar local functions (the inverse of a wavelet).

Natural image statistics and neural representation.

TLDR
It has long been assumed that sensory neurons are adapted to the statistical properties of the signals to which they are exposed, but recent developments in statistical modeling have enabled researchers to study more sophisticated statistical models for visual images, to validate these models empirically against large sets of data, and to begin experimentally testing the efficient coding hypothesis.

Sparse coding and decorrelation in primary visual cortex during natural vision.

Theoretical studies suggest that primary visual cortex (area V1) uses a sparse code to efficiently represent natural scenes. This issue was investigated by recording from V1 neurons in awake behaving

Internal representations for associative memory

TLDR
A class of feed forward neural network models for associative content addressable memory (ACAM) which utilize sparse internal representations for stored data and study two classes of internal label representations: the unary representation and various sparse, distributed representations.

A theory of maximizing sensory information

A theory is developed on the assumption that early sensory processing aims at maximizing the information rate in the channels connecting the sensory system to more central parts of the brain, where

Sparse population coding of faces in the inferotemporal cortex.

TLDR
Population analysis of cells in the temporal cortex of monkeys showed that information is carried at the level of the population and that representations of complex stimuli in the higher visual areas may take the form of a sparse population code.

From synchrony to sparseness

How do visual neurons respond in the real world?

...