• Corpus ID: 27199345

Probabilistic Synapses

@inproceedings{Aitchison2014ProbabilisticS,
  title={Probabilistic Synapses},
  author={Laurence Aitchison and Alexandre Pouget and Peter E. Latham},
  year={2014}
}
Learning, especially rapid learning, is critical for survival. However, learning is hard: a large number of synaptic weights must be set based on noisy, often ambiguous, sensory information. In such a high-noise regime, keeping track of probability distributions over weights — not just point estimates — is the optimal strategy. Here we hypothesize that synapses take that optimal strategy: they do not store just the mean weight; they also store their degree of uncertainty — in essence, they put… 

Figures from this paper

Presynaptic stochasticity improves energy efficiency and helps alleviate the stability-plasticity dilemma

It is found that presynaptic stochasticity and plasticity improve energy efficiency and show that the network allocates most energy to a sparse subset of important synapses, thus connecting aPresynaptic notion of importance to a computational role in lifelong learning.

Presynaptic Stochasticity Improves Energy Efficiency and Alleviates the Stability-Plasticity Dilemma

It is found that presynaptic stochasticity and plasticity improve energy efficiency and show that the network allocates most energy to a sparse subset of important synapses, thus connecting a presyaptic notion of importance to a computational role in lifelong learning.

Rapid Bayesian learning in the mammalian olfactory system

Olfactory learning is formulated as a Bayesian optimization process, then map the learning rules into a computational model of the mammalian olfactory circuit, which is capable of odor identification from a small number of observations, while reproducing cellular plasticity commonly observed during development.

Ongoing, rational calibration of reward-driven perceptual biases

It is shown that for an asymmetric-reward perceptual decision-making task, three monkeys produced adaptive biases in response to changes in reward asymmetries and perceptual sensitivity that were consistent with a normative accumulate-to-bound process.

References

SHOWING 1-10 OF 48 REFERENCES

Network Plasticity as Bayesian Inference

It is proposed that inherently stochastic features of synaptic plasticity and spine motility enable cortical networks of neurons to carry out probabilistic inference by sampling from a posterior distribution of network configurations.

Bayesian inference with probabilistic population codes

This work argues that the Poisson-like variability observed in cortex reduces a broad class of Bayesian inference to simple linear combinations of populations of neural activity, and demonstrates that these results hold for arbitrary probability distributions over the stimulus, for tuning curves of arbitrary shape and for realistic neuronal variability.

Simple statistical gradient-following algorithms for connectionist reinforcement learning

This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units that are shown to make weight adjustments in a direction that lies along the gradient of expected reinforcement in both immediate-reinforcement tasks and certain limited forms of delayed-reInforcement tasks, and they do this without explicitly computing gradient estimates.

A Practical Bayesian Framework for Backpropagation Networks

  • D. Mackay
  • Computer Science
    Neural Computation
  • 1992
A quantitative and practical Bayesian framework is described for learning of mappings in feedforward networks that automatically embodies "Occam's razor," penalizing overflexible and overcomplex models.

Probabilistic brains: knowns and unknowns

The challenges that will emerge as researchers start focusing their efforts on real-life computations, with a focus on probabilistic learning, structural learning and approximate inference are discussed.

Unified pre- and postsynaptic long-term plasticity enables reliable and flexible learning

It is shown that spike-timing-dependent plasticity with both pre- and postsynaptic expression develops receptive fields with reduced variability and improved discriminability compared to post Synaptic plasticity alone, which matches recent sensory perception experiments.

Triplets of Spikes in a Model of Spike Timing-Dependent Plasticity

A triplet rule is examined, a rule which considers sets of three spikes and is possible to fit experimental data from visual cortical slices as well as from hippocampal cultures and can be mapped to a Bienenstock–Cooper–Munro learning rule.