• Corpus ID: 27199345

Probabilistic Synapses

@inproceedings{Aitchison2014ProbabilisticS,
  title={Probabilistic Synapses},
  author={Laurence Aitchison and Alexandre Pouget and Peter E. Latham},
  year={2014}
}
Learning, especially rapid learning, is critical for survival. However, learning is hard: a large number of synaptic weights must be set based on noisy, often ambiguous, sensory information. In such a high-noise regime, keeping track of probability distributions over weights — not just point estimates — is the optimal strategy. Here we hypothesize that synapses take that optimal strategy: they do not store just the mean weight; they also store their degree of uncertainty — in essence, they put… 

Figures from this paper

Presynaptic stochasticity improves energy efficiency and helps alleviate the stability-plasticity dilemma
TLDR
It is found that presynaptic stochasticity and plasticity improve energy efficiency and show that the network allocates most energy to a sparse subset of important synapses, thus connecting aPresynaptic notion of importance to a computational role in lifelong learning.
Presynaptic Stochasticity Improves Energy Efficiency and Alleviates the Stability-Plasticity Dilemma
TLDR
It is found that presynaptic stochasticity and plasticity improve energy efficiency and show that the network allocates most energy to a sparse subset of important synapses, thus connecting a presyaptic notion of importance to a computational role in lifelong learning.
Ongoing, rational calibration of reward-driven perceptual biases
TLDR
It is shown that for an asymmetric-reward perceptual decision-making task, three monkeys produced adaptive biases in response to changes in reward asymmetries and perceptual sensitivity that were consistent with a normative accumulate-to-bound process.
Rapid Bayesian learning in the mammalian olfactory system
TLDR
Olfactory learning is formulated as a Bayesian optimization process, then map the learning rules into a computational model of the mammalian olfactory circuit, which is capable of odor identification from a small number of observations, while reproducing cellular plasticity commonly observed during development.

References

SHOWING 1-10 OF 48 REFERENCES
Simple statistical gradient-following algorithms for connectionist reinforcement learning
TLDR
This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units that are shown to make weight adjustments in a direction that lies along the gradient of expected reinforcement in both immediate-reinforcement tasks and certain limited forms of delayed-reInforcement tasks, and they do this without explicitly computing gradient estimates.
A Practical Bayesian Framework for Backpropagation Networks
  • D. Mackay
  • Computer Science
    Neural Computation
  • 1992
TLDR
A quantitative and practical Bayesian framework is described for learning of mappings in feedforward networks that automatically embodies "Occam's razor," penalizing overflexible and overcomplex models.
Probabilistic brains: knowns and unknowns
TLDR
The challenges that will emerge as researchers start focusing their efforts on real-life computations, with a focus on probabilistic learning, structural learning and approximate inference are discussed.
Unified pre- and postsynaptic long-term plasticity enables reliable and flexible learning
TLDR
It is shown that spike-timing-dependent plasticity with both pre- and postsynaptic expression develops receptive fields with reduced variability and improved discriminability compared to post Synaptic plasticity alone, which matches recent sensory perception experiments.
Triplets of Spikes in a Model of Spike Timing-Dependent Plasticity
TLDR
A triplet rule is examined, a rule which considers sets of three spikes and is possible to fit experimental data from visual cortical slices as well as from hippocampal cultures and can be mapped to a Bienenstock–Cooper–Munro learning rule.
Multiplicative Dynamics Underlie the Emergence of the Log-Normal Distribution of Spine Sizes in the Neocortex In Vivo
TLDR
The ongoing dynamics of spine sizes can be captured by a simple phenomenological model that operates at two timescales of days and months, bridging the gap between synaptic dynamics and the stationary distribution of synaptic efficacies.
Interpreting Neural Response Variability as Monte Carlo Sampling of the Posterior
TLDR
This paper proposes an alternative view in which the variability of cortical sensory neurons is related to the uncertainty, about world parameters, which is inherent in the sensory stimulus, and provides simulations suggesting how some aspects of response variability might be understood in this framework.
Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits
TLDR
The local cortical network structure can be viewed as a skeleton of stronger connections in a sea of weaker ones, likely to play an important role in network dynamics and should be investigated further.
...
...