Learning in Spiking Neural Networks by Reinforcement of Stochastic Synaptic Transmission

@article{Seung2003LearningIS,
  title={Learning in Spiking Neural Networks by Reinforcement of Stochastic Synaptic Transmission},
  author={H. Sebastian Seung},
  journal={Neuron},
  year={2003},
  volume={40},
  pages={1063-1073}
}
  • H. Seung
  • Published 2003
  • Psychology, Medicine
  • Neuron
It is well-known that chemical synaptic transmission is an unreliable process, but the function of such unreliability remains unclear. Here I consider the hypothesis that the randomness of synaptic transmission is harnessed by the brain for learning, in analogy to the way that genetic mutation is utilized by Darwinian evolution. This is possible if synapses are "hedonistic," responding to a global reward signal by increasing their probabilities of vesicle release or failure, depending on which… Expand
Reinforcement Learning in Spiking Neural Networks with Stochastic and Deterministic Synapses
TLDR
Experimental results show that the coordination of diverse synaptic plasticities can make the RL model learn in a rapid and stable form. Expand
Spike-Based Reinforcement Learning in Continuous State and Action Space: When Policy Gradient Methods Fail
TLDR
The theoretical approach shows how learning new behaviors can be linked to reward-modulated plasticity at the level of single synapses and makes predictions about the voltage and spike-timing dependence of synaptic plasticity and the influence of neuromodulators such as dopamine. Expand
A reinforcement learning algorithm for spiking neural networks
  • R. Florian
  • Computer Science
  • Seventh International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC'05)
  • 2005
TLDR
A new reinforcement learning mechanism for spiking neural networks that recovers a form of neural plasticity experimentally observed in animals, combining spike-timing-dependent synaptic changes with non-associative synaptic changes of the opposite sign determined by presynaptic spikes. Expand
Gradient estimation in dendritic reinforcement learning
TLDR
It is suggested that the availability of nonlocal feedback for learning is a key advantage of complex neurons over networks of simple point neurons, which have previously been found to be largely equivalent with regard to computational capability. Expand
A Reinforcement Learning Framework for Spiking Networks with Dynamic Synapses
TLDR
Generality, along with the reported results, supports adopting the introduced approach to benefit from the biologically plausible synaptic models in a wide range of intuitive signal processing. Expand
A Spiking Neural Network Model of an Actor-Critic Learning Agent
TLDR
A spiking neural network model is presented that implements actor-critic temporal-difference learning by combining local plasticity rules with a global reward signal and it is demonstrated that the network learns with a similar speed to its discrete time counterpart and attains the same equilibrium performance. Expand
Training spiking neural networks using reinforcement learning
TLDR
This project proposes biologically-plausible alternatives to backpropagation to facilitate the training of spiking neural networks and investigates the candidacy of reinforcement learning (RL) rules in solving the spatial and temporal credit assignment problems to enable decision-making in complex tasks. Expand
Reinforcement learning with modulated spike timing dependent synaptic plasticity.
TLDR
This model offers a novel and biologically plausible implementation of reinforcement learning that is capable of training a neural population to produce a very wide range of possible mappings between synaptic input and spiking output. Expand
Stimulus sampling as an exploration mechanism for fast reinforcement learning
TLDR
It is shown that there is in fact no need to rely on intrinsic noise and that ongoing synaptic plasticity triggered by the naturally occurring online sampling of a stimulus out of an entire stimulus set produces enough fluctuations in the synaptic efficacies for successful learning. Expand
A model of operant learning based on chaotically varying synaptic strength
TLDR
Simulations show that the proposed new hypothesis for operant learning at the single neuron level based on spontaneous fluctuations of synaptic strength caused by receptor dynamics can support operantlearning in a feed-forward neural circuit, a recurrent neural circuits, and a spiking neural circuit controlling an agent learning in a dynamic reward and punishment situation. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 80 REFERENCES
Neural Networks with Dynamic Synapses
TLDR
A unified phenomenological model is proposed that allows computation of the postsynaptic current generated by both types of synapses when driven by an arbitrary pattern of action potential activity in a presynaptic population and allows for derivation of mean-field equations, which govern the activity of large, interconnected networks. Expand
Coding of temporal information by activity-dependent synapses.
TLDR
This work quantitatively analyze the information about previous interspike intervals, contained in single responses of dynamic synapses, using methods from information theory applied to experimentally based deterministic and probabilistic phenomenological models of depressing and facilitating synapses. Expand
Spike-Based Synaptic Plasticity and the Emergence of Direction Selective Simple Cells: Simulation Results
TLDR
This work shows that the spatial displacement and the appropriate degree of synaptic depression required for DS can develop within an unbiased input scenario by means of temporally asymmetric spike-timing dependent plasticity (STDP) which modifies both the synaptic strength and the degree ofaptic depression. Expand
Redistribution of synaptic efficacy between neocortical pyramidal neurons
TLDR
It is shown that an increase in the synaptic response, induced by pairing action-potential activity in pre- and postsynaptic neurons, was only observed when synaptic input occurred at low frequencies, and arises because of a redistribution of the available synaptic efficacy. Expand
A Synaptically Controlled, Associative Signal for Hebbian Plasticity in Hippocampal Neurons
TLDR
Action potentials provide a synaptically controlled, associative signal to the dendrites for Hebbian modifications of synaptic strength and induced a robust LTP in CA1 neurons. Expand
Chaotic Balanced State in a Model of Cortical Circuits
TLDR
The chaotic nature of the balanced state of this network model is revealed by showing that the evolution of the microscopic state of the network is extremely sensitive to small deviations in its initial conditions. Expand
Simple statistical gradient-following algorithms for connectionist reinforcement learning
TLDR
This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units that are shown to make weight adjustments in a direction that lies along the gradient of expected reinforcement in both immediate-reinforcement tasks and certain limited forms of delayed-reInforcement tasks, and they do this without explicitly computing gradient estimates. Expand
Changes in reliability of synaptic function as a mechanism for plasticity
TLDR
It is found that LTP increases synaptic reliability, and LTD decreases it, both without a change in the size of those postsynaptic currents that do occur, thus LTD is a functional inverse of LTP. Expand
A more biologically plausible learning rule for neural networks.
TLDR
A more biologically plausible learning rule is described, using reinforcement learning, which is applied to the problem of how area 7a in the posterior parietal cortex of monkeys might represent visual space in head-centered coordinates and shows that a neural network does not require back propagation to acquire biologically interesting properties. Expand
Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs
TLDR
In dual whole-cell voltage recordings from pyramidal neurons, the coincidence of post Synaptic action potentials and unitary excitatory postsynaptic potentials was found to induce changes in EPSPs. Expand
...
1
2
3
4
5
...