The Deferred Event Model for Hardware-Oriented Spiking Neural Networks

Abstract

Real-time modelling of large neural systems places critical demands on the processing system’s dynamic model. With spiking neural networks it is convenient to abstract each spike to a point event. In addition to the representational simplification, the event model confers the ability to defer state updates, if the model does not propagate the effects of the current event instantaneously. Using the SpiNNaker dedicated neural chip multiprocessor as an example system, we develop models for neural dynamics and synaptic learning that delay actual updates until the next input event while performing processing in background between events, using the difference between “electronic time” and “neural time” to achieve real-time performance. The model relaxes both local memory and update scheduling requirements to levels realistic for the hardware. The delayed-event model represents a useful way to recast the real-time updating problem into a question of time to the next event. 1 Real-time neural networks: the update timing challenge Accurately modelling the continuous-time behaviour of large neural networks in real time presents a difficult computational choice: when to update the state? Conventional sequential digital processing usually prohibits real-time updates except on the largest, fastest computers, but dedicated parallel neural network hardware needs some time model. Broadly, two different architectures have become popular. One, the neuromorphic approach, e.g. [1], circumvents the state update problem altogether by using continuous-time analogue circuitry at the price of hardwiring the model of computation into the system. The other, the parallel neurocomputer approach, e.g. [2] retains general-purpose digital processing but attempts to use a neural-specific, multiprocessor architecture with some loss of speed and accuracy of state update. An ideal approach would combine the process abstraction capabilities of digital devices with the asynchronous-time model of analogue devices. SpiNNaker, a chip using event-driven asynchronous digital circuitry timed by its input rather than by an instruction clock, makes ? Corresponding author this feasible for spiking neural networks having real axonal and synaptic delays, by abstracting a spike to an event. During the time between events, it is in effect “outside time”, and can perform necessary background processing without affecting the model update. Having previously demonstrated a basic neural implementation on SpiNNaker, here we present a general method to maintain accurate temporal dynamics by deferring pending updates until the next input event. The method realises a solution to the update scheduling problem that suggests the possibility of practical large-scale, real-time neural simulation. 2 Deferred update dynamics Real dynamic properties of neural networks allow us to relax the timing requirements dramatically. Most models, whether using temporal [3] or rate coding [4], [5], assume that the spike timing irrespective of shape determines the information coding. A typical active neuron fires at ∼10-20 Hz up to a maximum of ∼100 Hz [6]. Only a small number of a given population of neurons, typically ∼1-0.1%, will be active at any time, with 10% a reasonable upper limit. For a “typical” neuron containing 5000 dendritic connections with 1% activity, spiking at 10 Hz, we therefore expect an average input rate of 500 events (input spikes) per second, requiring an update rate of only 2ms. A worst-case situation: 100k inputs, 10% activity, 100 Hz firing rate, would require 1μs update rate. These are leisurely rates for typical digital processors running at hundreds of MHz. With real neurons having axonal delays, usually of the order of 1-20 ms, if the processor can propagate the required updates following an event in less time than the interval between events that affect a given output, it can use that time difference to defer the event processing until the occurrence of the next event. Simple time-domain multiplexing [7] is an established approach, but in addition, the processor can make use of the “dead space” to wait on contingent input events that may likewise affect the output. By performing temporal reordering of input events, it can ignore the fine-grained order of inputs. In Fig. 1, axons have finite delays. The processor schedules the updates, and the neuron can respond to further potential future input events with nonzero axonal delays, reordering them as necessary so that the neuron exhibits the proper dynamic behaviour even with nondeterministic input order. Delayed event processing thus allows more than time multiplexing; it decouples the temporal dynamics of the processing system from the temporal dynamics of the model. 3 Implementation of a neuron and synapse on SpiNNaker 3.1 The SpiNNaker hardware system We have developed a universal spiking neural network chip multiprocessor, SpiNNaker (fig. 2), to support large scale real-time simulation [8]. 20 ARM968 processor cores with dedicated 64KB local memory implement neurons and their associated dendrites. A large off-chip SDRAM memory device stores synaptic data, I 2 Activity I 1 Activity Synaptic Weight Update Neuron State Update δ axon = 13 ms

DOI: 10.1007/978-3-642-03040-6_128

Extracted Key Phrases

5 Figures and Tables

Cite this paper

@inproceedings{Rast2008TheDE, title={The Deferred Event Model for Hardware-Oriented Spiking Neural Networks}, author={Alexander D. Rast and Xin Jin and Muhammad Mukaram Khan and Stephen B. Furber}, booktitle={ICONIP}, year={2008} }