Spiking Neural Network Benchmarks

Approaches

Deciding how a spiking network should learn is harder than for a conventional artificial network, for one fundamental reason: the spike is a threshold event whose derivative is zero almost everywhere and undefined at the threshold itself. Ordinary gradient descent has no slope to descend, so backpropagation cannot be applied directly. Learning is further complicated by time: a spiking neuron is a stateful dynamical system whose output depends on its own history, so error must be assigned not only across layers but across the timesteps of a simulation. The approaches below reflect the different answers the field has given to this credit-assignment problem, and they are the same categories used to group the results elsewhere on this site.

Supervised Learning

Supervised methods use a labelled target to drive weight changes. The most widely used route is ANN-to-SNN conversion: a conventional network is trained normally with backpropagation, and its rectified-linear activations are then mapped onto the firing rates of spiking neurons. Rueckauer et al. (2017) gave spiking equivalents of the common building blocks (max-pooling, softmax, batch normalisation) and converted networks such as VGG-16, while Sengupta et al. (2019) scaled the technique to deep VGG and residual networks on CIFAR-10 and ImageNet. A separate lineage trains directly on precise spike times: SpikeProp (Bohte et al., 2002) derived an error-backpropagation rule for temporally coded networks, and the Tempotron (Gütig & Sompolinsky, 2006) trains a neuron to fire or stay silent in response to spatiotemporal patterns, exploiting information beyond the mean firing rate.

Unsupervised Learning

Unsupervised methods learn structure from unlabelled input using local, biologically plausible rules, chiefly spike-timing-dependent plasticity (STDP), under which a synapse strengthens when the presynaptic spike precedes the postsynaptic one and weakens when the order is reversed. Combined with lateral inhibition and adaptive thresholds, STDP produces competitive, winner-take-all dynamics in which neurons specialise on distinct input features. The canonical demonstration is Diehl & Cook (2015), whose two-layer network reached about 95% on MNIST with 6,400 excitatory neurons using no labels during training; labels were assigned to neurons only afterward. Such methods are naturally suited to neuromorphic hardware, but their accuracy trails supervised training and they scale poorly to deep networks.

Reinforcement Learning

Reinforcement approaches learn from a scalar reward rather than an explicit target. The characteristic mechanism is reward-modulated STDP, a three-factor learning rule in which the Hebbian coincidence of pre- and postsynaptic spikes sets a synapse-specific eligibility trace, but the weight actually changes only when a third, global factor (typically a neuromodulator such as dopamine) signals reward. The eligibility trace bridges the delay between an action and its consequence, allowing credit to reach the responsible synapse. Frémaux & Gerstner (2016) set out the theory of these three-factor rules, which also connect naturally to modern policy-gradient and actor-critic formulations.

Hybrid Learning

Hybrid methods combine the strengths of the categories above. A common recipe uses unsupervised STDP to learn feature detectors in the early layers and then trains a supervised classifier on top, marrying biological plausibility with the accuracy that labelled data affords. Another initialises a network by ANN-to-SNN conversion and then fine-tunes it with surrogate gradients to recover accuracy at low latency. "Hybrid" is best understood as an umbrella for such combinations rather than a single technique; the strongest overview is the tutorial-survey of Eshraghian et al. (2023).

Spike-Based Backpropagation Variants

This is the dominant frontier method, and it confronts the non-differentiable spike head-on. The idea of the surrogate gradient is to keep the hard threshold in the forward pass but substitute a smooth, well-behaved function (a fast sigmoid, say) for its derivative in the backward pass (Neftci, Mostafa & Zenke, 2019). Training then proceeds by backpropagation-through-time over the network unrolled across its timesteps, exactly as for a recurrent network with a spiking nonlinearity. Spatio-Temporal Backpropagation (Wu et al., 2018) propagates error jointly through the spatial and temporal domains, while SLAYER (Shrestha & Orchard, 2018) introduces a temporal credit-assignment scheme and learns axonal delays alongside weights. These methods underpin the modern SNN training toolkits.

Reservoir Computing and Liquid State Machines

Reservoir computing avoids training the recurrent weights altogether. A liquid state machine, introduced by Maass, Natschläger & Markram (2002), is a fixed, randomly connected recurrent pool of spiking neurons whose high-dimensional transient dynamics act as a fading memory of recent input; only a simple, usually linear, readout is trained. Because the expensive recurrent connections are never learned, training is cheap and stable, and the fixed reservoir maps cleanly onto neuromorphic hardware. It is the spiking counterpart of the echo-state networks in the wider reservoir-computing family.