Spiking Neural Networks (SNNs) are biologically inspired models that mimic the temporal information processing of biological neurons through discrete spikes (Vreeken, 2003). Unlike the first two generations of artificial neural networks, SNNs offer encode information in spike timing rather than just average firing rates (Taherkhani et al., 2019; Ponulak & Kasinski, 2011). In this guide we will explore the biological foundations of spiking neural networks from the biophysics of a bioligical neuron to the metastable behaviour of the human brain.
Biological Brains
Recent studies using advanced cell-counting techniques, such as the isotropic fractionator method, estimate that the average adult human brain contains approximately 86 billion neurons (Herculano-Houzel, 2009). In addition to this vast number of neurons, the human brain also contains approximately 85 billion non-neuronal cells, such as glial cells, which provide support and protection for neurons (Herculano-Houzel, 2012).
In terms of mass the human brain weighs around 1.5kg, with 82% of this mass belonging to the outer part of the brain known as the cerebral cortex. It is interesting to note that neuron count is only one part of the picture however as the cerebral cortex has only around 19% of the brain's neurons (Herculano-Houzel, 2012).
The key missing piece to the brain's complexity lies in it's interconnectedness with each of these 86 billion neurons typically connected to on average more than 10,000 post synaptic neurons (Wulfram Gerstner, 2002). This complexity is far from homogeneous however, with the brain having specialised regions that have different functions and evolutionary histories. The most complex and relatively unique to developed brains is the previously mentioned Cerebral Cortex where neurons have on arvage around 50,000 postsynaptic neurons.
Another important region of the brain is the cerebellum which is responsible for motor control and coordination. The cerebellum contains around 69 billion neurons, which is more than the cerebral cortex, but is much smaller in terms of mass (Herculano-Houzel, 2009). The cerebellum is also unique in that it contains a large number of neurons called Purkinje cells which are some of the largest neurons in the brain.
Another remarkable property of the brain is both it's relative energy consumpton and it's efficiency. The human brain consumes around 25% of total body's energetic cost, which is incredibly high, suggesting evolution has fought hard against the trade-off of intelligence and energetic costs. On the other hand it is remarkable low compared to modern computing hardware, at around 20W the human brain uses far less energy than large language models such as GPT-3, which require over 1000 MW-hours to train (around 500 million human brain hours) (Zador, 2022).
Biological Neurons
