Rethinking the Energy Efficiency of SNNs and ANNs: A Perspective from Neural Network Design

Spiking Neural Networks (SNNs) have shown great potential for energy-efficient computing. However, current SNN designs primarily focus on synaptic operations as the main energy metric, often overlooking the impact of memory access when compared with ANNs. To help researchers design energy-efficient SNNs, we propose a novel analytical methodology aimed at comparing the energy consumption and latency of SNNs and ANNs based on validated low-level details from accelerator prototypes. This work explores how neural network design influences energy consumption and latency, a previously underexplored area, to uncover counterintuitive results that highlight opportunities for optimizing the energy and latency of SNNs.

Based on the results obtained from our methodology, we conclude that (a) SNN architectures have the potential to achieve better energy efficiency with comparable accuracy when handling tasks with Transformer-based systems and Multi-Layer Perceptrons. However, (b) when handling complex neural networks and datasets, the energy efficiency and latency of SNNs are affected by the network topology, timestep, and firing rate. Furthermore, (c) if we take into account pruning techniques, SNNs remain less efficient than traditional ANN systems for CNN architectures. Experiments show that pruned SNNs still consume 1.61 × more energy than pruned ANNs. Consequently, most of the existing SNN algorithms still lag behind ANNs in terms of energy efficiency and latency due to their large number of timesteps, and high firing rate in modern networks. We open-source the methodology and examples at https://github.com/nus-comparch/SNN-vs-ANN to support reproducibility and allow the broader community to use this work.