Reducing Energy Consumption for Machine Learning Inference on Edge Devices using C++20 Coroutines

Increasingly, machine learning inference is implemented on relatively low-powered edge devices, where battery life is a key performance criterion. In this work, we demonstrate how C++20 coroutines can be used to reorganise the execution order of an iterative inference task on an edge device. A Prognostic and Health Management (PHM) application receives streams of vibration data as envelope spectra from a wireless sensor network and processes them locally through an array of Support Vector Machines. In our experiments on ARM Cortex A72 & A53 64-bit SoCs, this method can reduce energy consumption for the task by up to 18%, reduce overall energy use by up to 20% and cut execution time by up to 20.5%. Furthermore, peak power levels are reduced by up to 4.5%, which can increase battery lifespan by reducing wear. We demonstrate that the necessary changes to the C++ code are simple, repeatable and generally applicable to iterative inference tasks.