An Adaptive Data-Precision Quantization Method for Deep Neural Networks

Fixed data precision settings cannot readily accommodate differences in numerical distributions across network layers during quantization. To address this issue, we propose an adaptive data-precision quantization method for deep neural networks. We analyze the numerical distributions of features across network layers and use single-precision floating-point (FP32) distributions as references to match precision settings to layer-specific distribution characteristics. Wasserstein distance measures the differences between the distributions obtained under candidate quantization settings and their FP32 references, guiding the adaptive selection of settings for each layer. The method uses the Posit number format to provide a configurable representation space and combines dynamic scaling with range constraints for low-bit mapping. We evaluate the method on YOLO object detection models using the Pascal VOC2012 and COCO2017 datasets. On Pascal VOC2012, it achieves 77.03% mAP@0.5, equivalent to 99.8% of the FP32 baseline result of 77.15%. On COCO2017, it achieves 55.87% mAP@0.5, equivalent to 99.2% of the FP32 baseline. These results indicate that adapting precision settings to layer-specific numerical distributions can preserve object detection performance.