DOI: 10.3390/app16167924 ISSN: 2076-3417

A Staged PEFT Framework for Industrial Pointer-Gauge Reading with Multimodal Large Language Models

Jian Wang, Xingyang Li, Wei Shen

Pointer gauges remain widely deployed in industrial environments because they are inexpensive, resistant to electromagnetic interference, and readable from a distance. However, automatic reading remains difficult in practice because reliable prediction requires jointly interpreting pointer geometry, scale layout, and unit-type consistency under challenging conditions such as glare, scratches, blur, and oblique viewpoints. Although multimodal large language models (MLLMs) offer a promising unified interface for visual understanding and structured output, their direct application to gauge reading is limited by weak geometric grounding, unit confusion, and unstable numeric generation. Rather than claiming a new model architecture or a new reading algorithm, this work frames the contribution as a practical adaptation and evaluation framework for applying existing MLLM and PEFT components to structured industrial gauge reading. Our framework combines three components: (i) a dedicated dataset and VQA-style annotation protocol covering multiple noise types and intensity levels; (ii) a unified screening pipeline for selecting a suitable MLLM backbone under zero-shot settings; and (iii) parameter-efficient adaptation of the selected model with Projector-LoRA, together with training and decoding mechanisms designed to improve reading robustness and output consistency. On our test set, the fine-tuned Granite-Vision 3.2 model achieves 99.9% type accuracy, 43.77% reading accuracy, and 43.60% joint accuracy. It obtains an MAE of 3.98 over valid numerical predictions, a parsing coverage of 98.65%, and an all-sample penalized normalized MAE of 0.052. These results substantially outperform the evaluated zero-shot MLLM baselines in structured prediction accuracy, although the lightweight CNN baseline remains slightly better in all-sample normalized numerical error.These results should be interpreted as evidence of promise and measurable improvement over untuned MLLMs, not as evidence that the system is already sufficient for safety-critical or fully autonomous industrial deployment. More broadly, the proposed framework offers a practical, traceable path for adapting large multimodal models to visual measurement tasks that require structured numerical outputs.

More from our Archive