Excavator Activity Recognition Using Gramian Angular Field (GAF) Encoding of Multi-Sensor Operational Data and Deep Learning

Efficient monitoring and recognition of excavator operational activities are critical for improving productivity, safety, and intelligent construction management. Traditional vision-based activity recognition systems remain sensitive to challenging construction site conditions, including occlusion, dust, variable illumination, and limited visibility. This paper presents an excavator activity recognition framework that, for the first time, applies Gramian Angular Field (GAF) encoding to multi-sensor excavator operational data, transforming them into spatial image representations and enabling deep convolutional neural networks (CNNs) to extract discriminative features. The proposed approach integrates synchronized excavator sensor signals including bucket positional coordinates, body orientation, fuel consumption, engine RPM, and joint angles to characterize operational behavior during four representative activities: digging, dumping, idle, and levelling. Unlike conventional sensor-based methods that directly process sequential time-series data, our GAF-based framework transforms operational signals into structured spatial representations that preserve temporal correlations while enabling effective feature learning through image-based deep learning architectures. Experimental results demonstrate that the proposed GAF-CNN-LSTM framework achieves 8.24 percentage points higher classification accuracy compared with LSTM networks, and 6.24 percentage points higher than 1D CNN baselines trained on the same sensor data. The method effectively captures discriminative operational signatures across multiple excavator activities in the collected dataset. This work bridges the limitations of both vision-based and traditional sensor-based approaches, providing a promising framework for excavator activity monitoring in construction and fleet management contexts, pending further validation under operational deployment conditions.