Data glove sign language recognition based on expanded WDCNN fusioned with GRU
Donghui Wu, Jiahui Yang, Jinfeng Wang, Guozhi Liu, Bin Jiang, Yanghai Gui, Hai Huang, Xican Zheng, Xiaomeng Jiang, Jianyun YePurpose
This study aims to address the issue of large intra-class variation in isolated sign language words caused by individual differences in hand morphology, motion amplitude and different habits among users, which affects the accurate recognition of sign language.
Design/methodology/approach
To address the scarcity of specialized public data sets, this paper proposes a heterogeneous sensor-integrated data glove, which reduces data throughput and enhances the overall stability and efficiency of data transmission. To address the significant intra-class variation in isolated sign language words, a novel deep learning framework named EWCGR is proposed in this paper. The first layer of the wide convolutional kernel deep convolutional neural network (WDCNN) is used to extract the local features of the signal and suppress the high-frequency noise. The Gated Recurrent Units (GRU) is then used to obtain global time series features of the signal.
Findings
The proposed EWCGR model achieves a recognition accuracy of 98.26%, outperforming CLT-net, CNN-GRU and CLA-net by 2.89%, 1.53% and 0.27%, respectively.
Originality/value
The proposed wearable sign language device and the EWCGR model remarkably enhances robustness to individual user variations by jointly extracting multi-scale spatiotemporal features.