Continuous Sign Language Recognition in Complex Background Based on Attention Mechanism
Published in Journal 69, 2022
Recommended citation: YANG Guangyi, DING Xingyu, GAO Yi, et al. Continuous Sign Language Recognition in Complex Background Based on Attention Mechanism [J]. J Wuhan Univ (Nat Sci Ed), 2023, 69 (01): 1-9. DOI: 10. 14188/j. 1671-8836. 2021. 0350 (Ch) https://www.cnki.com.cn/Article/CJFDTOTAL-WHDY20221128002.htm
In this work, an attention-based 3D convolutional neural network (ACN) is proposed for continuous sign language recognition in complex background. Firstly, the sign language video containing complex background is preprocessed with the background removal module. Then, the spatio-temporal fusion information is extracted by 3D-ResNet (3D residual convolutional neural network) based on spatial attention mechanism. Finally, the long short-term memory (LSTM) network combined with the time attention mechanism is used for sequence learning to obtain the final recognition result. Extensive experiments show that the algorithm performs well on the large-scale Chinese continuous sign language dataset CSL100. The algorithm shows good generalization performance facing different complex background, and the spatio-temporal attention mechanism introduced by the model is effective.
Recommended citation: YANG Guangyi, DING Xingyu, GAO Yi, et al. Continuous Sign Language Recognition in Complex Background Based on Attention Mechanism [J]. J Wuhan Univ (Nat Sci Ed), 2023, 69 (01): 1-9. DOI: 10. 14188/j. 1671-8836. 2021. 0350 (Ch)