Continuous Chinese Sign Language Recognition Using Millimeter-Wave Radar With DeepSeek Semantic Enhancement

Le résumé fourni par la source

Chinese sign language recognition plays a critical role in supporting communication for the hearing-impaired. Recently, Millimeter-wave radar has attracted growing attention in gesture recognition due to its advantages of non-contact sensing and privacy preservation. By extracting micro-Doppler signatures, Millimeter-wave radar enables gesture recognition without reliance on visual or wearable inputs. However, most existing radar-based methods focus on isolated gestures and lack the ability to segment gesture boundaries, making them unsuitable for continuous Chinese sign language recognition. Moreover, their outputs are often limited to disjointed word-level predictions, hindering semantic fluency. To address these limitations, we propose a novel radar-based continuous Chinese sign language recognition framework that uses time-frequency singularity analysis to segment gestures from radar echo signals. In response to the challenges of low spatial resolution and high noise in radar-based gesture recognition, we propose a hybrid model combining Swin Transformer and BiLSTM to capture both spatial and temporal features, enabling robust performance in continuous sign language recognition. To enhance sentence-level fluency and reduce confusion between similar gestures, we incorporate the large-scale language model DeepSeek for semantic correction and natural language refinement. Experimental results demonstrate that our method achieves 96.8% average frame-level classification accuracy and reduces word error rate to 5.6%, significantly outperforms traditional methods. The results demonstrate the superiority and effectiveness of the proposed approach.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.