Multi-Modal Emotion Temporal Environment Facial Classification to Help Children with ASD

Le résumé fourni par la source

Children with Autism Spectrum Disorder (ASD) frequently encounter difficulties in recognizing and interpreting emotions, primarily when their surroundings affect emotional perception. Traditional emotion recognition systems predominantly emphasize facial features, neglecting the influence of environmental cues on emotional comprehension [1]–[2]. This paper introduces a multi-modal Environment Emotion Recognition (EER) framework that combines environmental and facial emotion cues to deliver a more environmentally thorough emotion classification. The suggested methodology utilizes ResNet-18 for environmental and emotional classification, and InceptionResNetV1 with MTCNN-based facial detection for facial emotion recognition. A weighted fusion layer integrates the outputs of both models, emphasizing facial emotion cues upon face detection. The system is evaluated through accuracy, precision, recall, F1-score, and confusion matrix analysis. Experimental results indicate that the fusion approach improves classification performance relative to independent models, especially in scenarios where environmental context is inadequate. The proposed framework presents potential applications in assistive technologies for ASD, facilitating more nuanced and reliable emotion recognition in real-world environments.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.