TAMS: Text-Assisted Multimodal Semantic Segmentation in Remote Sensing

Le résumé fourni par la source

Semantic segmentation in remote sensing is critical for precise land cover classification. Despite significant advancements in current single-modal methods, challenges remain in accurate semantic prediction in intricate scenes with diverse object classes and complex features. To address these issues, we develop a prompt template to generate standardized descriptions of object attributes in remote sensing images using ChatGPT-4o, effectively mitigating the scarcity of textual data in image datasets. Building on this, we propose TAMS-Net, a Text-Assisted Multimodal Semantic Segmentation network that leverages textual modality information to enhance semantic understanding. Specifically, a Channel-Spatial Cross-Attention (CSCA) module and contrastive learning are employed to seamlessly align image and text features for improved multimodal integration. Experimental results on the class-imbalanced LoveDA dataset demonstrate that integrating text-assisted identification markedly improves segmentation performance, underscoring the potential of multimodal approaches in remote sensing tasks.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.