GIMMNet: Geometry-Aware Interactive Multi-Modal Network for Semantic Segmentation of High-Resolution Remote Sensing Imagery

Le résumé fourni par la source

Remote sensing semantic segmentation holds significant application value in urban planning, environmental monitoring, and related fields. In recent years, multimodal approaches that fuse optical imagery with normalized Digital Surface Models (nDSM) have attracted widespread attention due to their superior performance. However, existing methods typically treat nDSM merely as an additional input channel, failing to effectively exploit its inherent 3D geometric priors, which limits segmentation accuracy in complex urban scenes. To address this issue, we propose a Geometry-aware Interactive Multi-Modal Network (GIMMNet), which explicitly models the geometric structure embedded in nDSM to guide the spatial distribution of semantic categories. Specifically, we first design a Geometric Position Prior Module (GPPM) to construct 3D coordinates for each pixel based on nDSM and extract intrinsic geometric priors. Next, a Geometry-Guided Disentangled Fusion Module (GDFM) dynamically adjusts fusion weights according to the differential responses of each modality to the geometric priors, enabling adaptive multimodal feature integration. Finally, during decoding, a Geometry-Attentive Context Module (GACM) explicitly captures the dependencies between land-cover categories and geometric structures, enhancing the model’s spatial awareness and semantic recovery capability. Experimental results on two public remote sensing datasets—Vaihingen and Potsdam—show that the proposed GIMMNet outperforms existing mainstream methods in segmentation performance, demonstrating that enhancing the model’s geometric perception capability effectively improves semantic segmentation accuracy. Notably, our method achieves an mIoU of 85.2% on the Potsdam dataset, surpassing the second-best multimodal approach, PACSCNet, by 2.3%.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.