Multimodal Instance Segmentation of Building Façade Damage with Frequency-Decoupled Feature Fusion

Le résumé fourni par la source

Building façade damage segmentation remains challenging because cracks, peeling, hollow-like areas, stains, and erosion often exhibit weak contrast, irregular boundaries, discontinuous structures, and substantial scale variations under complex surface textures and illumination conditions. To address these challenges, this study proposes MMDF-Net, a multimodal instance segmentation network for paired RGB–IR façade damage images. The network adopts dual RGB and infrared branches and performs multi-scale cross-modal interaction across P2–P5 levels. A Frequency-Decoupled Cross-Modal Bridge is designed to separately model low-frequency material responses and high-frequency local damage details, thereby enabling gated bidirectional exchange between the two modalities. A Crack Topology-Aware Encoder further enhances the continuity of thin, curved, branched, and locally discontinuous cracks through directional strip convolution and bending-aware modeling. In addition, a Multi-Scale Damage Decoding Pyramid integrates high-resolution boundary details, mid-level structural cues, and high-level semantic context to support stable mask prediction across different damage scales. Experiments were conducted on a custom-built RGB–IR building façade damage dataset containing 1500 paired image samples across five damage categories. MMDF-Net achieved 82.1% mask precision, 78.6% mask recall, 72.3% mask mAP50, and 65.6% mask mAP50–95, with 7.8 M parameters and 23.5 GFLOPs, outperforming representative RGB and RGB–IR segmentation baselines. These results indicate that task-oriented multimodal feature organization can improve façade damage segmentation without relying on excessive model expansion.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.