GRRSIS: Generalized Referring Remote Sensing Image Segmentation

Le résumé fourni par la source

Referring Remote Sensing Image Segmentation (RRSIS) is a challenging task that involves segmenting target instances within a top-view image guided by a natural language expression. Existing classic RRSIS methods commonly support target expressions only, i.e., the target described by the expression is present in the image. No-target expressions are excluded. Under this constraint, the model may face significant challenges. For instance, a small error, such as a typographical mistake, could cause a complete failure of the model. To overcome this issue, in this paper, we introduce a new benchmark called Generalized Referring Remote Sensing Image Segmentation (GRRSIS), which extends classic RRSIS by allowing expressions to refer to no-target objects. Towards this, we construct the first large-scale dataset for GRRSIS, called GRRSIS-D, which includes multi-target, single-target, and no-target expressions. Core challenges in GRRSIS stem from the fact that objects in aerial images often occupy only a small number of pixels, exhibit significant orientation variations, and present varying levels of recognition difficulty. To tackle these challenges, we propose an Oriented-aware Multi-Scale Network with an Adaptive Angle Sensing module that integrates Adaptive Rotated Convolution and a gating mechanism to capture diverse object orientations while suppressing irrelevant features for more accurate representations. Additionally, we introduce a novel Online Hard Case Mining Loss, which allocates varying levels of attention to foreground and background regions and reshapes the standard loss by down-weighting well-segmented examples, effectively addressing the issues caused by low pixel occupancy and uneven sample difficulty. The proposed approach achieves state-of-the-art performance on both the newly introduced GRRSIS and classic RRSIS tasks.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.