Low-data cross-modal adaptation for remote sensing with proxy-enhanced multi-granularity feature caching

Le résumé fourni par la source

Despite the potential of vision-language models for open-vocabulary recognition, their deployment in remote sensing is limited by the limited effectiveness of generic prompts, the scarcity of annotated datasets, and insufficient domain-specific feature discriminability. To address these limitations, we propose a proxy-enhanced multi-granularity feature caching adaptation framework for cross-modal remote sensing imagery under low-data settings. The proposed architecture integrates three interdependent mechanisms. (1) An LLM-augmented prompt module transforms generic class labels into descriptive attribute sets, such as spatial patterns and spectral characteristics, thereby providing the vision encoder with more informative textual representations. (2) A proxy-enhanced semantic calibration mechanism constructs class-level visual proxies within the frozen visual embedding space, enabling reliable pseudo-label support set generation and improved semantic alignment under limited supervision. (3) A multi-granularity feature cache stores both patch-level texture features and scene-level topological representations. During inference, the cached features are retrieved and combined with zero-shot CLIP predictions, thereby reducing the semantic gap between image and text modalities in remote sensing. The integration of these components strengthens semantic grounding through LLM-augmented prompts and proxy-based support sets, while feature caching and proxy calibration mitigate domain-specific representation gaps. The proposed framework exhibits stable performance in few-shot scenarios where conventional fine-tuning approaches fail to converge. Extensive evaluations on multiple benchmark datasets show that our method outperforms existing cross-modal adaptation approaches.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.