Image-Text to Text based Multimodal AI using FLORENCE-2 for Remote Sensing Application

Le résumé fourni par la source

Visual Question Answering (VQA) in remote sensing represents a critical advancement at the intersection of computer vision and natural language processing, facilitating context-aware interpretation of satellite and aerial imagery without human involvement. This study advances the domain by fine-tuning Microsoft’s FLORENCE-2, a lightweight multi-modal vision language model, for highresolution remote sensing images. The model is fine-tuned on the RSVQA-HR dataset to bridge the gap between textual queries and complex geospatial imagery, addressing challenges such as spectral variability, occlusion, and scale diversity. The architecture integrates DaViT-based hierarchical vision encoding, enabling multi-scale feature extraction across spatial dimensions (128, 256, 512, and 1024), while a transformer-based decoder with cross-modal attention mechanisms ensures effective alignment of image embeddings with linguistic representations. Final Results demonstrate a higher accuracy and interpretability of satellite image-based VQA tasks by using the fine-tuned model. This research underscores the potential of large language vision models in enhancing geospatial intelligence, extending Artificial Intelligence (AI) application to environmental monitoring, disaster management, and urban planning.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.