MMNP: Multi-Modal Negative Prototypes for Few-Shot Open-Set Recognition

Le résumé fourni par la source

Few-Shot Open-set Recognition (FSOR) is a challenging task as it poses the dual challenge of classifying known classes with limited labeled samples while effectively identifying unknown classes. Existing approaches typically rely solely on visual features to construct negative prototypes for modeling the open-set space, which limits their representational capacity and generalization. These methods overlook the rich semantic information in class names, which can significantly enhance the modeling of known and unknown spaces. To address this limitation, we propose Multi-Modal Negative Prototypes (MMNP), a novel FSOR framework that integrates visual and semantic modalities to generate discriminative multi-modal class and negative prototypes. MMNP introduces a Semantic Extractor and a Feature Fusion Network to construct discriminative multi-modal class prototypes. In the meta-learning phase, we introduce a Multi-Modal Negative Prototypes (NPs) Generator to generate multi-modal negative prototypes via an attention mechanism. We validate the effectiveness of our method through extensive experiments on two standard FSOR datasets, demonstrating that MMNP outperforms existing state-of-the-art FSOR methods in both 1-shot and 5-shot settings.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.