OphFusionNet: Uncertainty-Driven Multi-Scale Multimodal Feature Fusion Network for Ophthalmic Diseases Classification

Le résumé fourni par la source

Multimodal imaging has become an essential tool in clinical ophthalmology, offering complementary perspectives for disease diagnosis. However, current automated diagnostic approaches often fail to fully exploit the rich, complementary information provided by different imaging modalities. In this paper, to advance automated ophthalmic disease diagnosis through effective multimodal data integration, we propose the OphFusionNet, a novel multimodal learning framework based on uncertainty-driven multi-scale multimodal feature fusion. Drawing inspiration from clinical observations that ophthalmic lesions often appear at multiple spatial scales, we design a multi-scale feature fusion module with sparse self-attention (MSFF-SSA). This module captures hierarchical representations while suppressing redundancy, thereby enhancing both the expressiveness and efficiency of the extracted features. To further improve multimodal fusion, we introduce an uncertainty-aware multimodal fusion module with a game-theoretic selection strategy (UMF-GTSS). This component estimates the uncertainty associated with different features and adaptively weights them based on their relative reliability, yielding more robust and trustworthy diagnostic outcomes. To mitigate the tendency to over-rely on dominant modalities and underutilize the informative potential of subordinate ones, we propose a modality distillation strategy (MDS), which leverages multimodal features to guide and refine the learning of single-modal representations. This strategy enhances generalization and boosts the discriminative capacity of each individual modality. The OphFusionNet was evaluated on four publicly available ophthalmic datasets. Extensive experiments demonstrate that our approach achieves superior multimodal integration, resulting in state-of-the-art (SOTA) performance in multimodal ophthalmic disease diagnosis. The code will be available at: https://github.com/wb66715/OphFusionNet.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.