A multi-modal fusion model via knowledge distillation for accurate tomato leaf disease diagnosis

Le résumé fourni par la source

Tomato is one of the most widely cultivated vegetable crops worldwide, and leaf diseases can significantly reduce yield and quality, leading to considerable economic losses. Existing tomato leaf disease recognition methods are mainly based on deep-learning image classification. However, in real field environments, complex natural backgrounds and high visual similarity among different diseases often limit the discriminative ability of image-only models, while high-performance models usually require substantial computational resources, which restricts real-time deployment in agricultural scenarios. To address these challenges, this study proposes a lightweight knowledge-distillation-driven text-image fusion framework, termed T-MDKM, for tomato leaf disease classification. Unlike conventional image-only methods, this study constructs a Tomato Leaf Disease Spectrum (TLDS) dataset based on 7,043 tomato leaf images collected at the Institute of Plant Protection, Hunan Academy of Agricultural Sciences. Each image is further annotated with fine-grained textual descriptions by agricultural experts, including leaf orientation, position, color, as well as the location and color of lesions, thereby providing complementary agronomic semantic information for disease recognition. The proposed framework enhances cross-modal consistency between visual and textual representations, progressively models multimodal interactions, and improves robustness under complex disease scenarios and class imbalance. To further support practical deployment, a distilled student model, TS-MDKM-AFC, is developed to transfer knowledge from the multimodal teacher model to a lightweight architecture, reducing computational cost while maintaining competitive classification performance. Experimental results on the TLDS dataset and two public benchmark datasets show that T-MDKM achieves 97.01% precision and 96.79% accuracy, outperforming the best competing method by 0.34 and 0.41 percentage points, respectively. Meanwhile, TS-MDKM-AFC incurs only a 0.55 percentage-point drop in accuracy while reducing the number of parameters, FLOPs, and inference time by 67.2%, 73.66%, and 62.15%, respectively. These results demonstrate that the proposed framework can improve disease recognition accuracy under complex field conditions while offering favorable efficiency for real-time agricultural deployment.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.