Masked Spectrum ViT with Cross-Scale Fusion for Universal Deepfake Detection

Résumé fourni par la source

With the rapid evolution of deepfake technologies, existing universal detection methods often fail to generalize when faced with new forgery patterns, as they tend to overfit specific artifacts found in the training data. To address this challenge, we propose a detection framework that integrates adaptive spectrum masking and cross-scale feature fusion, termed Masked Spectrum Vision Transformer with Cross-scale Fusion (MSViT-CF). Our approach introduces two key innovations: (1) Multi-band Artifact Enhancement Module (MAEM) reconstructs input images through wavelet decomposition and strategically perturbs high-frequency subbands via adaptive masking, amplifying subtle forgery traces while forcing the model to learn generalized artifact representations; (2) Dynamic Scale Fusion Transformer (DSFT) integrates multi-resolution frequency features through parallel convolutional-transformer pathways, dynamically weighting local spectrum anomalies and global structural inconsistencies. MAEM enhances artifact sensitivity through frequency-space discrepancy learning, while DSFT establishes cross-scale relationships between pixel-level irregularities and semantic-level inconsistencies via learnable attention gates. Experimental results demonstrate that MSViT-CF significantly outperforms existing state-of-the-art methods in detecting deepfake images generated by various GANs and diffusion models, exhibiting superior universality and robustness.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.