Dual Stream Framework with Self-Correcting Memory for Semi-Supervised Video Object Segmentation

Le résumé fourni par la source

Recent memory-based approaches have shown promising performance in semi-supervised video object segmentation. However, they still face two core challenges. First, their heavy reliance on pixel-level feature matching makes them vulnerable to background distractors that closely resemble the target object, resulting in erroneous segmentations. Second, these incorrect segmentation masks are indiscriminately stored in memory, causing error accumulation and deteriorating performance over time. To address these challenges, we propose the Instance Aware Self-Correcting Model (IASC), which integrates three key components: the Dual-Stream Framework, the Online Adaptation Module (OAM), and the Self-Correcting Module (SCM). The Dual-Stream Framework leverages complementary representations by combining the Appearance Feature Stream, which recovers fine-grained pixel-level details through similarity-based matching, and the Semantic Mask Stream, which generates instance-level attention using reference masks to suppress distractors and enhance segmentation robustness. The OAM dynamically fine-tunes the key projector using the first annotated frame, improving the accuracy of pixel-level matching and the reliability of instance-level guidance. Additionally, the SCM refines unreliable predictions, effectively mitigating error propagation during memory updates. Extensive experiments on benchmark datasets demonstrate that IASC achieves state-of-the-art performance, striking an effective balance between segmentation accuracy and computational efficiency.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.