DTA-PDVC: Dynamic Temporal Anchor Boxes for Parallel Dense Video Captioning

Le résumé fourni par la source

Dense video captioning aims to localize events within a video and generate corresponding captions. The event proposals of single-stage methods are responsible for event localization and captioning simultaneously, which leads to suboptimal semantic representation of proposals and prolongs the training process. In this paper, we introduce the concept of Dynamic Temporal Anchor Box and Scale-Modulated Attention to explicitly inject positional information into event proposals, enabling the model to better localize events of varying sizes within the video and accelerating model convergence. Concurrently, we design a bidirectional spatiotemporal encoder based on Mamba to better model the contextual information of the video while boosting the model efficiency. Extensive experiments on the ActivityNet Captions and YouCook2 datasets show that our model can generate high-quality captions, achieving state-of-the-art results among visual-only models and rivaling some methods trained with additional data. The code is available at https://github.com/JohnnyHaytham/DTA-PDVC.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.