SceneRAG: Scene-Level Retrieval-Augmented Generation for Video Understanding

Le résumé fourni par la source

Despite recent progress in retrieval-augmented generation (RAG) for video understanding, handling long-form video remains challenging due to the scale and complexity of video data. Existing RAG methods usually segment videos into fixed-length chunks, disrupting contextual continuity and overlooking authentic scene boundaries. Inspired by the human ability to structure continuous experiences into coherent scenes, we propose SceneRAG—a unified framework that employs large language models to segment videos into narrative-consistent scenes using ASR transcripts and temporal metadata, further refined by lightweight heuristics and iterative correction. For each scene, SceneRAG integrates visual and textual modalities to construct a dynamic knowledge graph, enabling robust multi-hop retrieval and long-range reasoning. We evaluate SceneRAG on a 134-hour long-form video benchmark spanning lectures, documentaries, and entertainment, SceneRAG achieves substantial gains over prior baselines, with win rates up to 70.8% on generation tasks.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.