DeepVLN: Vision-and-Language Navigation via Deep Reasoning and Collaborative Mechanisms Based on Large Language Models

Le résumé fourni par la source

Vision-and-language navigation (VLN) requires agents to follow natural language instructions and navigate in previously unseen environments. Large language models (LLMs) bring strong reasoning and generalization abilities to VLN. However, existing LLM-based methods still suffer from information loss in vision-to-text conversion, rigid prompting, or manually designed reasoning steps, as well as weak closed-loop control, leading to error accumulation. We propose DeepVLN, an LLM-based VLN framework that enables autonomous chain-of-thought (CoT) reasoning and adaptive navigation. DeepVLN first adapts an open-source LLM to VLN via a three-stage supervised fine-tuning pipeline, and then further optimizes its closed-loop policy with reinforcement learning, encouraging robust feedback use and online error correction without predefined reasoning templates. Additionally, an API-based collaborative reasoning module enables a lightweight local agent to selectively query a stronger cloud LLM under high uncertainty, thereby balancing performance and computational cost. Experiments on R2R, RxR, and REVERIE show that DeepVLN achieves competitive or superior results to strong VLN-specific and LLM-based baselines, with higher success rates and path efficiency in most settings. These results demonstrate the effectiveness of equipping LLMs with autonomous, closed-loop reasoning for embodied navigation.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.