VFI-LoRA : Leveraging Video Diffusion Models for Video Interpolation Through LoRA Finetuning

Le résumé fourni par la source

We present VFI-LoRA, a method for generating precise and realistic intermediate frames given only a first and a last frame. By leveraging a pretrained image-to-video diffusion model (Stable Vdeo Diffusion), our approach produces frames that remain both spatially and temporally consistent. To adapt the diffusion model for video frame interpolation, we employ Low-Rank Adaptation (LoRA) to efficiently finetune the model, enabling it to handle large motions effectively. Additionally, we introduce a technique to refine generated frames during the denoising stages of inference. To address scenarios where objects may disappear in sequences with large motions, we further perform renosing and denosing steps after generating latent features with backward process of inference. We compare our method against both existing diffusion-based and CNN-based interpolation methods, demonstrating its effectiveness, particularly for sequences with complex, nonlinear motions.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.