UMA: Ultra-detailed Human Avatars via Multi-level Surface Alignment

Le résumé fourni par la source

Learning an animatable and clothed human avatar model with vivid dynamics and photorealistic appearance from multi-view videos is a foundational research problem in computer graphics and vision. Fueled by recent advances in implicit representations, animatable avatars have reached unprecedented quality by attaching the implicit representation to drivable human template meshes. However, they usually fail to preserve the highest level of detail, e.g., fine textures and yarn-level patterns, especially when the camera zooms in or renders at 4K and higher. We argue that this limitation stems from inaccurate surface tracking, specifically, depth misalignment and surface drift between character geometry and the ground truth surface, which forces the detailed appearance model to compensate for geometric errors. To address this, we adopt a latent deformation model and supervise the 3D deformation of the animatable character using guidance from foundational 2D video point trackers, which offer improved robustness to shading and surface variations, and are less prone to local minima than differentiable rendering. To mitigate the drift over time and lack of 3D awareness of 2D point trackers, we introduce a cascaded training strategy that generates consistent 3D point tracks by anchoring point tracks to the rendered avatar, which ultimately supervise our avatar at vertex and texel level. Furthermore, a lightweight Gaussian texture super-resolution module is employed to reconstruct challenging appearance details and micro-level structures using localized information. To validate the effectiveness of our approach, we introduce a novel dataset comprising five multi-view video sequences, each over 10 minutes in duration, captured using 40 calibrated 6K-resolution cameras, featuring subjects dressed in clothing with challenging texture patterns and wrinkle deformations. Our approach significantly improves rendering quality and geometric accuracy over the state of the art.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.