MACE-Dance: Motion-Appearance Cascaded Experts for Music-Driven Dance Video Generation

Le résumé fourni par la source

With the rise of online dance‑video platforms and rapid advances in AIGC, music‑driven dance generation task has emerged as a compelling research direction. Despite substantial progress in related domains such as music-driven 3D dance generation, pose-driven image animation, and audio-driven talking-head synthesis, these approaches are not readily transferable to this task due to fundamental mismatches in generation targets and constraints. Moreover, research on music-driven dance video generation remains limited and fails to capture the inherently 3D nature of dance, resulting in compromised motion quality and visual appearance. Accordingly, we present MACE-Dance, a music-driven dance video generation framework with cascaded Mixture-of-Experts (MoE). The Motion Expert performs music-to-3D motion enforcing kinematic plausibility and artistic expressiveness, while the Appearance Expert carries out motion-and-reference conditioned video synthesis, preserving visual identity with spatiotemporal coherence. Specifically, the Motion Expert adopts Diffusion Model with BiMamba-Transformer hybrid architecture and Guidance-Free Training (GFT) strategy, achieving state-of-the-art (SOTA) performance in 3D dance generation task; the Appearance Expert adopts a decoupled Kinematic–Aesthetic fine-tuning strategy, achieving state-of-the-art (SOTA) performance in the pose-driven image animation task. To better benchmark this task, we curate a large-scale dataset, and design a motion–appearance evaluation protocol. Based on them, MACE-Dance also achieves the state-of-the-art (SOTA) performance. Code is available at https://github.com/AMAP-ML/MACE-Dance.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.