Convergence of Spectral Descent for Non-smooth Optimization
Yixuan Yang, Yuqing He, Song Li
The Muon optimizer has recently demonstrated remarkable empirical success in training large language models. However, the theoretical understanding of its mechanisms remains limited. Current convergence guarantees for Muon rely heavily on smoothness assumptions, leaving its non-smooth convergence behavior largely unexplored. In this …
ru, cn (code pays fourni par la source)
0 citations
arXiv (Cornell University)