Scaling Intelligence Through Model Merging: A Comprehensive Survey

Le résumé fourni par la source

Model merging offers a solution for achieving artificial general intelligence by combining multiple pre-trained or task-specific models into a unified architecture. This approach aims to improve model efficiency, adaptability, and scalability while maintaining or enhancing performance across diverse tasks. Model merging is particularly important when handling multiple tasks simultaneously, as deploying several task-specific experts at once can be computationally expensive and resource-intensive, posing considerable challenges in the supporting system platform design. However, a systematic review of model merging methods and underlying theories is still lacking. In this paper, we present a comprehensive investigation of model merging methods and theories. Moreover, we provide an overview of applications developed using different merging algorithms and discuss strategies for performing alignment in the merging process. In addition to providing an extensive survey from an algorithmic standpoint, we also examine the challenges of model merging and potential future research directions. This survey serves as a valuable resource for researchers aiming to understand the theory of model merging and its algorithmic implementation, offering detailed insights into recent advancements and practical applications.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.