Federated LoRA Fine-Tuning of LLMs With Only Transmitting Matrix A or B

Le résumé fourni par la source

Recent advancements in large language models (LLMs) have been largely driven by vast amounts of training data. As public data are exhausted, fine-tuning LLMs with private data has become increasingly important. However, due to privacy concerns associated with sensitive data, traditional centralized training approaches are not feasible. Federated learning (FL) offers a promising solution, as it enables collaborative model training across multiple mobile devices without data exchange. Nevertheless, it is challenging to fine-tune LLMs with FL on resource-constrained and heterogeneous clients. To reduce the cost, we employ Low-Rank Adaptation (LoRA) for efficient fine-tuning. Existing federated LoRA fine-tuning methods suffer from inaccurate aggregation, and the resulting noise may lead to performance loss that may be more severe for LLMs. In this paper, we propose a novel method, FedOTAB, to mitigate this issue. FedOTAB first correctly aggregates all local LoRA modules and then fixes the matrix$\bf {A}$or$\bf {B}$to optimize the corresponding best approximate matrix$\bf {B}^*$or$\bf {A}^*$for each local LoRA module by minimizing the induced noise. In addition, we design a mechanism to freeze the matrices$\bf {A}$and$\bf {B}$alternately to stabilize optimization. By transmitting only the updated part, that is, half of the LoRA module, FedOTAB enjoys favorable communication efficiency. By explicitly splitting LoRA into local and global parts, FedOTAB effectively integrates both local and global knowledge. Extensive experiments demonstrate that FedOTAB excels in both homogeneous and heterogeneous rank scenarios, consistently outperforms state-of-the-art methods with remarkably low communication and computational overhead. We hope that this work provides valuable insights for federated LoRA fine-tuning, offering a practical and scalable solution to fine-tuning large models in mobile environments.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.