Knowledge Mechanisms Across the Lifecycle of Large Language Models

Le résumé fourni par la source

Large language models (LLMs) have reshaped artificial intelligence. They now achieve strong performance across question answering, commonsense reasoning, code generation, and mathematical problem solving. These striking capabilities ultimately rest on the knowledge a model acquires during training. The internal processes by which a model acquires, encodes, uses, revises, and transfers this knowledge constitute its \emph{knowledge mechanisms}. However, our understanding of these mechanisms has lagged behind the steady advance of the models' external capabilities. How a model with billions of parameters takes in, organizes, and deploys knowledge is no longer only an engineering concern about reducing hallucinations. It has become a central question for interpretability, and answering it is a prerequisite for putting the study of these models on a scientific footing. The study of knowledge in LLMs has recently gone through a shift in perspective. Earlier surveys largely took a macro view of the knowledge lifecycle, treating the model as an implicit knowledge base and organizing the field around its observable phenomena. Such a view captures external behavior but not the internal machinery, so failures such as factual fabrication, answer inconsistency, and capability degradation after editing remain hard to explain. Recent work has instead turned to the micro-level mechanisms inside the model, examining knowledge localization, feature superposition in activation space, knowledge conflict in retrieval-augmented generation, and the way multi-step reasoning unfolds. Yet this work remains scattered across communities with their own concepts, methods, and evaluation standards. To bridge this gap, we examine knowledge mechanisms across the full lifecycle of large language models. We adopt a unified, stage-driven taxonomy that treats knowledge as an object under continuous change, tracing stage by stage how it is transformed as it moves through this lifecycle. Specifically, we organize the survey around six interconnected stages: - Knowledge Acquisition: How knowledge enters the model, and how training strategies shape the knowledge ultimately acquired. - Knowledge Representation: How the acquired knowledge is encoded and organized in the model's parameters and activation space. - Knowledge Utilization: How the model accesses its internal knowledge and augments it with external sources, and the conflicts that arise among these sources in practice. - Knowledge Evolution: How knowledge is updated or removed after training, and the side effects of these interventions. - Knowledge Transfer: How knowledge transfers across different models, modalities, and languages. - Knowledge Evaluation: How to evaluate what a model knows across the lifecycle. This survey helps trace where the knowledge-related failures of current models originate and how interventions at one stage propagate to the others. These connections are difficult to see from a single-stage view. This lifecycle perspective points beyond the black-box view toward models whose knowledge is mechanistically transparent, continually adaptable, and trustworthy.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.