EULoInf: Efficient Hessian-Free Entropy Based Uncertainty-Aware Data Influence Approximation

Le résumé fourni par la source

In Large Language Model post-training, highquality data effectively enhances model performance with fine-tuning, highlighting the need to identify high-quality and beneficial fine-tuning data.However, one of the most popular data valuation paradigms, influence function and its variants, is computationally expensive due to its reliance on inverse Hessian-Vector Products (iHVP) computations that scale poorly with increasing model size.To examine whether influence values correlate with efficiently computable intrinsic features, we empirically investigate the distribution of top influential data for the model in fine-tuning, and observe that data with high influence tend to be those with high predictive uncertainty.Yet such highly uncertain samples exhibit a dual nature, which can be either beneficial or detrimental noisy data.Unlike traditional methods that treat uncertainty as a standalone criterion, we introduce a directional indicator to rigorously disentangle these opposing effects.Formally, we propose EULoInf (Entropy-based Uncertainty-aware Lookahead Influence), a computationally efficient valuation framework.By approximating influence via uncertainty and gradient-based validation loss lookahead, EU-LoInf avoids iHVP computation, effectively reducing the iHVP-induced quadratic complexity in model parameters to linear time.We rigorously derive our framework from the influence function.Empirically, it matches or even outperforms prior methods across diverse data valuation tasks and LLM architectures, including mislabel detection and data selection, while reducing computational time and memory usage by over 50%.The code is available at github.com/Rachelcoll/EULoInf.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.