Epistemic Fine-Tuning of Open-Weight LLMs for Deep Research: Nehanda v3 and the FACTS Grounding Benchmark

Le résumé fourni par la source

Frontier language models (GPT-4o, Claude 3.5 Sonnet, Gemini 2.5 Pro) are trained to be helpful and fluent. In retrieval-augmented generation (RAG) settings — where a model is given source documents and asked to synthesize a grounded response — this training produces a characteristic failure mode: the model generates plausible-sounding claims that are not supported by the provided sources. We ask whether targeted fine-tuning on a smaller open-weight model can produce epistemic behavior that exceeds frontier models on source-grounded generation, without matching their parameter count or training budget. We fine-tune Qwen3.6-27B VL (an open-weight 27B model) using a five-stage stacked QLoRA pipeline that trains 1.15% of parameters across epistemic foundation, evidence hardening, RAG synthesis, constitutional alignment, and preference optimization. The resulting model — Nehanda v3 — is evaluated on FACTS Grounding (Jacovi et al., 2025), a public benchmark of 860 examples (google/FACTS-grounding-public) requiring long-form responses grounded in provided context documents. Nehanda v3 achieves a factuality score of 88.7% (763 of 860 examples judged both eligible and grounded), compared to 87.8% for Gemini 2.5 Pro, 83.8% for Claude 3.5 Sonnet, and 79.8% for GPT-4o. On the 816 valid responses (excluding hardware failures), the grounding rate is 93.5%. The result suggests that epistemic behavior — source fidelity, evidence boundary enforcement, refusal to fabricate — is a trainable capability that targeted fine-tuning can install more efficiently than scale alone.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.