Stepwise Question Rephrasing: Enhancing Long-Context Reasoning in Small-Scale Language Models

Résumé fourni par la source

Small-scale language models (SLMs) offer practical advantages in efficiency and deployment cost, but they often struggle with long-context question answering (QA) tasks that require multi-hop reasoning over dispersed or implicit evidence. We present a lightweight and model-agnostic method that improves QA performance by dynamically rephrasing complex questions into stepwise subquestions, effectively guiding the reasoning process without additional training or architectural modifications. Experiments on the LongBench and LongBench v2 benchmarks demonstrate consistent improvements, achieving average gains of 2–5% and reaching up 10% in specific domains. We further validate the generalizability of our approach by applying it to two representative SLM families, LLaMA and GLM, observing consistent gains across both.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.