An exploratory study into the relationship between sycophantic behavior in large language models and military decision makers engaged in naturalistic decision making

Résumé fourni par la source

Abstract In an attempt to gain the decision advantage, military commanders in the contemporary decision space are facing increasing pressure to conceptualize information at speed. With the rapid proliferation of AI, the tech industry has pivoted to capitalize on the opportunity, and growing demand, to support commanders in their application of a craft that takes years to hone. While the desire to integrate AI into traditional decision‐making models is understandable, research suggests it should be approached with caution, emphasizing the importance of first understanding the complex nexus of the human and machine interface before embedding it within a system that executes use‐of‐force in extreme cases. In its current iteration, AI's most sophisticated form—Large Language Models (LLMs) are showing tendencies to act with strategic deception and align with user bias in a phenomenon known as sycophancy. An exploratory study into the impact of LLM behavior on the intuitive decision making of military decision makers, engaged in hasty planning, was conducted. Designed from the naturalistic decision making (NDM) perspective—30 military officers were tested in a Tactical Decision Game (TDG) engineered to align with characteristics of the Multi‐Domain Operations military environment, and where time pressure favored swift sense‐making and action over exhaustive analysis. Their ability to conceptualize a problem and end state and to produce two Courses of Action (COAs) to address the prescribed mission was evaluated. Two experiment groups working with the support of an AI Agent, each reflecting an embedded bias—one sycophantic and the other objective, alongside a control group working under status quo conditions. The AI supported groups used a bespoke application, with ChatGPT‐4o (gpt‐4.1‐nano‐2025–04–14) embedded as an information retrieval, situational awareness, and decision support tool. Qualitative results show LLM behavior has the potential to impact the recognition primed decision making (RPD) process, where linguistic outputs amplify the user's initial framing and smooth the cognitive process to facilitate progression in the problem rather than encourage critically analyzing pertinent information. Findings suggest both LLM biases deviated from the embedded prompt—impacting the participants’ contextual understanding of affordances within the scenario and calling into question the reliability of prompt execution. As such, sycophantic behavior remains difficult to define with any accuracy with its impact shaped by context, user experience and expertise and the intensity of conditions existent in the decision space.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.