Optimizing GPT-5 for Operation-Procedure-Code-Extraction from Operative Reports in Meningioma Surgery: Feasibility and Comparison of Context-Enhancements

Le résumé fourni par la source

Abstract In our recent study we showed that GPT's ability to perform OPS-code extraction from operational reports was equivalent to coding neurosurgeons. In this study we aim to evaluate the effect of context enhancement on GPT-5's code extraction abilities. We provided OpenAIs GPT-5 with 100 operational reports (OR) of patients that underwent meningioma surgery. A chat prompt was generated instructing GPT to generate the correct OPS coding. Five groups were formed and provided with different context-enhancing modalities: 1. No additional context (GPT-5s), 2. the current OPS-catalogue (GPT-5o), 3. specified rules on Meningioma coding (GPT-5r), 4. code-triggering sample phrases based on 50 additional ORs (GPT-5e) and 5. all enhancements combined (GPT-5c). We analyzed code-extraction abilities, mistakes and hallucinations. Non context enhanced GPT-5s showed the lowest rate of correct coding (44%) and highest number of hallucinations (105) and total mistakes (132). GPT-5o showed identical accuracy to GPT-5s (44%, p = 1.0), but fewer hallucinations (51, p = 0.002) and mistakes (78, p = 0.008). All other models were significantly superior to GPT-5s and GPT-5o in accuracy (GPT-5r 79%, GPT-5e 70%, GPT-5c 86%, p < 0.001), hallucinations (GPT-5r 8, GPT-5e 2, GPT-5c 6, p < 0.001) and mistakes (GPT-5r 21, GPT-5e 35, GPT-5c 14, p < 0.001). Highest coding accuracy was achieved by GPT-5c (GPT-5c vs GPT-5r, p = 0.143, GPT-5c vs GPT5e p = 0.008). GPT-5r and GPT-5e performed equally regarding accuracy, GPT-5r produced fewer mistakes (p = 0.034), while GPT-5e hallucinated less (p = 0.057). We show, that specific context enhancement significantly improves GPT-5s ability to perform OPS-code extraction from operational reports, while significantly lowering mistake- and hallucination rates.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.