Diagnostic accuracy of ChatGPT and physicians in patients with abdominal pain: a cohort study (Preprint)

Le résumé fourni par la source

BACKGROUND Economic growth has increased the demand for healthcare resources, but has also led to challenges such as lengthy appointment waiting times and a shortage of medical professionals. The uneven distribution of medical infrastructure in some regions has resulted in limited healthcare services in rural or impoverished areas. ChatGPT-3.5, the latest and most popular conversational artificial intelligence(AI), has demonstrated its potential in providing real-time health information and alleviating the burden on healthcare workers. While ChatGPT has performed well in medical knowledge examinations, its capabilities in clinical decision-making remain uncertain. OBJECTIVE Evaluate the potential value of GPT in medical diagnosis. METHODS The diagnostic accuracy of ChatGPT was compared among three groups: patients, questionnaire respondents, and physicians. The results showed that the accuracy was lowest in the patient group (True: 19.1%, False: 80.9%), highest in the physician group (True: 59.6%, False: 39.6%), and moderate in the questionnaire group (True: 51.1%, False: 48.9%). The difference between the patient group and the other groups was statistically significant (p<0.05). Among all disease categories, the highest diagnostic accuracy was observed for appendicitis and pancreatitis, while gastrointestinal tumors were difficult to diagnose accurately across all groups. RESULTS The diagnostic accuracy of ChatGPT was compared among three groups: patients, questionnaire respondents, and physicians. The results showed that the accuracy was lowest in the patient group (True: 19.1%, False: 80.9%), highest in the physician group (True: 59.6%, False: 39.6%), and moderate in the questionnaire group (True: 51.1%, False: 48.9%). The difference between the patient group and the other groups was statistically significant (p<0.05). Among all disease categories, the highest diagnostic accuracy was observed for appendicitis and pancreatitis, while gastrointestinal tumors were difficult to diagnose accurately across all groups. CONCLUSIONS This study reveals that ChatGPT demonstrates promising diagnostic accuracy in abdominal pain-related diseases when provided with detailed information. However, limitations in patient self-expression, information-gathering, and humanistic care prevent it from fully replacing doctors. Further development and research are needed to enhance AI's role in assisting medical professionals and providing medical consultation services to patients. CLINICALTRIAL none

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.