Diagnostic Accuracy of a Custom Large Language Model on Rare Pediatric Disease Case Reports
Rattachement africain : us. Niveau de preuve : code pays fourni par la source.
Le résumé fourni par la source
Accurately diagnosing rare pediatric diseases frequently represent a clinical challenge due to their complex and unusual clinical presentations. Here, we explore the capabilities of three large language models (LLMs), GPT-4, Gemini Pro, and a custom-built LLM (GPT-4 integrated with the Human Phenotype Ontology [GPT-4 HPO]), by evaluating their diagnostic performance on 61 rare pediatric disease case reports. The performance of the LLMs were assessed for accuracy in identifying specific diagnoses, listing the correct diagnosis among a differential list, and broad disease categories. In addition, GPT-4 HPO was tested on 100 general pediatrics case reports previously assessed on other LLMs to further validate its performance. The results indicated that GPT-4 was able to predict the correct diagnosis with a diagnostic accuracy of 13.1%, whereas both GPT-4 HPO and Gemini Pro had diagnostic accuracies of 8.2%. Further, GPT-4 HPO showed an improved performance compared with the other two LLMs in identifying the correct diagnosis among its differential list and the broad disease category. Although these findings underscore the potential of LLMs for diagnostic support, particularly when enhanced with domain-specific ontologies, they also stress the need for further improvement prior to integration into clinical practice.
Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.
Le contrôle bibliographique ouvert
DOI retrouvé dans Crossref DOI retrouvé ; titre concordant.
- Titre Crossref
- Diagnostic Accuracy of a Custom Large Language Model on Rare Pediatric Disease Case Reports
- Date Crossref
- 13/09/2024
- Éditeur
- Wiley
- Type
- journal-article
Ce recoupement confirme des métadonnées liées au DOI. Il ne confirme ni la méthode ni les conclusions de l’étude, et il ne compte pas comme une seconde source scientifique indépendante.
Où se fait cette recherche
-
Harvard University pays non établi dans la noticeUniversité ou école supérieure
-
Mass General Brigham pays non établi dans la noticeÉtablissement de santé
-
Massachusetts General Hospital pays non établi dans la noticeÉtablissement de santé
-
Harvard Medical School Boston Massachusetts USA pays non établi dans la noticeInstitution
-
Medically Engineered Solutions in Healthcare Incubator pays non établi dans la noticeEntreprise
Harvard University, Mass General Brigham et Massachusetts General Hospital, avec 2 autres affiliations.
Une affiliation ne permet pas de déduire la nationalité d’un auteur.