Validation of the QAMAI tool in Italian for the evaluation AI-generated health information in head and neck surgery

Résumé fourni par la source

BACKGROUND: This study aimed to validate the Italian version of the Quality Assessment of Medical Artificial Intelligence (IT-QAMAI) tool, designed to evaluate the reliability of AI-generated health information in the context of head and neck surgery.METHODS: The IT-QAMAI tool was adapted from the original English version and involved a rigorous translation and back-translation process. The validation involved 18 researchers from 13 centers across Europe, assessing 24 AI-generated responses categorized into clinical scenarios, theoretical questions, and patient inquiries. The tool’s reliability was measured using Cronbach’s alpha for internal consistency, the Intraclass Correlation Coefficient (ICC) for inter-rater reliability, and Pearson’s correlation for test-retest reliability.RESULTS: The IT-QAMAI demonstrated high internal consistency (Cronbach’s alpha = 0.850) and good inter-rater reliability (ICC=0.750). Test-retest reliability was strong (rs=0.887). Significant differences were found in the quality of AI-generated responses across different question types.CONCLUSIONS: The IT-QAMAI tool is a reliable and valid instrument for assessing the quality of AI-generated health information in Italian, with significant implications for its use in clinical practice and research in head and neck surgery.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.