P-387 Exploring the Potential of Artificial Intelligence in Providing Infertility Consultations and Answers: can it replace the infertility specialist? A hybrid study.

Résumé fourni par la source

Abstract Study question What are the differences between the accuracy of answers given by an AI-based infertility consultation system compared to those of an infertility specialist? Summary answer OpenAI, an AI platform, does not have the potential to provide proper consultation to women searching for answers regarding their infertility. What is known already Artificial intelligence (AI) has been recently introduced in the field of human reproduction, both in controlled ovarian stimulation algorithms and laboratory decisions. Additionally, AI-enabled technology has been used to facilitate online communication between patients and healthcare professionals. However, the recent widespread use of AI-based chatbots is causing serious concerns about their use in providing medical advice. Furthermore, the capacity of AI-based chatbots to answer intricate questions about infertility and its treatment has yet to be explored. Despite AI being declared to not be able to provide accurate medical advice, a study assessing this has not yet been conducted. Study design, size, duration The aim of this prospective observational study was to assess the ability of OpenAI, an artificial intelligence (AI) platform, to provide appropriate infertility answers to women asking online questions. OpenAI generated answers and answers provided by infertility specialists (control) were both judged by external, blinded infertility specialists in terms of scientific appropriateness (Correctness), detailed feature (Accuracy), ability to properly answer the infertility question (Precision) and providing human kindness (Empathy). Participants/materials, setting, methods 20 online infertility questions were retrieved from the web. For each question, the openAI playground was questioned “Answer this question. You are the infertility specialist”. The AI platform was available on the internet website https://beta.openai.com. The evaluators were four different infertility specialists with more than 20 years' experience and they were blinded to the purpose of the study and the presence of AI answers. Four criteria were used (1-5): Correctness, Accuracy, Precision and Empathy. Main results and the role of chance The mean total score for the human answer without the empathy criteria was 11.18/15 (14.53/20 with empathy). The AI answer provided total score of 8.95/15 and 12.50/20 respectively. At the Mann-Whitney test, both the total scores showed a statistically significant difference (p < 0.001). With a tolerance of -/+ 0.5 total score without empathy, the AI answers were evaluated similar for 4/20 answers, compared to the human answers and better than the human infertility specialist answer in 2/20. Of outmost importance, none of the evaluators suspected the presence of an AI engine tool. Intriguingly, evaluating the absolute impact, the scores obtained from AI were higher than we expected and showed great results. This result strongly suggests that, although not able to overcome an infertility consultation, AI has shown great abilities in mimicking an infertility specialist. AI usage in providing infertility answer or consultation should be not considered due to lack of scientific appropriateness, answering details, and question-points precision. However, the results obtained from AI showed borderline performance with the human’s even at the evaluation of a team of blinded infertility specialists. Indeed, the AI tool should be appropriately addressed given its worldwide spread. Limitations, reasons for caution Further research is needed to understand the limitations and assess the validity of this engine in providing appropriate medical advice. Caution and ethical concerns should be taken into consideration for the coming future. Wider implications of the findings OpenAI also contributed to develop this study, by giving answers to our questions on the potential study design. Moreover, and intriguingly, this abstract is the first hybrid-written (human and AI) of the history of ESHRE abstract presentation. Trial registration number Not Applicable

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.