Evaluating the Potential Risks of Employing Large Language Models in Peer Review (Preprint)

Le résumé fourni par la source

BACKGROUND LLMs are increasingly used in academic processes, including peer review. While they can address challenges like reviewer scarcity and review efficiency, concerns about fairness, transparency, and potential biases in LLM-generated reviews have not been thoroughly investigated. OBJECTIVE This study aims to assess the potential ethical risks and challenges posed by using large language models (LLMs) in the peer review process for cancer research articles. METHODS Claude 2.0 was used to generate peer review reports, rejection recommendations, citation requests, and refutations for 20 cancer-related articles. AI detection tools assessed whether the reviews were identifiable as LLM-generated, while experts evaluated their quality and ethical implications. RESULTS LLM-generated reviews were somewhat consistent with human reviews but lacked depth, especially in detailed critique. LLMs easily generated persuasive rejection recommendations and citation requests, including requests for unrelated references. AI detectors struggled to identify LLM-generated reviews, raising concerns about transparency. CONCLUSIONS While LLMs can assist in the peer review process, they pose risks such as biased reviews and the potential manipulation of academic integrity. Guidelines and detection tools are needed to ensure LLMs enhance rather than harm the peer review process.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.