Automation of literature screening for meta-analyses in the field of psychology: an evaluation of two text-mining approaches

Le résumé fourni par la source

Abstract Background The process of consolidating research through meta-analyses is time-consuming and costly. Using text-mining approaches to (semi-)automate the step of screening and selecting articles eligible for inclusion might potentially save hours of workload and costs. A lack of evaluative research on the use of text-mining for literature screening prevents the widespread adoption of text-mining approaches among the psychology research community. This study explores the possibilities of applying text-mining approaches to facilitate or (semi-)automate screening articles for inclusion in psychology meta-analyses. Methods The performances of two publicly accessible text-mining approaches were evaluated and compared against three recent meta-analyses from different subfields of psychology. Results Averaged over 10 text-mining runs per meta-analysis, across the three meta-analyses the approaches achieved recall performances between 62.5% and 99.3%, specificities between 60.1% and 92.7%, and saved between 54.0% and 92.0% of screening workload. Higher recall was observed when more articles were screened manually in the text-mining process. Conclusions These findings suggest that text-mining approaches can substantially reduce screening workload while maintaining high recall. These efficiency gains come with a trade-off: the more workload is saved, the lower the recall performance. Text-mining can therefore serve as a valuable tool for the initial identification of potentially eligible studies or as a secondary screening aid, but the risk of missing relevant studies should be mitigated by retaining an appropriate amount of manual screening.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.