repliCATS-SCORE: Elicited human predictions of social and behavioural science replicability

Le résumé fourni par la source

The repliCATS-SCORE dataset consists of five human elicitation studies predicting the replicability of published research in the social and behavioural sciences, conducted by the repliCATS project (Collaborative Assessments for Trustworthy Science) as part of DARPA’s Systematizing Confidence in Open Research and Evidence or SCORE program (2019-2022). The human elicitations were conducted using a structured group deliberation technique – the IDEA protocol – for a total of 4000 published research claims and an additional 25 claim pilot study. The dataset for each study includes participant-level judgements across two elicitation rounds, mathematically aggregated group judgements or “confidence scores” calculated using a suite of 38 aggregation methods, as well as participant free-text reasoning. The repliCATS-SCORE dataset can be re-used in support of studies interrogating scholarly judgements of replicability, plausibility and overall credibility, post-publication peer review and critical evaluation training, developing novel mathematical aggregation models that forecast replication outcomes, and comparative studies.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.