Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability
Rattachement africain : us, gb, fr, rs, cz, sk, lk, hu, ch, pl, it, de, se, at, pt, nl, be, ca, bo, br, vn, nz, cn. Niveau de preuve : code pays fourni par la source.
Le résumé fourni par la source
Replication studies in psychological science sometimes fail to reproduce prior findings. If these studies use methods that are unfaithful to the original study or ineffective in eliciting the phenomenon of interest, then a failure to replicate may be a failure of the protocol rather than a challenge to the original finding. Formal pre-data-collection peer review by experts may address shortcomings and increase replicability rates. We selected 10 replication studies from the Reproducibility Project: Psychology (RP:P; Open Science Collaboration, 2015) for which the original authors had expressed concerns about the replication designs before data collection; only one of these studies had yielded a statistically significant effect ( p < .05). Commenters suggested that lack of adherence to expert review and low-powered tests were the reasons that most of these RP:P studies failed to replicate the original effects. We revised the replication protocols and received formal peer review prior to conducting new replication studies. We administered the RP:P and revised protocols in multiple laboratories (median number of laboratories per original study = 6.5, range = 3–9; median total sample = 1,279.5, range = 276–3,512) for high-powered tests of each original finding with both protocols. Overall, following the preregistered analysis plan, we found that the revised protocols produced effect sizes similar to those of the RP:P protocols (Δ r = .002 or .014, depending on analytic approach). The median effect size for the revised protocols ( r = .05) was similar to that of the RP:P protocols ( r = .04) and the original RP:P replications ( r = .11), and smaller than that of the original studies ( r = .37). Analysis of the cumulative evidence across the original studies and the corresponding three replication attempts provided very precise estimates of the 10 tested effects and indicated that their effect sizes (median r = .07, range = .00–.15) were 78% smaller, on average, than the original effect sizes (median r = .37, range = .19–.50).
Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.
Le contrôle bibliographique ouvert
DOI retrouvé dans Crossref DOI retrouvé ; titre concordant.
- Titre Crossref
- Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability
- Date Crossref
- 01/09/2020
- Éditeur
- SAGE Publications
- Type
- journal-article
Ce recoupement confirme des métadonnées liées au DOI. Il ne confirme ni la méthode ni les conclusions de l’étude, et il ne compte pas comme une seconde source scientifique indépendante.
Où se fait cette recherche
-
University of Virginia Department of Psychology pays non établi dans la noticeUniversité ou école supérieure
-
Quantitative BioSciences pays non établi dans la noticeOrganisation à but non lucratif
-
Stanford University Quantitative Sciences Unit pays non établi dans la noticeUniversité ou école supérieure
-
University of Houston Department of Psychology pays non établi dans la noticeUniversité ou école supérieure
-
Ashland University Department of Psychology pays non établi dans la noticeUniversité ou école supérieure
-
Grand Valley State University Department of Psychology pays non établi dans la noticeUniversité ou école supérieure
-
University of Edinburgh pays non établi dans la noticeUniversité ou école supérieure
-
Boston College Psychology and Neuroscience Department pays non établi dans la noticeUniversité ou école supérieure
-
Institut Universitaire de France pays non établi dans la noticeUniversité ou école supérieure
-
Université Grenoble Alpes LIP/PC2S pays non établi dans la noticeUniversité ou école supérieure
-
University of Belgrade pays non établi dans la noticeUniversité ou école supérieure
-
Czech Academy of Sciences pays non établi dans la noticeStructure de recherche
Department of Psychology — University of Virginia, Quantitative BioSciences et Quantitative Sciences Unit — Stanford University, avec 9 autres affiliations.
Une affiliation ne permet pas de déduire la nationalité d’un auteur.