Statistical Power in a Recent Study by Schoenfeld et al.

Le résumé fourni par la source

Dear Editor-in-Chief, In a recent article by Schoenfeld et al. (1) about the effect of training volume on muscle growth and strength, the authors justified a sample size of n = 36 by “a priori power analysis in G*power.” This power analysis used “a target effect size (ES) of f = 0.25, alpha of 0.05 and power of 0.80,” with “group (one, three, or five sets) as the factor” and “baseline value as a covariate.” Although the authors provided the said information, they did not provide sufficient information to replicate their calculations. More specifically, the numerator df and number of groups were missing. When replicating this power analysis using G*power and the same values, assuming Numerator df = 2 (3(factor levels) − 1) and Number of Groups = 3, I obtained a different sample size of n = 158. In addition to this finding, I have checked what the correct original power estimate would have been, using the sample size of n = 36 and the preexperiment chosen effect size of f = 0.25, and found it to be 0.23. Therefore, this study design by Schoenfeld et al. (1) is deemed to be underpowered for the effect size chosen (2). Although some methodologists and statisticians advise to abandon low-power studies, and many ethics review boards find them unethical (3), low-powered trials should be published irrespective of their results, thereby becoming available for meta-analysis (4,5). However, these low-powered studies must report their methods and results properly to avoid misinterpretation (2), which was not the case in this article. In addition, even if the a priori power analysis was correct and the sample of n = 36 was sufficient for a power value of 0.8, the number of participants in the final analysis was smaller than 36 (n = 34) (1). As a result, power analysis of this final sample size, with the predetermined effect size of f = 0.25, results in even lower value of power, 0.218. Underpowered study designs undermine the purpose of scientific research, as they decrease the chance to detect a true effect (i.e., they have high risk of type II error) (6). In addition, underpowered studies have a substantial risk to report inflated effect sizes because of the sampling distribution of the means having a high variance compared with appropriately powered studies (7). Therefore, for a study design with a power of 0.23, the risk of reporting substantially inflated effect sizes should be considered relatively high. Eliran Mizelman EM-SportScience, Vancouver, BC, CANADA Sports Analytics Group and Department of Biomedical Physiology and Kinesiology Simon Fraser University, Burnaby, BC, CANADA

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.