Impact of interobserver variability of echocardiography derived clinical metrics on sample size estimation in clinical trials

Le résumé fourni par la source

Abstract Introduction Clinical metrics obtained from echocardiography usually involve manual analysis, where annotation introduces variability and bias, reducing the reliability of the findings [1]. Traditionally, sample size estimations consider patient variability but often overlook interobserver variability, leading to underpowered and suboptimal studies and an increased risk of clinical study failure [2]. Purpose This study aims to assess the power reduction due to a neglected error sourced in the interobserver variability and its impact on clinical trial size. Methods Interobserver variability was evaluated in an in-house cohort (median age 72, IQR: 52-77; predominantly male, 70%) by recruiting a set of 35 raters, considerably larger than previous studies. Ventricular and atrial endocardium were delineated at end-diastole (ED) and end-systole (ES) in apical 4-chamber (A4C) and 2-chamber (A2C) views, totalling 120 independent annotations per rater. Standard clinical metrics were accordingly derived, and interobserver variability was calculated as the average of the raters’ standard deviations across patients. Population variability was estimated from this cohort and from SHaRe (hypertrophic cardiomyopathy with systolic dysfunction) and Miyazaki (myocardial infarction) registries [3,4]. Sample sizes were estimated using power analysis (two-sided alternative hypothesis assumption) for an independent two-sample t-test, considering a pooled standard deviation combining population and raters’ variability. Results The sample size required for all the evaluated metrics to maintain a given power level after adjusting for the interobserver variability was significantly higher (52.7% average increase). This increase was up to 3-fold for left ventricular ejection fraction. Likewise, the adjusted sample size doubled when considering population variability from the SHaRe and Miyazaki cohorts (Table 1). In the cases where the estimated increase was relatively small (e.g. 5% for left atrium volume), the absolute number of additional patients was still substantial, around several hundred, depending on the expected effect size. The sample size increases exponentially with the inverse of the effect size, and the proportional increase in the adjusted sample size is roughly the same for a wide range of effects (Figure 1). Therefore, reducing the interobserver variability has a major impact on the sample size for any expected effect. Only when the interobserver variability relative to the population one falls below 25% can the effect of the interobserver variability be neglected in the trial size calculations. Conclusion This study evaluates the impact of interobserver variability on trial size estimation. The findings highlight the importance of this source of variability and the elevated risk of underpowered studies when neglected. This ultimately argues for a swift towards automated echocardiography analysis to reduce interobserver variability.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.