Accurate estimation of interobserver variability in echocardiography segmentation measured from 35 experts and impact on derived clinical metrics

Le résumé fourni par la source

Abstract Introduction The delineation of cardiac structures is crucial to the analysis of echocardiography images and a prerequisite for the calculation of clinical metrics derived from heart chamber areas and volumes. Nevertheless, significant interobserver variability exists due to anatomical variability and echocardiography image quality, leading to discrepancies in endocardial border tracing [1]. Estimations of interobserver variability reported in the literature are based on measurements from a reduced number of annotations per sample [2]. Purpose This work aims to elaborate on previous interobserver variability studies presented in the literature to improve the accuracy and robustness of the estimations by considering (1) a large cohort of annotators and (2) a controlled environment, where the interobserver variability is isolated from other sources of variability, such as image acquisition, cardiac cycle selection or labelling method. Methods A group of 35 experienced sonographers independently delineated the heart chambers at end-systole (ES) and end-diastole (ED) in apical 4-chamber (A4C) and 2-chamber (A2C) views from a cohort of 10 patients (median age 72, IQR: 52-77; predominantly male, 70%), resulting in a total of 120 corresponding annotations per sonographer. The ES and ED frames were preselected by a clinical committee for each sequence to remove this source of variability. Segmentations of each sonographer were used to derive the area of the heart chambers, along with several clinical metrics routinely used to assess cardiac morphology and function. Variability was quantified via relative standard deviation (RSD) and intraclass correlation coefficient (ICC). Results The analysis confirmed a large interobserver variability, with a global RSD of 15.4±5.2% and ICC of 0.89 in area (Figure 1). Variability was significantly higher at ES phase (p<0.001) and in A2C view (p=0.02). No significant differences were found across heart chambers. The annotation variability directly affected the calculation of clinical metrics, leading to particularly large RSD values for those metrics derived from multiple annotations (Figure 2). The variability of left ventricular ejection fraction (LVEF) was 16.0±3.4%, in line with the literature [3]. However, the LVEF ICC was highly compromised (0.187), suggesting that the interobserver variability is larger than the variability of the metric itself. This raises concerns about the use of LVEF derived from manual segmentation as a robust diagnostic reference [4]. These findings argue the need for enhanced standardization protocols and a swift towards automated annotation that mitigate this variability, as evidenced in the EchoNet trial [5]. Conclusion This study accurately evaluates the echocardiography segmentation interobserver variability and its impact on clinical metrics. The findings confirm a large variability and highlight the need to mitigate it, ultimately improving diagnostic assessment.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.