The Role of Mimicry in Defining Statistical Health

Résumé fourni par la source

The interpretation of quantitative pathology results relies on the comparison of results to either (a) the typical distribution of results in apparent health (reference intervals) or to (b) the typical distribution of results in disease (clinical decision limits) (1). Supporting clinical decisions should be the priority, and while these limits are prevalent in diabetes and dyslipidaemia, most interpretative limits are based on reference intervals. Direct reference interval studies rely on recruiting apparently healthy individuals, but this is no easy task, especially given the high prevalence of subclinical disease in the community. Reference interval studies need both questionnaires and exclusion testing to allow rejection of conditions that will cause undesirable heterogeneity in the reference distribution. If we accept that at the end of this selection process we need 120 validated reference individuals, the numbers can become overwhelming when we may also need 120 of each gender, or 120 of both pre- and postmenopausal women, or 120 children at each stage of development, or 120 pregnant women at each stage of gestation. The task becomes almost impossible, especially for routine laboratories. The Clinical Laboratory Standards Institute (CLSI) is currently reviewing the C28-A3 standard (2) for establishing and verifying reference intervals, and while 2 separate publications (one for defining and one for verifying) are likely to be proposed, the difficulties of a priori selection of apparently healthy reference individuals will remain, and this will remain a resource intensive endeavor for assay manufacturers and users. The new standard will probably not address clinical decision limits, especially because this requires significant collaboration with clinicians, who are essential when defining disease states and the relevant clinical questions and decisions. Indirect methods to establish reference intervals use laboratory data a posteriori and aim to disentangle the underlying uniform distribution that represents individuals unaffected by disease (3). Compared to the effort and cost of direct methods, indirect methods represent an efficient, cost-effective, and easy-to-use alternative for laboratories to establish, or verify, reference intervals. Numerous indirect methods have been proposed over the years, and they vary in both the principles of identifying the reference distribution as well as their assumptions on the potential shape(s) of the reference distribution, e.g., normal vs log normal vs Box–Cox- transformed. Until now, no comprehensive and objective evaluation has existed that could showcase the strengths and weaknesses of the different approaches. In this issue, Ammer et al. (4) have developed a software tool that can objectively assess the different indirect approaches and hopefully also facilitate improvement of existing, or development of future, methods. Their “RIBench” software has been written using open-source R statistical programming language and will be available through the Comprehensive R Archive Network (CRAN). R is arguably the language of choice for shared, exploratory statistical studies in the clinical chemistry domain. The article in this issue compares 5 indirect methods against an array of hypothetical overlapping healthy and diseased distributions and establishes some important principles regarding the indirect methodologies. The largely historical Hoffman indirect method cannot even tolerate 5% disease prevalence and even the most sophisticated modern methods cannot reliably identify the reference population when disease is more prevalent than health in the dataset (i.e., >50% of the overall distribution has pathology). When disease prevalence is below 30%, modern methods can provide reliable reference interval estimates with as few as 1000 laboratory results. Furthermore, at such low disease prevalence, they seem to outperform the imagined gold standard of direct methods using 120 results. When reference distributions are skewed, more laboratory data are required (n ≥ 5000) to use indirect methods; direct reference limit determinations with skewed distributions also require the use of larger quantities of data for high confidence. Heavily skewed data sets increase the likelihood of failure, and for distributions that are skewed and shifted, no indirect method can achieve results that are even comparable to the direct method regardless of disease prevalence. The predicted shape of the reference distribution is a critical issue. A reference population is assumed to be homogeneous, evidenced by the requirement that in the presence of known heterogeneity due to age or gender, partitioning must be used to maintain the (95%) specificity definitions of the reference limits. Understanding the impacts of physiology on reference intervals is a pre-requisite to both direct and indirect reference interval studies (5). Whenever skewing is identified, it is vital to ensure that skewness is not due to physiological or sampling heterogeneity. The authors acknowledge that inadequate partitioning is a real-world issue and recognize that their synthetic test sets assume optimal input conditions, whereas in most real-world settings a thorough data pre-filtering will be required to achieve best possible results, which is beyond the scope of the benchmarking suite. I am personally concerned that sophisticated transformations available in the software can normalize most skewed real-world distributions but may mask underappreciated inadequate partitioning. In a real-world scenario, the unimodal assumptions of the software would require, at a minimum, appropriate physiological partitioning and would contain only one measurement per subject. The authors accept that their simulations are not capable of representing suboptimal real-world scenarios. The assumption of homogeneity of the reference population distribution is also important when it is also assumed for the pathological population distribution. The RIBench software currently assumes homogeneity of the pathological population; however, disease can be clinically heterogenous because disease can have multiple etiologies, and disease often represents a continuum of stages from subclinical to end-stage. This potential multimodal nature of pathology is further impacted by the frequency of testing in different forms and stages of disease. Nevertheless, I would expect that the shape of the pathological distributions is of secondary importance compared to the shape of the reference distribution, but this could be tested in future studies using multimodal pathological distributions. The authors hope to receive feedback regarding RIBench from users and readers of Clinical Chemistry to gain more insights into potential useful distribution types that could broaden the scope of the proposed benchmark in future versions. While indirect methods can provide robust results in a wide variety of settings, challenging data sets can result in reference interval estimates with large deviations and it is therefore highly recommended that all results from indirect reference interval analysis be critically assessed by experts in laboratory medicine who can detect implausible results. While some recommendations regarding the principles of indirect reference interval methods have been published (6), the new CLSI standards for establishing or verifying reference intervals are unlikely to provide detailed guidance on the best way to deploy indirect methods because there is not any consensus on the optimal pre-processing of laboratory data, let alone the optimal statistical approaches to identify the underlying reference distribution. RIBench represents an important advance because we now have a tool that allows comparison of statistical methods to explore and refine the robustness of those methods in indirect reference interval determinations. All authors confirmed they have contributed to the intellectual cont