Preliminary assessment of the data appropriateness of electronic health records for real-world studies: A survey of hospitals in a developing region in China

Résumé fourni par la source

To the Editor: Assessment of data appropriateness is a process to answer whether electronic health records (EHRs) from routine healthcare practices could fit intended study purposes, for example, available to be proceeded, with enough individual records, with relevant information able to be extracted from records, etc.[1–3] This has been increasingly underscored as a prerequisite when using EHRs (one important type of real-world data [RWD]) for scientific purposes.[4–6] Specifically, coverage, consistency in variable types, comparability between multicenter databases and accessibility to databases have been most proposed for the preliminary stage of assessment on the variable level.[1] Although controversies remain on the definitions, types and assessment methods of the dimensions of RWD appropriateness, the latest official guidelines (i.e., from the U.S. Food and Drug Administration [FDA][4] and China National Medical Products Administration [NMPA][6]) suggest that the assessment start from a preliminary stage on variable existence, and subsequently deepen into the issue of data value (such as missing value, outliers, etc.), as the preliminary assessment is the foundation of the overall RWD appropriateness.[7] Little is known about the appropriateness of EHRs in developing regions, which feature high visit volumes and a great number of hospitals. Therefore, this study aimed to preliminarily investigate RWD appropriateness from hospitals in developing regions from the perspective of core variables’ comparison. This study was conducted from March to May 2021 in Fuzhou City, Jiangxi Province, a typical developing city in China (ranked medium in national gross domestic product [GDP] performance 2020) and famous for traditional Chinese medicine (TCM). We defined EHR databases from each hospital as both clinically and administratively related information systems, such as electronic medical records (EMRs), laboratory information management systems (LIS), picture archiving and communication systems (PACS), etc. All hospitals in this developing region (n = 28, including 25 secondary and 3 tertiary hospitals) were investigated through chief administrators of hospital EHR systems after signing written informed consent. This study was approved by the Ethics Committee of Peking University Institutional Review Board (IRB00001052-21047). Basic characteristics of hospital databases (database composition, quality management, degree of paperless documentation and restoration, and expanded functions of pop-up operations on EHR databases) and three dimensions of data appropriateness (coverage, consistency, accessibility) were evaluated by face-to-face interviews. All analyses and figures were conducted using R version 4.3.0 (R Foundation for Statistical Computing, Vienna, Austria). This study showed favorable coverage of core variables from EHRs (overall coverage of clinically relevant variables >75%); however, inconsistent data types and standardized coding systems hindered practical RWD utilization. Furthermore, reluctance to share data might aggravate data isolation in heterogeneous EHR databases. In China, the incompatibility of hospital information systems developed by hundreds of individual providers induced and aggravated the inconsistency in data structuring, types, and coding systems, which has been reflected in this study. The lack of a normalized medical terminology system in China would raise information bias, such as disease misclassifications, in the process of exchanging data among standardized coded variables. Although unstandardized and uniform data could be largely solved by using common data models and other data management processes, the underlying solution for data quality is always to ensure that raw data are documented from routine medical practices. This requires the standardized input of clinical data by clinicians and other data collectors. For the latter, regular inspections and staff training are crucial guarantees for EHR appropriateness. Most chief administrators of hospital EHR systems were reluctant to share data [Table 1], considering ethical worries, technical difficulties (i.e., low data appropriateness, colossal expenditure, and time for data processing tool implementation), and inadequate impetus from pilot hospitals, healthcare authorities, and enterprises. Table 1 - Willingness to data sharing and scientific cooperation in 28 hospitals. Items Unwilling of data sharing and reasons [N (%)] Willing to data sharing [N (%)] Total Concerns on administrative approval Concerns on privacy preservation Concerns on insufficient EHRs technique Accepting data preprocessing 19 (67.9) 9 (47.4) 2 (10.5) 7 (36.8) 9 (32.1) Sharing de-identified data 18 (64.3) 9 (50.0) 3 (16.7) 5 (27.8) 10 (35.7) Applying de-identified data to RWS and RWE publication 19 (67.9) 9 (47.4) 2 (10.5) 7 (36.8) 9 (32.1) Among 28 hospitals, 1 hospital was unwilling to data sharing and scientific cooperation. This unwilling hospital only responded to the three multiple choices but did not give a specific reason, thus causing the missing (sum of categorical reasons for unwillingness ≠ the total number of unwilling hospitals). EHRs: Electronic health records; RWE: Real-world evidence; RWS: Real-world study. This study conducted a preliminary assessment of RWD appropriateness from the perspective of variables, in comparison with sentinel common data model (SCDM) and Observational Medical Outcomes Partnership common data model (OMOP CDM), from all hospital EHR databases in one developing region in China. For unintegrated but indispensable EHR databases in developing regions, this study provides a framework for evaluating RWD databases regardless of specific scientific purposes, as the in-depth assessment of data value needs to take concrete clinical medical scenarios into consideration.[7] This study had some limitations. First, EHRs from primary healthcare centers and developed regions were not considered, although the census on secondary and tertiary hospitals could represent the majority of medical records in Fuzhou, a medium GDP and healthcare level nationwide. Second, the assessment tools were limited to two CDMs (SCDM and OMOP CDM), which were most widely adopted, while other CDMs (i.e., European Union adverse drug reactions project CDM) were not included. Third, questionnaires were answered by primary custodians in hospitals’ informatics departments, and the assessment results inevitably relied on custodians’ understanding. Fourth, in addition to coverage, consistency, and accessibility, other dimensions of the two main aspects of RWD appropriateness (relevancy and reliability), such as accuracy and timeliness, were not explored in this study as our focus was on the variable comparison. Additionally, due to the unavailability of patient-level, longitudinal medical records from targeted hospitals, the coverage and consistency assessments were restricted to the proportion of qualified core variables or hospitals. In summary, this study attached importance to the RWD appropriateness of unintegrated hospital EHR databases in developing regions. Poor consistency and low willingness to share data were found and called for administrative improvements, otherwise threatening EHRs’ accessibility and interoperability. This requires the collaboration of the data administrator and the data generation and entry staff (usually clinicians). The establishment of data-integrating networks piloted in developing regions could be raised to overcome technical isolations and accelerate RWD utilization. Funding This study was supported by grants from the National Natural Science Foundation of China (Nos. 82173616 and 82003536), Peking University Medicine Fund of Fostering Young Scholars’ Scientific & Technological Innovation (No. BMU2022PYB035), Fundamental Research Funds for the Central University, Key Clinical Projects of Peking University Third Hospital (No. BYSYZD2021030), and Institute of

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.