Evaluating the impact of digital health information: A scoping review of measures (Preprint)

Le résumé fourni par la source

BACKGROUND High-quality, reputable digital health information (DHI) provides a cost-effective and widely accessible means to enhance health professionals' knowledge and empower individuals to make informed health decisions. However, with the growing proliferation of digital resources, there is increasing pressure for DHI providers to demonstrate the impact of their resources and document measurable value. While systems data is analysed to propose metrics for assessing user engagement and website usability, identifying and observing broader effects of accessing DHI such as knowledge acquisition or behavioural changes remains challenging. OBJECTIVE To identify and map the methods and measures used to determine the impact of DHI within the published literature. METHODS We conducted a systematic scoping review of English-language studies published since 2013 using Ovid Medline and CINAHL databases and hand searched the Journal of Medical Internet Research (JMIR) suite of titles. Due to the lack of a clear definition of ‘impact’ in the context of digital information, we broadly defined it as any measurable change or action resulting from engagement with DHI. Our search strategy covered four concepts related to the described DHI: (1) type of information resource, (2) its purpose, (3) evaluation approach, and (4) impact. Included studies evaluated DHI products such as knowledge hubs, toolkits, or elearning resources, provided users accessed them independently and asynchronously. Citations and full text papers were dual reviewed with conflicts resolved through discussion. Data were extracted using a pre-designed template. RESULTS Of 2762 articles retrieved, 333 were included in the review. Studies primarily investigated the impact of access to DHI on physical (26.7%), mental (15.0%), lifestyle (13.8%) and professional development (11.1%) factors, targeting patients (31.5%) and health professionals (27.0%). Randomised controlled trials were the most common design (42.6%), with outcomes measured using validated (50.2%) or non-validated repeated measures (42.3%). Various terms were used to describe impact. Only 11.1% of studies used the term ‘impact’ to describe outcome changes, 42% used ‘effect’ while 5.1% focussed on measuring acceptability or feasibility. Impact on individuals’ knowledge or behaviour predominated with only 3.3% of studies exploring change or benefits at the organisational, economic, or societal level. CONCLUSIONS Despite the widespread availability of digital health information and its potential to influence individuals, organizations, and societies, our review found a limited range of methods and measures used in the biomedical research literature to assess its impact. Approaches from public health might offer valuable insights for evaluating broader impacts beyond the individual level. A framework that aligns various metrics with the specific goals of digital health resources could enhance their evaluation. By carefully selecting and applying these metrics, organisations that fund digital health information projects may better understand the effectiveness of the resource and make informed decisions to justify expenses and improve the impact.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.