Evidence in Digital Health for EFfectiveness of INterventions with Evaluative Depth (Evidence DEFINED)

The new standard of excellence framework for clinical assessment of digital health products (DHPs).

Without the proper evidence for DHPs, stakeholders across the healthcare ecosystem, including payers, will encounter obstacles that can result in companies being unable to bring their products to market. Evidence is needed to determine the reliability and value of DHPs and ensure commercial success.

To help, DiMe collaborated with healthcare stakeholders to create Evidence DEFINED to define ‘what good looks like’ when it comes to evidence evaluation.

Access Evidence DEFINED framework

View all the resources

Learn more about the Evidence DEFINED framework and how it:

Offers payers, employers, health systems, and other stakeholder users a harmonized, rigorous, rapid approach to assessing the clinical value of DHPs
Underlies the evidence to help payers make quicker coverage decisions on DHPs being presented to them daily to ensure good market access for innovative health interventions
Helps DHP companies navigate their commercial strategy and demonstrate the value of their product to payer audiences

Meet the experts who developed the Evidence DEFINED framework

A group of 17 experts with different disciplinary backgrounds collaborated to develop the Evidence DEFINED framework. This sprint team represented experts from a variety of different work settings and multiple regulatory and geographic regions.

Meet the team:

Jordan Silberman, Paul Wicks, Smit Patel, Siavash Sarlati, MD, Siyeon Park, Igor O. Korolev, Jenna R Carl, Jocelynn T. Owusu, Vimal Mishra, Manpreet Kaur, Vincent J. Willey, Madalina L. Sucala, Tim R. Campellone, Cindy Geoghegan, Isaac R. Rodriguez-Chavez, Benjamin Vandendriessche, and Jennifer C. Goldsack

We would love to hear how you have used, or plan to use, Evidence DEFINED to determine the reliability and value of your digital health products. Please share your story with us, and we’ll create a case study for you to showcase your leadership in this space!

Share your Story

Evidence DEFINED Resources

Quick Start Guide

A process overview for the Evidence DEFINED Framework

Download or share this resource:

Definitions Matter

Digital health products are patient-facing products that meet the criteria shown below. Use the table to help define the product you are using.

1. The product falls into one of the three classes of digital health technologies that were defined in a collaboration of stakeholders representing digital health trade organizations.

Product Class	Product Class Definition
Product Class: Digital Health	Product Class Definition: Digital health includes technologies, platforms, and systems that engage consumers for lifestyle, wellness, and health-related purposes; capture, store or transmit health data; and/or support life science and clinical operations.
Product Class: Digital Medicine	Product Class Definition: Digital medicine includes evidence-based software and/or hardware products that measure and/or intervene in the service of human health.
Product Class: Digital Therapeutics	Product Class Definition: Digital therapeutic (DTx) products deliver evidence-based therapeutic intervention to prevent, manage, or treat a medical disorder or disease.

2. The product is designed to change one or more health behaviors.

3. The value of the product to the evaluator is contingent on the degree to which it improves one or more health outcomes. These can include clinical outcomes (eg, incidence of diabetic retinopathy) or surrogate outcomes (eg, HbA1C).

Download or share this resource:

Analysis of Prior Evidence Assessment Frameworks

A total of 78 prior frameworks were identified as relevant by the Evidence DEFINED team. Prior frameworks are not sufficiently adapted to address evidence considerations unique to digital health, creating ambiguity for various stakeholders. This table includes frameworks that assess evidence alone, or evidence as well as other domains of assessment.

View Supplementary Figure 1: PRISMA 2020 flow diagram to learn how we identified frameworks for inclusion into the directory above.

Download or share this Resource:

Checklist of Evidence Quality Criteria

This checklist was designed to supplement established evidence assessment frameworks.

Download or share this Resource:

Evidence-to-Recommendation Guidelines

These guidelines suggest degrees to which the adoption of DHPs may be warranted based on clinical evidence.

Actionability Level	Criteria	Adoption level that may be appropriate	Approx. enrollment that may be appropriate
Actionability Level: 0	Criteria: One or more of the following: Clear evidence of harm or ineffectiveness for the current DHI version The DHI is not clinically appropriate, per advice of clinical subject matter experts. The risk balance is unfavorable due to safety concerns, per subject matter experts. There are unaddressed concerns regarding misleading or false claims.	Adoption level that may be appropriate: Adoption not recommended.	Approx. enrollment that may be appropriate: N/A
Actionability Level: 1	Criteria: All of the following: Very low or low-quality evidence (per GRADE definitions; “very low” includes no evidence) Low clinical risk or well-managed risk with appropriate clinical rationale Plausibility of clinically meaningful impact relative to usual care (or an alternate, relevant comparator) OR noninferior clinical outcomes with plausible improvement in a domain such as access, health equity, user experience, or cost. Clinically meaningful impact is defined by an effect size magnitude at or above a minimal clinically important difference, as established in credible guidelines and/or peer-reviewed literature.	Adoption level that may be appropriate: Feasibility Pilot: Focus is on enrollment, engagement, user experience, safety.	Approx. enrollment that may be appropriate: N ≤ ~100
Actionability Level: 2	Criteria: All of the following: Meets or exceeds all criteria for Actionability Level 1 Low-to-moderate quality evidence (per GRADE definitions). Real-world evidence may be included. No or minimal uncertainty (per GRADE) around value to stakeholders (often patients and their families) Acceptable or likely acceptable (per GRADE) to stakeholders	Adoption level that may be appropriate: Small Clinical Pilot: Primary outcomes are clinical.	Approx. enrollment that may be appropriate: Up to several hundred.
Actionability Level: 3	Criteria: All of the following: Meets or exceeds all criteria for Actionability Levels 1-2 Moderate-to-high quality evidence (per GRADE). Real-world evidence may be included.	Adoption level that may be appropriate: Large Clinical Pilot: Primary outcomes are clinical.	Approx. enrollment that may be appropriate: ~300 ≤ N ≤ ~3,000
Actionability Level: 4	Criteria: All of the following: Meets or exceeds all criteria for Actionability Levels 1-3 Two or more high-quality RCTs support efficacy and safety Preferred: One or more RCTs have 3rd-party data monitoring and analysis Preferred: Real-world evidence of safety and effectiveness	Adoption level that may be appropriate: May be appropriate to scale.	Approx. enrollment that may be appropriate: No limit for appropriate patients.

Download or share this Resource:

Get Involved!

Catch the replay!

Watch as the co-authors showcased and discussed Evidence DEFINED during a DiMe Journal Club. Tune into the recording or access the presentation slides.

In this journal club, the co-authors came together to showcase the framework, which offers payers, employers, health systems, and other users a harmonized, rigorous, rapid approach to assessing the clinical value of DHPs and bringing them to market faster.

Watch the recording

Engage in an upcoming project

If you would like to engage in DiMe’s upcoming IEP project and help streamline the path to regulatory and commercial success to optimize health outcomes for the greatest number of patients, learn more and share your interest to join:

Submit Form