Each month we will select a manuscript tackling an important topic in digital medicine and DiMe members can register to participate in an intimate discussion with the manuscript author(s).
Immediately after the session, the recording will be posted to a new Slack channel in DiMe’s Workspace. For the next week, we will moderate a discussion in Slack between all of DiMe’s members and the author(s).
DiMe Journal Club: Regulatory Pathways for Qualification and Acceptance of Digital Health Technology-Derived Clinical Trial Endpoints: Considerations for Sponsors
Despite widespread interest and substantial investment in the adoption of sensor-based digital health technologies (sDHTs) for remote data capture in drug development trials, no drug has been approved based on an sDHT-derived primary endpoint in the United States (US). One reason for this lack of advancement is the complexity of obtaining regulatory endorsement for those endpoints within current US regulatory pathways. The goal of our review is to describe the two choices currently available to pharmaceutical study Sponsors: (i) they may navigate the traditional route of compiling the evidence to support the sDHT-derived endpoint in their investigational new drug (IND) application, requiring specific expertise and substantial resources; or (ii) they may navigate the drug development tool (DDT) pathway with the goal of qualifying their sDHT-derived endpoint as a biomarker or clinical outcome assessment applicable to a broader context of use (COU), either alone or as part of a partnership or consortium. We describe the nuances of each pathway; the evidentiary requirements for supporting an sDHT-derived endpoint and the technology used to capture it; and the impact that an sDHT’s regulatory status may have on a Sponsor’s decision to use it for data capture. By systematically comparing the IND and DDT pathways, our over-arching goals are to support the increasing deployment of sDHTs within the clinical research setting and help advance regulatory science in the field of digital medicine.
Takeaway messages:
Sponsors considering incorporating a digital endpoint into a drug development pipeline may pursue the following regulatory pathways in the United States: 1) measure qualification via the Drug Development Tool (DDT) program; or 2) measure acceptance via the Investigational New Drug (IND) pathway. Multiple considerations may influence this critical decision, including drug development timelines, the context of use, risk sharing, and the need for partnerships or consortia.
Regardless of the pathway, the evidentiary package supporting a digital endpoint includes key elements to ensure the data are accurate, reliable, and appropriate for the specified context of use. Additional evidence is required to support the classification of the measure as a biomarker versus a clinical outcome assessment.
Sponsors may select a digital health technology that is A) a regulated medical device; B) a data capture product developed specifically for research purposes; or C) a low-risk general wellness product. While there is no requirement to use a product with a medical device designation for data collection in clinical trials, the scope of evidence that a Sponsor must compile to support their choice may vary depending on the technology category in addition to the context of use.
Watch to learn about the pros and cons of these decisions within the context of United States regulations.
Net financial benefits of digital endpoints publication in Clinical and Translational Science
Digital measures offer the clinical trials community unparalleled insights into people’s real-life experiences, enhancing our understanding of numerous conditions. So, why haven’t we implemented digital endpoints on a larger scale?
Findings from research conducted by DiMe, Tufts Center for the Study of Drug Development, Johnson & Johnson, Roche, Genentech, UCB, Bayer, Takeda, and MindMed, now published in the Journal of Clinical & Translational Science, showcase evidence of significant cost reductions and financial benefits tied to leveraging digital endpoints in trials.
DiMe Journal Club: Wearable Digital Health Technology
There is a growing body of evidence to support the integration of digital health technologies into clinical care, yet several roadblocks must still be addressed to fully realize the potential of DHTs to advance understanding of health and disease. Our work describes several of these challenges, including data ownership, patient trust and empowerment, data standards and interoperability, and return of value for health systems, and a call to action for the DHT community to align on solutions to ensure the greatest impact to patients.
Defining the Dimensions of Diversity to Promote Inclusion in the Digital Era of Healthcare
As conventional medicine shifts towards digital medicine, we have the opportunity to intentionally develop and deploy DHTs with an inclusion focus. The first step is ensuring that the multiple dimensions of diversity are captured; we propose a lexicon that encompasses elements critical for implementing an inclusive approach to advance healthcare quality and health services research in the digital era.
Listen to experts from the FDA, patient advocacy, and the life science and med-tech industries as they discuss why it is important that the intersectionality of the diversity dimensions is represented in healthcare and clinical research. The lexicon for the dimensions of diversity can inform strategies and resources for more inclusive digital product development and deployment.
DiMe Journal Club: The promise of artificial intelligence (AI) and machine learning (ML) for improving clinical outcomes
The paper addresses the challenges and potential solutions for developing and implementing clinical predictive algorithms (CPAs) in healthcare. The authors highlight that existing CPAs are failing to meet their potential due to an overreliance on the area under the receiver operating characteristic curve (AUC) and a lack of emphasis on performing rigorous clinical trials to quantify the benefits and risks. They propose a more holistic approach to optimizing the development of CPAs, akin to the development of new medicinal products, biomarkers, and other complex interventions.
The study suggests two key recommendations for improving the utility and value of CPAs: first, the authors advocate for considering prevalence of the characteristic of interest, positive predictive value (PPV), and negative predictive value (NPV) to assess the value and utility of the CPA, as these metrics are critical for interpreting CPA results for individual patients. Furthermore, the value of correct predictions must be weighed against the cost of false predictions, and the total value of the CPA must be estimated. Secondly, the authors propose a fit-for-purpose, sequential clinical development approach analogous to the development of new medicinal products, biomarkers, and other complex interventions. This approach involves careful consideration of clinical setting characteristics and the iterative and sequential optimization of CPAs through the selection of cut-off values and the evaluation of outcomes in clinical trials conducted in real-life hospital settings.
Incorporating Digitally Derived Endpoints within Clinical Development Programs by Leveraging Prior Work
Digital health technologies (DHTs) enable remote data collection, support a patient-centric approach to drug development, and provide real-time data in real-world settings. With increasing use of DHTs in clinical care and development, we expect a growing body of evidence supporting use of DHTs to capture endpoint data in clinical trials. As the body of evidence grows, it will be critical to ensure that available prior work can be leveraged. We propose a framework to reuse analytical and clinical validation, as well as verification data, generated for existing DHTs. We apply real life case studies to illustrate our proposal aimed at leveraging prior work, while applying the V3 framework (verification, analytical validation, clinical validation) and avoiding duplication. Utilizing our framework will enable stakeholders to share best practices and consistent approaches to employing these tools in clinical studies, build on each other’s work, and ultimately accelerate evidence generation demonstrating the reproducibility and value add of these new tools.
Large language model AI chatbots require approval as medical devices
By the end of 2022, large language models (LLMs) such as Open AI’s ChatGPT or Google’s PaLM had evolved to the point that they could fluently generate conversation-like text. Trained on a body of human-generated text from across news articles, literature, Wikipedia, and the wider Internet, users quickly began trying to use it to have medical conversations such as identifying the cause of a new symptom or seeking therapy-like support on mental health issues. As digital health and regulatory specialists, we argue that when LLMs enter the realm of medical decision making, they become medical devices, and must be regulated as such. While we agree the potential for such technology is significant, our work on numerous medical software applications deployed in clinical practice suggest that in their current form, LLMs risk significant harm and must not attempt to sidestep regulations. We believe this is an addressable challenge and encourage developers to be specific about their intended purpose, calibrate their development to the appropriate risk class, constrain open LLM prompts and responses to improve safety, and to pursue an intentional program of quality management throughout.
Nocturnal Scratch: A study bringing patient input into development of novel digital measures
As a result of successful DiMe collaboration, we have published the results of a mixed methods study that interviewed and surveyed patients with atopic dermatitis. The publication titled “A patient-centered conceptual model of nocturnal scratch and its impact in atopic dermatitis: A mixed-methods study supporting the development of novel digital measurements” is providing insights into everyday patient experience with atopic dermatitis.
Apart from signs, symptoms and effects on everyday life, we explored patients’ attitudes towards evaluation of nocturnal scratching as well as use of digital tools for its measurement. These results can further serve as a basis for development of patient-centered clinical outcome assessments that leverage digital health technologies (DHTs).
DiMe Journal Club: Digital health technology derived measures: Biomarkers or clinical outcome assessments?
Digital health technologies (DHTs) present unique opportunities for clinical evidence generation but pose certain challenges. These challenges stem, in part, from existing definitions of drug development tools, which were not created with DHT-derived measures in mind. DHT-derived measures can be leveraged as either clinical outcome assessments (COAs) or as biomarkers since they share properties with both categories of drug development tools. Examples from the literature indicate a variety of applications for DHT-derived data, including capturing disease physiology, symptom tracking, or response to therapies. The distinction between the categorization of DHT-derived measures as COAs or as biomarkers can be very fine, with terminology variability among regulatory authorities. This has significant implications for integration of DHT-derived measures in clinical trials, leading to confusion regarding the evidence required to support these tools’ use in drug development. There is a need to amend definitions and create clear evidentiary requirements to support broad adoption of these new and innovative tools. The biopharma industry, the technology sector, consulting businesses, academic researchers, and regulators need a dialogue via multi-stakeholder collaborations to clarify questions around DHT-derived measures, to unify definitions, and to create the foundations for evidentiary package requirements, providing a path forward to predictable results.
Evidence DEFINED Framework – A Rigorous, Rapid Approach to Assess the Clinical Value of Digital Health Interventions I Public Launch Event
Without the proper evidence for digital health products (DHPs), there are obstacles that payers encounter when evaluating these, which can result in companies being unable to bring their products to market. Evidence is needed to determine the reliability and value of DHPs.
As the expert in ethical, effective, equitable, and safe use of digital medicine, the Digital Medicine Society (DiMe) collaborated with healthcare stakeholders to create a new harmonized framework, Evidence in Digital Health for EFfectiveness of INterventions with Evaluative Depth (Evidence DEFINED). The framework offers payers, employers, health systems, and other users a harmonized, rigorous, rapid approach to assessing the clinical value of DHPs and bringing them to market faster.
How V3 might be applied to wearable dosimeters and light loggers
Light exposure is an important driver and modulator of human physiology, behavior and overall health, including the biological clock, sleep-wake cycles, mood and alertness. Light can also be used as a directed intervention, e.g., in the form of light therapy in seasonal affective disorder (SAD), jetlag prevention and treatment, or to treat circadian disorders. Recently, a system of quantities and units related to the physiological effects of light was standardized by the International Commission on Illumination (CIE S 026/E:2018). At the same time, biometric monitoring technologies (BioMeTs) to capture personalized light exposure were developed. However, because there are currently no standard approaches to evaluate the digital dosimeters, the need to provide a firm framework for the characterization, calibration, and reporting for these digital sensors is urgent.
This article provides such a framework by applying the principles of verification, analytic validation and clinical validation (V3) as a state-of-the-art approach for tools and standards in digital medicine to light dosimetry.
Defining digital measurement of scratching during sleep, or “Nocturnal Scratch”: A review of the Literature
Atopic dermatitis affects up to 2.4% of the world’s population, with nighttime scratching being a predominant and burdensome symptom for patients. This publication details DiMe’s systematic literature review to standardize nocturnal, scratch, and itch definitions as part of our work to drive the adoption of nocturnal scratch as a digital endpoint for atopic dermatitis.
Digital sensing solutions can assess the severity of atopic dermatitis symptoms, mainly scratching at night. Prior to DiMe’s work, a lack of understanding of this disease has made it difficult to compare which digital sensing solutions are best fit to measure these symptoms. Now, with the proper knowledge of this disease and a shared understanding of its definitions, clinical researchers and drug developers can collect essential information about patients’ nighttime scratching and measure and quantify this behavior in real-time using the correct digital tools and sensor-generated data.
When using digital health technology (DHT) in clinical trials, we need to specify an endpoint. However, does your digital endpoint matter to the patients? Is it based in the context of their life? For regulatory purposes this seems to be an imperative. As such “black box” algorithms, despite their many benefits such as high predictive accuracy, could be a barrier to gaining a label claim.
This paper discusses the difference between a digital biomarker and a digital clinical outcomes assessment. We build on the foundations by focussing on the meaningful aspects of health and suggesting a framework for DHT development based on the DiMe Playbook. We also go one step further in this talk than we did in the paper. Here we start blue sky thinking about how to combine the highly predictive algorithms underlying digital biomarkers with the patient relevant aspects of digital clinical outcomes assessments.
Advancing Digital Health Innovation in Oncology: Priorities for High-Value Digital Transformation in Cancer Care
Although health care delivery is becoming increasingly digitized, driven by the pursuit of improved access, equity, efficiency, and effectiveness, progress does not appear to be equally distributed across therapeutic areas. Oncology is renowned for leading innovation in research and care; digital pathology, digital radiology, real-world data, next-generation sequencing, patient-reported outcomes, and precision approaches driven by complex data and biomarkers are hallmarks of the field. However, remote patient monitoring, decentralized approaches to care and research, “hospital at home,” and machine learning techniques have yet to be broadly deployed to improve cancer outcomes.
New findings from the Digital Medicine Society and Moffitt Cancer Center’s multi stakeholder roundtable discussion with leading experts in cancer care and digital innovation are available in the JMIR viewpoint publication linked below. Roundtable participants and viewpoint co-authors found that the digital innovation in oncology lags behind other therapeutic areas and that multiple persistent challenges in oncology research and care delivery can be disintermediated with effective digital solutions.
During this journal club, thought leaders from leading oncology care centers and research institutes, digital health solution providers, and patient groups in the digital health cancer arena will reflect on the findings of the viewpoint and share insights for strategies and actions needed to bring the promise of digitization to cancer treatment to improve lives.
Participants discussed the promise of digital solutions for reducing the cost and administrative burden of advanced cancer care and for supporting and accelerating access to precision oncology treatments. Participants also shared more about upcoming work co-sponsored by DiMe and the Moffitt Cancer Center focused on the advancement of cancer care and research through digital innovation.
Applicability of Artificial Intelligence in Healthcare in Resource-Poor Settings
This article focuses on institutional and resource constraints that have held back innovation and the scaling up of Artificial Intelligence (AI) in many Low and Middle Income Countries (LMICs). Given the proper infrastructure, AI-driven interventions hold promising transformations for public health in resource-poor countries. The results confirm the potential of startups implementing AI in resource poor settings such as Asia and Africa. Additionally, it highlights several prerequisites to a robust healthcare system: clarity in policies and regulations, well-defined roles and responsibilities of government bodies, political stability, and high investment in primary healthcare. Data collected and literature review on AI companies in LMICs are widely published. To gain deeper insights on current challenges in adoption and implementation of AI, we interviewed healthcare startups in LMICs that have the potential for a global outreach.
When Work Hits Home: The Cancer-Treatment Journey of a Clinical Scientist Driving Digital Medicine
Patients’ input on outcome measures, measurement tools, and a general understanding of experience going through treatment has become increasingly important for scientific communities, drug developers, and regulators. We discussed a unique example of a treatment journey of a patient who is also a scientist. This patient shares her experience of digital self-monitoring after a suspected cancer diagnosis and treatment. The patient assembled a digital kit at home, enabling the collection of vital signs and physical activity data, kept a diary, and reviewed the wearable data, sharing the results with treating physicians. The patient’s data set was also analyzed to identify data parameters that could be of utility for future research aimed at improving remote care delivery and, in addition, to understand the generalizability of the N=1 results. This is a rare opportunity to derive insights about digital measures of health status with direct access to the patient who is uniquely qualified to collect data and support interpretation of the results.
Should Alexa diagnose Alzheimer’s?: Legal and Ethical Issues with At-Home Consumer Devices
Voice-based AI-powered digital assistants, such as Alexa, Siri, and Google Assistant, present an exciting opportunity to translate healthcare from the hospital to the home. But building a digital, medical panopticon can raise many legal and ethical challenges if not designed and implemented thoughtfully. This paper highlights the benefits and explores some of the challenges of using digital assistants to detect early signs of cognitive impairment, focusing on issues such as consent, bycatching, privacy, and regulatory oversight. By using a fictional but plausible near-future hypothetical, we demonstrate why an “ethics-by-design” approach is necessary for consumer-monitoring tools that may be used to identify health concerns for their users.
Join the #Askmeanything style discussion with author David Simon, Ph.D., J.D., LL.M., Research Fellow at the Petrie-Flom Center for Health Law Policy, Biotechnology, & Bioethics, Harvard Law School.
SciKit Digital Health: Python Package for Streamlined Wearable Inertial Sensor Data Processing
SciKit Digital Health (SKDH) is a compilation of algorithm implementations based on previous work. The goal of SKDH is to provide commonly used algorithms in mobility research from wearable inertial sensors under a common framework, with sensible defaults, and an easily extensible framework that allows for customization based on the end-users needs. To this end, SKDH provides modules for data ingestion, pre-processing, and processing in gait, sit-to-stand, activity, and sleep, making going from raw data to digital health biomarkers fast and easy. Authors Yiorgos Christakis, MSc and Lukas Adamowicz, MSc, I Quantitative Scientists at Pfizer presented their publication, ” SciKit Digital Health: Python Package for Streamlined Wearable Inertial Sensor Data Processing at DiMe’s October Journal Club in an ask me anything format.
Considerations for Conducting Bring Your Own “Device” (BYOD) Clinical Studies
Advances in digital technology and increasing competitive space have led to a steady exploration and increase in the number of trials that are focused on a bring-your-own-device (BYOD) strategy. The BYOD approach not only is a cost and resource-effective measure of data collection, but it also saves the need to source and provision devices. While this provides a very user-friendly avenue to conduct trials, the absence of formal guidance and various concerns associated with BYOD trials has led to the present body of work. This workstream has explored in depth the different segments that need to be considered while designing a BYOD trial. Watch the recording now!
Quantifying the Benefits of Digital Biomarkers and Technology-Based Study Endpoints in Clinical Trials: Project Moneyball
Have you ever been asked what tangible benefits digital biomarkers bring to clinical studies? Inspired by the book and film Moneyball, the authors developed a novel method to quantify the values of patient selection and endpoint technologies using Monte Carlo simulation. Their work illustrates the model and potential application using Parkinson’s Disease example. This presentation will discuss:
1. The project rationale and Moneyball metaphor
2. SV95C and its clinical study impact
3. Monte Carlo approach and the model
4. The results and the way forward
Moving from “Should Do” to “How To”: A Deep Dive into DATAcc Toolkits for Inclusivity
Learn more about the key tools in the DATAcc Toolkits for Inclusivity – what they are, how they can be used and the impact they can have on your work in the development or deployment of digital health measurement products – in a conversation with four of the experts involved in creating them.
We’ll spend the first half discussing the Toolkit for Inclusive Development and the second half the Toolkit for Inclusive Deployment.
Regulatory Acceptance of Patient-Reported Outcome (PRO) Data from Bring-Your-Own-Device (BYOD) Solutions to Support Medical Product Labeling Claims
The June DiMe Journal Club featured “Regulatory Acceptance of Patient-Reported Outcome (PRO) Data from Bring-Your-Own-Device (BYOD) Solutions to Support Medical Product Labeling Claims.”
This presentation will outline the current landscape for BYOD adoption and why there is still hesitancy in utilizing this method to capture PROs in clinical trials. It will touch upon the only publicized example of a PRO endpoint captured electronically using BYOD as a primary safety outcome in a phase 3 pivotal trial. It will conclude with the proposal to set-up a database where sponsors can transparently contribute details about the PRO endpoints they have captured using BYOD, followed by a live Q&A.
Listen in to the discussion with Florence Mowlem, PhD Director, eCOA at Medable
Recommendations for Defining and Reporting Adherence Measured by Biometric Monitoring Technologies
Connected sensor technologies represent a potential solution to capturing adherence data accurately, objectively, and continually throughout a clinical trial. However, best practices for measuring and reporting adherence to digital health technologies are unclear
The DiMe Research Committee’s latest publication describes the methods and definitions by which adherence has been captured and reported using BioMeTs in recent years. Their work identified nine key recommendations for investigators planning on capturing and reporting adherence data using digital tools. Join us on May 12 at 11 am ET for a fantastic presentation on the Research Committee’s work and recommendations.
Identifying the Factors Influencing Patients’ Telehealth Visit Satisfaction: Survey Validation Through a Structural Equation Modeling
Patient experience and expectations have changed after the pandemic, as telehealth is becoming the new norm replacing in-person visits. At our April #AskeMeAnything Journal Club, our colleagues from the Abigail Wexner Research Institute at Nationwide Children’s Hospital will share 2 of their recent studies focusing on telehealth satisfaction survey development, implementation, and validation.
(1) The goal of this study was to assess patient satisfaction and assess the quality of services. The team developed a telehealth patient satisfaction survey (TPSS) with a multi-stakeholder group. In this first study, we shared survey components and explained how we developed the survey. We also proposed a conceptual research model to explain patient behavior towards telehealth satisfaction.
(2) In the second paper, the team validated their model, the telehealth patient satisfaction model, (TPSM) testing the factors (admission process, perceived quality of service) influencing patient satisfaction.
Advancing digital health applications: priorities for innovation in real-world evidence (RWE) generation
To accelerate innovation in evidence generation to support broad acceptance of digital health applications, DiMe and the health innovation hub (hih) of the German Federal Ministry of Health is releasing a manuscript end of February to define global priorities in evidence generation related to digital health applications that will speed the use of high-quality digital medicine products in routine care. Over the last year, an international group of researchers and experts in the use of real-world data and real-world evidence generation for digital medicine products joined a set of roundtable discussions and identified innovative approaches to digital medical product evaluation.
Join the co-authors of this manuscript in an open discussion and learn about the methodological challenges and opportunities for this emerging field of research, alongside examples of novel approaches to health evaluation research that can be readily applied to the evaluation of DiGA in Germany and novel digital health applications more broadly globally.
Sensor Data Integration: A New Cross-Industry Collaboration to Articulate Value, Define Needs, and Advance a Framework for Best Practices
The Digital Medicine Society recently launched a multi-stakeholder Sensor Data Integration Tour of Duty to address these challenges and more, provide a clear direction on how sensor data can fulfill its potential to enhance patient lives.
Data integration, the processes by which data are aggregated, combined, and made available for use, has been key to the development and growth of many technological solutions. In health care, we are experiencing a revolution in the use of sensors to collect data on patient behaviors and experiences. Yet, the potential of this data to transform health outcomes is being held back. Deficits in standards, lexicons, data rights, permissions, and security have been well documented, less so the cultural adoption of sensor data integration as a priority for large-scale deployment and impact on patient lives. The use and reuse of trustworthy data to make better and faster decisions across drug development and care delivery will require an understanding of all stakeholder needs and best practices to ensure these needs are met.
Listen in as the co-authors of this paper on sensor data integration in an open discussion and learn more about the new cross-industry collaboration with the aim to articulate value, define needs, and advance a framework for best practices.
Listen up! Using digital tools when designing patient-centric clinical trials
The use of digital health technologies in clinical studies introduces unique design complexities, as well as exciting opportunities, to measure what matters to patients. Using Parkinson’s as a case example, the 3DT (Digital Drug Development Tools) team of the Critical Path consortium shares recommendations that integrate the patient voice at every step of the way in order to maximize engagement, optimize recruitment, and increase retention and protocol adherence. The objective of this presentation is to overview a set of guidelines, recommendations, and considerations for integration of DHT, regardless of the type of device, in PD clinical studies in order to improve the overall study design and execution, with the engagement of patients as a key component of this process.
Listen in to the discussion with the panelists:
– Johan Hellsten, PhD, Senior specialist in Patient Insights (R&D), Lundbeck
– Martijn Müller, PhD, Senior Scientific Director, Critical Path for Parkinson’s, Critical Path Institute
– Cindy Zadikoff, MD, Medical Director & CPP Industry Co-Director, AbbVie
– Mark Frasier, MD, Chief Science Officer, Research Programs, Michael J. Fox Foundation
A patient’s health state can be characterized by a multitude of signals from many different data modalities. This high-dimensional, personalized data stream aggregated over patients’ lives has spurred interest in developing new clinical AI models. One of the rate-limiting factors in developing AI models that generalize to real-world scenarios is the very attribute that makes the data exciting—their high-dimensional nature.
At DiMe’s #AskMeAnything Journal, author Visar Berisha, PhD led a discussion on how “the curse of dimensionality” can doom models to failure, even when they seem to work well during development. We explored the key highlights of his Nature publication “Digital medicine and the curse of dimensionality“, Visar also provided some suggestions on how to develop clinical AI models that are more likely to fare well during prospective validation.
A Systematic Review of Recent Academic Research on Clinically-Relevant Digital Measures
The collaborators on The Playbook have published findings of a systematic review that identifies the need for more research funding related to digital clinical measures. Among all its findings, one that stands out is that in the last two years, out of 295 research studies published on digital clinical measures, there was only one academic publication reporting cybersecurity research, one study examining data rights and governance, and zero publications reporting research into the ethical implications of remote patient monitoring tools. Join the co-authors of this manuscript in a discussion about the current state of funding for academic research and help shape an integrated and coordinated effort across academia, academic partners, and academic funders to establish the field of digital clinical measures as an evidence-based field worthy of our trust.
“It takes a village: development of digital measures for computer scientists”
As digital measures become more complex, in terms of 1) the technologies, methods and data used to derive them, and 2) in terms of the aspects of health that they address, computer scientists, electrical engineers and other people coming from a data or computing background are increasingly important members of this village.
What can we do to bridge the gap from data-orientated background to clinical application of their skills? Can a unified lexicon aid communication throughout this process?
Ieuan Clay and Jen Goldsack presented their publication, “It takes a village: development of digital measures for computer scientists” and opened up a discussion on the range of challenges and considerations where computer scientists can have a particular impact on the development of a new digital measure.
Statistical considerations for successful digital health innovation
Why should you report your modeling plan or statistical analysis plan before seeing any data? Why should we all ditch the term ‘statistical significance’ but keep statistical evidence? And how? A fantastic discussion with Eric Daza, Lead Statistician for Digital Health Outcomes at Evidation Health, as he dives into key themes from his recent pieces: Artifice or intelligence? and Ditch ‘statistical significance’.
Our conversation explored two proposed research changes for our field: 1) Splitting all gathered data into a small number of random subsamples to test reproducibility/replicability of results; 2) For exploratory analyses, continue to report CI’s and p-values—but explicitly state as possible uncertainty to expect using new data.
Age and environment-related differences in gait in healthy adults using wearables
Do we fully understand the sensitivity of gait speed as a potential endpoint for clinical trials studies? Matthew D. Czech, Isik Karahanoglu, Xuemei Cai, Charmaine Demanuele, and their colleagues aimed to find out through their recent study, “Age and environment-related differences in gait in healthy adults using wearables” in NPJ. Their work shows that a single lumbar-worn sensor can be used for monitoring gait under free-living conditions and capture meaningful information about real-world functions that might not be possible in controlled settings. Their work also shows that despite higher variability, at-home gait speed was able to capture age-related group differences in healthy volunteers, which were not observed during in-lab gait assessments. Furthermore, they present the statistical methodology for deriving the number of monitoring days required to reliably estimate at-home gait speed that can be used to optimize clinical study design.
Science is catalyzed by new technology, but the process of technology adoption and implementation typically requires more time and ingenuity than often appreciated. Join us for DiMe’s #AskMeAnything Journal Club with Dr. David Shaywitz as he reviews the life cycle of technology innovations (Perez model), and then introduces five new books he recently reviewed in the Wall Street Journal that attempt to contextualize where we are in the artificial intelligence (AI) journey.
A physician-scientist by training, Dr. Shaywitz has focused his career on biomedical innovation as an operator and investor. In January 2020, he founded Astounding HealthTech, advising senior executives on digital health and connected fitness. He is a lecturer in the Department of Biomedical Informatics at Harvard Medical School, and an adjunct scholar at the American Enterprise Institute in Washington, D.C. He lives in the Boston Area with his wife, three daughters, and a clumber spaniel named Roscoe.
Listen in to this #AskMeAnything Journal event and join the conversation with Dr. Shaywitz and your fellow DiMe colleagues. You can check out his review of several AI books here!
The EVIDENCE (EValuatIng connecteD sENsor teChnologiEs) checklist promotes high-quality reporting in studies where the primary objective is an evaluation of a digital measurement product or its constituent parts. The checklist is a product of DiMe’s most recent Tour of Duty and will be published on May 18.
The EVIDENCE checklist is applicable to five types of evaluations: (1) proof of concept; (2) verification, (3) analytical validation, and (4) clinical validation as defined by the V3 framework; and (5) utility and usability assessments. Using EVIDENCE, those preparing, reading, or reviewing studies evaluating digital measurement products will be better equipped to distinguish necessary reporting requirements to drive high-quality research. With broad adoption, the EVIDENCE checklist will serve as a much-needed guide to raise the bar for quality reporting in published literature evaluating digital measurement products.
Did you miss it? You can still check out the following:
Digital Measurement of Mental Health: Challenges, Promises, and Future Directions
With the adoption of digital phenotyping in clinical research and patient care, a common vision for the future of these technologies remains unclear. Listen in to this Journal Club recording with author Anzar Abbas, PhD who opened up a discussion on his work, “Digital Measurement of Mental Health: Challenges, Promises, and Future Directions.” The discussion explored how to classify emerging tools for digital measurement of mental health and discuss the promises and challenges they face.
Anzar Abbas is a neuroscientist focused on developing technology to improve the measurement of health, increase access to care, and inform clinical decision-making using data-driven insights. He is one of the co-creators of OpenDBM, an open-source library of methods in digital phenotyping.
Biopharma and Digital Health: Winning with digital medicine products
“Personalized therapies in the Future of Health: Winning with digital medicine products.” Deloitte Insights. Davis B., Ahmed A., Elsner N., Miranda W. March 2021.
Using Smartphones and Wearable Devices to Monitor Behavioral Changes During COVID-19.
Narayan VA,et al RADAR-CNS Consortium. “Using Smartphones and Wearable Devices to Monitor Behavioral Changes During COVID-19.” J Med Internet Res 2020;22
Digital Measures That Matter to Patients: A Framework to Guide the Selection and Development of Digital Measures of Health.
Manta, Christine, Bray Patrick-Lake, and Jennifer C. Goldsack. “Digital Measures That Matter to Patients: A Framework to Guide the Selection and Development of Digital Measures of Health.” Digital Biomarkers 4.3 (2020).
Fit‐for‐Purpose Biometric Monitoring Technologies (BioMeT): Leveraging the Laboratory Biomarker Experience.
Izmailova, E., Godfrey, A. A., Vandendriessche, B., Bakker, J. P., Fitzer‐Attas, C., Gujar, N., Hobbs, M., … & Zipunnikov, V. Clinical and Translational Science. (2020).
Germany’s Digital Health Reforms in the COVID-19 Era: Lessons and Opportunities for Other Countries
Gerke, S., Stern, A.D. & Minssen, T. “Germany’s digital health reforms in the COVID-19 era: lessons and opportunities for other countries.” npj Digit. Med. 3, 94 (2020).
Development of digital biomarkers for resting tremor and bradykinesia using a wrist-worn device
Mahadevan, N., Patel, S. et al. Development of digital biomarkers for resting tremor and bradykinesia using a wrist-worn wearable device. npj Digit. Med. 3, 5 (2020).
Continuous digital monitoring of walking speed in frail elderly patients: non interventional validation study and longitudinal clinical trial.
Mueller, Arne, et al. “Continuous digital monitoring of walking speed in frail elderly patients: noninterventional validation study and longitudinal clinical trial.” JMIR mHealth and uHealth 7.11 (2019): e15191.
