DiMe Journal Club: The promise of artificial intelligence (AI) and machine learning (ML) for improving clinical outcomes
Date/Time
February 15, 2024
Discussion Topic
The paper addresses the challenges and potential solutions for developing and implementing clinical predictive algorithms (CPAs) in healthcare. The authors highlight that existing CPAs are failing to meet their potential due to an overreliance on the area under the receiver operating characteristic curve (AUC) and a lack of emphasis on performing rigorous clinical trials to quantify the benefits and risks. They propose a more holistic approach to optimizing the development of CPAs, akin to the development of new medicinal products, biomarkers, and other complex interventions.
The study suggests two key recommendations for improving the utility and value of CPAs: first, the authors advocate for considering prevalence of the characteristic of interest, positive predictive value (PPV), and negative predictive value (NPV) to assess the value and utility of the CPA, as these metrics are critical for interpreting CPA results for individual patients. Furthermore, the value of correct predictions must be weighed against the cost of false predictions, and the total value of the CPA must be estimated. Secondly, the authors propose a fit-for-purpose, sequential clinical development approach analogous to the development of new medicinal products, biomarkers, and other complex interventions. This approach involves careful consideration of clinical setting characteristics and the iterative and sequential optimization of CPAs through the selection of cut-off values and the evaluation of outcomes in clinical trials conducted in real-life hospital settings.
