Phase II Amount
$1,995,966
Advanced machine learning (ML) has consistently been shown to outperform expert opinion and more simpleanalytics for predicting clinical outcomes. However, there has been a paucity of successful prospective clinicalimplementations of such tools. The unique barriers to advanced ML implementation and adoption in healthcareare (1) the technological challenges of running and displaying these models in real-time within existing workflowsand (2) a general distrust for black box algorithms among highly skilled providers. As a result, the promise ofthese tools is largely lost in healthcare. This is particularly problematic in COVID-19, where patients candeteriorate rapidly, from appearing stable to suddenly being in respiratory failure or shock with little obviouswarning. Early recognition of this deterioration is vital to proactive interventions, which can improve outcomes.eCART is a predictive analytic that has been developed iteratively at the University of Chicago over the pastdecade to identify hospitalized patients at risk for acute clinical deterioration. A simple (logistic regression based)ML model (eCARTv2) is commercially available within electronic health records on AgileMD's clinical decisionsupport platform. eCARTv2 was developed in a retrospective multicenter dataset and its use in clinical practicewas associated with a 29% relative risk reduction in mortality in a multicenter trial. Our team recently completeddevelopment and validation of a gradient boosted machine (GBM) version of the model (eCARTv4), using nearly100 variables, including trends and interactions. The advanced ML model was significantly more accurate thanthe simple ML and other models for predicting acute clinical deterioration across all hospital settings, in bothseptic and non-septic patients as well as in COVID-19 patients. The next challenge is clinically implementing it.The goals of this project are to a) upgrade the existing AgileMD platform to support the previously derived andvalidated eCARTv4 model and overhaul the human-machine interface for an advanced user experience (UX)that provides, for the first time, interpretable, graphical insight into the contribution of individual variables to areal-time EHR-embedded advanced ML analytic, and b) measure the impact of the new tool on HCPeffectiveness, efficiency and satisfaction. We hypothesize that the combination of high accuracy andinterpretability afforded by the advanced ML and UX will result in earlier recognition of acute deterioration as wellas increased System Usability Scores (SUS) and usefulness scores in the treatment of deteriorating COVID-19patients over standard care.
Public Health Relevance Statement: Project Narrative Advanced machine learning (ML) has the potential to improve patient outcomes by alerting health care providers to early signs of patient deterioration they may otherwise miss. However, clinical teams have been hesitant to fully embrace these tools because of their black box nature and inability to verify the logic underpinning them. This project intends to give medical staff the transparency they need to confidently leverage sophisticated ML tools to better treat hospitalized patients, particularly those with COVID-19.
Project Terms: Adoption ; Adult ; 21+ years old ; Adult Human ; adulthood ; Algorithms ; Chicago ; Goals ; Health Personnel ; Health Care Providers ; Healthcare Providers ; Healthcare worker ; health care personnel ; health care worker ; health provider ; health workforce ; healthcare personnel ; medical personnel ; treatment provider ; hemodynamics ; Hemorrhage ; Bleeding ; blood loss ; Hospitals ; Human ; Modern Man ; Interview ; Logic ; Medical Staff ; mortality ; Patients ; Risk ; Running ; Shock ; Circulatory Collapse ; circulatory shock ; Testing ; Time ; Universities ; User-Computer Interface ; human computer interface ; human machine interface ; man machine interface ; Weight ; Measures ; Workload ; Work Load ; Healthcare ; health care ; Data Set ; Dataset ; base ; improved ; Acute ; Clinical ; Phase ; Multicenter Trials ; Multi-center trial ; Logistic Regressions ; Ensure ; Evaluation ; insight ; Individual ; Trust ; satisfaction ; Relative Risks ; uptake ; septic ; tool ; Nature ; machine learned ; Machine Learning ; System ; experience ; Expert Opinion ; simulation ; novel ; Deterioration ; Modeling ; response ; Intervention Strategies ; interventional strategy ; Intervention ; Risk Reduction ; Provider ; lung failure ; pulmonary failure ; Respiratory Failure ; Effectiveness ; Patient-Focused Outcomes ; Patient outcome ; Patient-Centered Outcomes ; Small Business Innovation Research Grant ; SBIR ; Small Business Innovation Research ; Validation ; trend ; Process ; Development ; developmental ; Electronic Health Record ; electronic health care record ; electronic healthcare record ; pandemic disease ; pandemic ; predictive modeling ; computer based prediction ; prediction model ; design ; designing ; Sepsis ; blood infection ; bloodstream infection ; prospective ; usability ; standard of care ; standard care ; standard treatment ; clinical practice ; Predictive Analytics ; improved outcome ; prediction algorithm ; predictive algorithm ; predictor algorithm ; predict clinical outcome ; clinical outcome prediction ; predictor of clinical outcome ; Retrospective cohort ; clinical decision support ; clinical implementation ; machine learning algorithm ; machine learned algorithm ; COVID-19 ; COVID19 ; CV-19 ; CV19 ; corona virus disease 2019 ; coronavirus disease 2019 ; Visualization ; distrust ; COVID-19 patient ; COVID infected patient ; COVID patient ; COVID positive patient ; COVID-19 infected patient ; COVID-19 positive patient ; COVID19 patient ; COVID19 positive patient ; SARS-CoV-2 infected patient ; SARS-CoV-2 patient ; SARS-CoV-2 positive patient ; coronavirus disease 2019 infected patient ; coronavirus disease 2019 patient ; coronavirus disease 2019 positive patient ; coronavirus disease infected patient ; coronavirus disease patient ; coronavirus disease positive patient ; coronavirus patient ; patient infected with COVID ; patient infected with COVID-19 ; patient infected with SARS-CoV-2 ; patient infected with coronavirus disease ; patient infected with coronavirus disease 2019 ; patient infected with severe acute respiratory syndrome coronavirus 2 ; patient with COVID ; patient with COVID-19 ; patient with COVID19 ; patient with SARS-CoV-2 ; patient with coronavirus disease ; patient with coronavirus disease 2019 ; patient with severe acute respiratory distress syndrome coronavirus 2 ; severe acute respiratory syndrome coronavirus 2 infected patient ; severe acute respiratory syndrome coronavirus 2 patient ; severe acute respiratory syndrome coronavirus 2 positive patient ;
