After five years of development, a new tool at Nationwide Children’s Hospital is substantially reducing deterioration events for its inpatient population. Using machine learning, the Deterioration Risk Index (DRI) predicts inpatient deterioration by automatically processing risk criteria in the EMR. After 18 months of use, DRI reduced deterioration events by more than two-thirds, according to a report published in Pediatric Critical Care Medicine.
“Predictive algorithms focused on improving clinical care have been increasingly developed over the years, but the vast majority are not operationalized,” says Laura Rust, M.D., emergency medicine physician and physician informaticist at Nationwide Children’s and lead author of the paper. “Transitioning the algorithm from the computer to the bedside can be a long process and requires engagement and collaboration from clinicians, data scientists and clinical informaticists. This project has been a five-plus year journey, and we are proud of the successful integration within our safety culture and the impact on patient outcomes.”
How it works: Predicting deterioration in patients
To develop the Deterioration Risk Index, three diagnostic groups were used to train three separate predictive models: structural heart defect (cardiac), oncology (malignancy), and general (neither cardiac nor malignancy).
“One of the design features that helped build trust with clinical teams is that we didn’t necessarily identify any new criteria. Our model simply identifies which existing situational awareness criteria are most important and weighs them accordingly,” says Tyler Gorham, data scientist in IT Research and Innovation at Nationwide Children’s and co-author of the publication.
According to Rust, there can be an overwhelming amount of clinical data within the electronic health record to process at any one time, especially after handoffs or transitions of care. The model helps to relieve this cognitive burden by automatically processing these risk criteria behind the scenes. Because it is integrated within the EMR, it has the benefit of having all the data from all previous points of time, not just the current shift.
Over the first 18 months, deterioration events decreased 77% compared to expected event rates in preceding years.
The DRI was built off the foundation of the Watchstander situational awareness program, already in use at Nationwide Children’s. To promote adoption, the team utilized the same response mechanisms for alerts:
- Patient assessment.
- Huddle with the bedside care team within 30 minutes.
- Risk mitigation.
- Escalation plan.
The team also conducted road shows—visiting clinical units where the tool would be deployed, answering questions, doing simulations with the bedside care teams and incorporating feedback.
The results: Reduction of deterioration in patients
The DRI was 2.4 times as sensitive as the existing situational awareness program while also requiring 2.3 times fewer alarms per detected event. Notably, the team observed a four-fold sensitivity gain for the cardiac group and a three-fold gain for the malignancy group. The pilot study after implementation found that over the first 18 months, deterioration events decreased 77% compared to expected event rates in preceding years.
Perhaps the most important aspect of the model, according to the developers, is its transparency.
“This is not a black box. We show clinicians what goes in and how the algorithm evaluates the data to trigger alarms,” says Gorham. “The tool helps support clinical decision making because the clinical team is able to see why an alarm was triggered.”
The DRI was 2.4 times as sensitive as the existing situational awareness program while also requiring 2.3 times fewer alarms per detected event.
More information, including details about the algorithm, is available in the publication.
“We shared our recipe in the publication,” says Gorham. “If others are interested, they could use their center’s data and retrain the model for their local populations. If we can share our success with others, we can hopefully support better outcomes for all children, even those who aren’t under our care.”
