Since the COVID-19 pandemic hit, predictive modeling has been helping guide the critical decisions that national, state, and local leaders need to make in the face of uncertainty. Predictive modeling can also help health systems decide how best to deploy limited resources, especially during times of crisis. We talked with Emily Westbrook, MHA, about how Kaiser Permanente Washington’s Learning Health System (LHS) Program has been helping the organization develop and deploy predictive risk models—both before and since COVID-19—and what work is on the horizon. Emily is the Director of Research Operations and Programs at Kaiser Permanente Washington Health Research Institute (KPWHRI) and the operations lead for the LHS Program.
Predictive modeling uses data analysis, machine learning, and statistics to identify patterns in data and recognize the chance of specific events occurring. One way predictive modeling is used in health care is to understand who in a population might be at risk of a harmful event or a negative health outcome. For example, a predictive model can help identify individuals with high medical need or disease burden who are at high risk of being hospitalized in the next 12 months.
Over the last several years, the Learning Health System Program has been part of an emerging interdisciplinary partnership to develop methods for responsible implementation of risk prediction models in care delivery at Kaiser Permanente Washington. This work brings together expertise from the Research Institute, Care Delivery, Care Delivery Technology Services, Finance, Business Intelligence, and other areas of the organization.
Our key successes before the COVID-19 pandemic were to develop, validate, and deploy risk prediction models that have:
Additionally, we’ve been exploring other uses of risk prediction models to help identify members who could benefit from palliative care services or members who could benefit from additional case management because they are at higher risk for a prolonged hospital stay.
When COVID-19 hit, we were able to utilize existing infrastructure from our flu vaccination intervention to rapidly deploy outreach efforts to Medicare members to make sure their care needs were being addressed. We were able to then quickly expand this to include outreach efforts to non-Medicare members with complex medical and social needs and other chronic conditions.
Having an interdisciplinary team is critical because it helps us avoid pitfalls that can lead to biased models. One example in the press recently is a health care risk-prediction algorithm that exacerbates health inequities.
The reason for the bias in this model is that it uses health care spending as a proxy for health care needs. But we know that structural racism impacts access, treatment, and outcomes for Black, Indigenous, and People of Color, and less money is spent on Black patients than is actually required to meet their needs. Thus, using a model based on health care spending channels a beneficial resource to white patients and withholds it from Black patients who have higher needs.
Risk models also have other important limitations that we need to be aware of. For example, models aren’t helpful unless they are matched with an evidence-based intervention that is correctly balanced with the accuracy of the model. If the intervention is low-risk and low-cost to the health system, it is not as critical to have a model with perfect accuracy to predict the outcome. If the intervention is potentially high-risk and high-cost to the organization, it would be important to have a model with very high accuracy to predict the outcome.
KPWHRI Assistant Investigator Yates Coley, PhD, has been instrumental in investigating and addressing racial and other types of bias in the clinical prediction algorithms we develop. A biostatistician and CATALyST K12 Learning Health System scholar, Yates is evaluating the performance of our suicide attempt risk prediction model across racial and ethnic subgroups to ensure that all patients benefit from the use of predictive analytics. Matching sophisticated statistical techniques with expert interdisciplinary teams helps us ensure the models we develop and deploy are accurate, actionable, and equitable.
As the pandemic unfolded, anecdotal evidence emerged suggesting that there could be significant decreases in the use of acute care during COVID-19 because people are afraid of contracting the virus—so afraid that they may not be seeking care when they really need it. Again, this is where the Learning Health System Program can help: We can use expert analytic methods to understand the extent of the problem and to identify ways to intervene and help prevent negative health outcomes for our members.
Our team is now focused on some key next steps, including:
Our ultimate goal is to harness the full power of being an integrated health system that combines care and coverage and has robust data sources to ensure that we are delivering on the promise of being the best place to receive care. We are in a unique position to really use the data we have wisely to provide high-quality, personalized care to our members during a time when they need it most.
New KPWHRI study helps confirm national guidelines that encourage people with high-risk conditions to get the shot.
Read more in Healthy Findings.
Katie Coleman describes how the pandemic boosted adoption of telehealth and how research is vital for it to thrive.