A team of researchers at Washington University (WashU) in St. Louis has developed a foundation AI model that uses clinical notes from surgical patients to predict complications like pneumonia, blood clots, and infections. The new model, details of which are published in npj Digital Medicine, could help reduce the rate postoperative complications that affect roughly 10% of patients, which can lead to longer intensive care unit stays, higher mortality rates, and higher costs.
“Surgery carries significant risks and costs, yet clinical notes hold a wealth of valuable insights from the surgical team,” said lead researcher Chenyang Lu, PhD, a professor of computer science and engineering at WashU. “Our large language model, tailored specifically for surgical notes, enables early and accurate prediction of postoperative complications. By identifying risks proactively, clinicians can intervene sooner, improving patient safety and outcomes.”
The challenge for healthcare provides looking to reduce rates of post-surgery complications has been finding an accurate method to predict risks early enough to intervene before problems arise. Traditional risk prediction models have relied on structured data, such as lab tests and patient demographics without using clinical notes data that can provide a more detailed, personalized assessment of each patient.
To overcome this limitation, Lu and the WashU team have developed a foundation AI model that uses large language models (LLMs) to analyze the unstructured data found in a patient’s clinical notes. Using LLMs, the new model outperformed traditional machine learning methods in forecasting postoperative complications. For example, for every 100 patients who experienced a complication, the new model correctly identified 39 more at-risk patients than previous models.
Charles Alba, a graduate student who co-authored the study, said that the model’s versatility is one of its strengths. “Foundation models can be diversified, so they’re generally more useful than specialized models,” he said. “We fine-tuned our model for multiple tasks at the same time and found that it predicts complications more accurately than models trained specifically to detect individual complications.”
An important capability of the model is its ability to identify risk of multiple complications. As complications often share underlying risk factors, a unified model can leverage these correlations to make more accurate predictions across various surgical outcomes.
“This versatile model has the potential to be deployed across various clinical settings to predict a wide range of complications,” said Joanna Abraham, PhD, an associate professor of anesthesiology at WashU Medicine. “By identifying risks early, it could become an invaluable tool for clinicians, enabling them to take proactive measures and tailor interventions to improve patient outcomes.”
The integration of AI into clinical workflows is not without its challenges, however. Trust and confidence in models from treating clinicians is vital to widespread adoption, and models must demonstrate superior performance to traditional methods while providing predictions and guidance that is understandable and actionable. The researchers noted that while current models are “black-box” systems, they are working on integrating interpretability mechanisms to improve model transparency.
A limitation of the current study is the data were drawn from a single healthcare system, Barnes-Jewish Hospital. More work is needed, the researchers said, to ensure the model generalizes well across different institutions where different surgical notes could include terminology and abbreviations that could affect the model’s accuracy. Future work on the model will include using data from multiple hospital systems to help refine the model for broader use in clinical settings.
link