12/13/2022 Bethan Owen
Bioengineering professor Yogatheesan Varatharajah and his team recently presented a paper on how to identify and avoid machine learning and AI model failure at the prestigious Conference on Neural Information Processing Systems, or NeurlPS.
Written by Bethan Owen
Machine learning and artificial intelligence (AI) are exploding in popularity in fields ranging from art to science and everything in between—bioengineering included. While these tools have the potential to bring about significant improvements in healthcare, the systems aren’t perfect. How can we identify when machine learning and artificial intelligence are suggesting solutions that aren’t effective in the real world?
Bioengineering professor Yogatheesan Varatharajah is working towards answering that question through his body of research, which is geared towards understanding when and how specific AI-generated models will fail. Varatharajah and his team recently presented a paper on the subject, titled Evaluating Latent Space Robustness and Uncertainty of EEG-ML Models under Realistic Distribution Shifts, at the prestigious Conference on Neural Information Processing Systems, or NeurlPS.
“Every domain in healthcare is using machine learning in one way or another, and so they’re becoming the mainstay of computational diagnostics and prognostics in healthcare,” Varatharajah said. “The problem is that when we do studies based on machine learning—to develop a diagnostic tool, for example—we run the models, and then we say, okay, the model performs well in a limited test setting and therefore it's good to go. But when we actually deploy it in the real world to make clinical decisions in real time, many of these approaches don't work as expected.”
Varatharajah explained that one of the most common reasons for this disconnect between models and the real world is the natural variability between collected data that is used to create a model and the data that is collected after a model is deployed. That variability might come from the hardware or protocol used to collect the data, or just differences between patients inside and outside the model. These small differences can add up to significant changes in model predictions, and, potentially, a model that fails to help patients.
“If we can identify those differences ahead of time, then we may be able to develop some additional tools to prevent those failures or at least know that these models are going to fail in certain scenarios,” Varatharajah said. “And that is the goal of this paper.”
To do this, Varatharajah and his students focused their efforts on machine-learning models based on electrophysiological data, specifically EEG recordings that are collected from patients with neurological diseases. From there, the team analyzed clinically-relevant applications, like comparing normal EEGs to abnormalities to determine whether it was possible to differentiate the two.
“We looked at what kind of variability can occur in the real world, especially those variabilities which could cause problems to machine learning models,” said Varatharajah. “And then we modeled those variabilities and developed some ‘diagnostic’ measures to diagnose the models themselves, to know when and how they are going to fail. As a result, we can be aware of these errors and take steps to mitigate them ahead of time, so the models are actually able to help clinicians with clinical decision making.”
Presenting this research at NeurIPS was particularly significant. As one of the premiere machine learning conferences in the world, Varatharajah stated how exciting it was to have the opportunity to engage with fellow presenters and attendees at NeurlPS. Bioengineering PhD student Neeraj Wagh presented this work at the conference.
“It’s quite an achievement to have a publication accepted at this venue—it gives us a name in this community,” Varatharajah said. “This will also give us the opportunity to further develop this tool into something that can be used in the real world.”
Work as wide-reaching as this isn’t a solo effort, and Varatharajah specifically mentioned several other people who made this research possible. This list included co-authors Neeraj Wagh, Jionghao Wei, Samarth Rawal, and Brent Berry, as well as the partnership between Illinois bioengineering and the Mayo Clinic’s Department of Neurology. This project was also facilitated through the Mayo Clinic and supported by the National Science Foundation.