Researchers from Carnegie Mellon University’s Human-Computer Interaction Institute (HCII) have studied how AI in ICU will assist clinicians
A study from Carnegie Mellon University’s Human-Computer Interaction Institute (HCII) explores the use of artificial intelligence (AI) in the intensive care unit (ICU) to assist clinicians in making complex decisions for critically ill patients.
Researchers collaborated with physicians and researchers from the University of Pittsburgh and UPMC to determine whether clinicians would trust AI assistance in their decision-making process.
The study provided 24 ICU physicians with access to an AI-based tool. The AI can help clinicians make decisions better. Though AI has the potential to aid clinicians in the ICU, further research is needed to address the challenges of implementing such technology.
It feels like clinicians are excited about the potential for AI to help them, but they might not be familiar with how these AI tools would work
Venkatesh Sivaraman, a Ph.D. student in the HCII and member of the research team explains: “It feels like clinicians are excited about the potential for AI to help them, but they might not be familiar with how these AI tools would work. So it’s really interesting to bring these systems to them.”
Using the AI Clinician model to design an interactive clinical decision support (CDS) interface
The interactive clinical decision support interface, called the AI Clinician Explorer, uses the AI Clinician model introduced in Nature in 2018. The interface provides recommendations for treating sepsis based on a data set of over 18,000 ICU patients diagnosed with sepsis.
The tool allows clinicians to:
- Filter and search for patients
- Visualize their disease progression
- Compare model predictions to actual treatment decisions.
“Clinicians are always entering a lot of data about the patients they see into these computer systems and electronic health records,” Sivaraman comments.
“The idea is that maybe we can learn from some of that data so we can try to speed up some of their processes, make their lives a little bit easier and also maybe improve the consistency of care.”
The team tested their system with a think-aloud study involving 24 ICU clinicians experienced in treating sepsis. Participants used a simplified AI Clinician Explorer interface to assess and make treatment decisions for four simulated patient cases.
“We thought the clinicians would either let the AI make the decision entirely or ignore it completely and make their own decision,” Sivaraman said.
Concerns about AI in ICU
The study found that clinicians’ responses to AI recommendations were not binary, and identified four common behaviors: ignore, rely, consider, and negotiate.
The “ignore” group did not let AI influence their decision, while the “rely” group consistently accepted AI’s input. In the “consider” group, physicians thought about AI recommendations before accepting or rejecting them, while the “negotiate” group accepted individual aspects of recommendations in some cases but not all.
Clinicians expressed concerns about the AI’s limited access to holistic patient data and were sceptical of recommendations that went against traditional teaching.
‘It’s impossible for any one human to amass all that knowledge’
“When the CDS deviates from what clinicians would normally do or consider to be best practice, there was not a good sense of why,” Sivaraman said.
“So right now, we’re focusing on determining how to provide that data and validate these recommendations, which is a challenging problem that will require machine learning and AI.”
Rather than replacing or replicating clinician decision-making, the team’s research aims to utilize AI to uncover patterns that may have been previously overlooked in patient outcomes.
“There are a lot of diseases, like sepsis, that might present very differently for each patient, and the best course of action might be different depending on that,” Sivaraman concludes.
“It’s impossible for any one human to amass all that knowledge to know how to do things best in every situation. So maybe AI can nudge them in a direction they hadn’t considered or help validate what they consider the best course of action.”
