In a study published in JAMA Internal Medicine, researchers from Beth Israel Deaconess Medical Center (BIDMC) have revealed that artificial intelligence (AI) may have better clinical reasoning capabilities compared to human physicians
The study compared the performance of a large language model (LLM), specifically ChatGPT-4, against internal medicine residents and physicians at two academic medical centres.
Lead author Dr Adam Rodman, an internal medicine physician and investigator at BIDMC, highlighted the significance of the findings, stating, “It’s a surprising finding that these things are capable of showing the equivalent or better reasoning than people throughout the evolution of clinical case.”
Physicians compete against AI
The study used a validated tool called the revised-IDEA (r-IDEA) score to assess clinical reasoning. Physicians and the AI model were tasked with working through 20 clinical cases, each comprising four sequential stages of diagnostic reasoning: triage data collection, system review, physical examination, and diagnostic testing/imaging.
Results revealed that the LLM achieved the highest median r-IDEA score, outperforming physicians and residents. While diagnostic accuracy and correct clinical reasoning were similar between humans and the AI model, the researchers noted that the LLM exhibited more incorrect reasoning.
AI in medical decision making
This highlights the potential of AI as a complementary tool rather than a replacement for human expertise in healthcare.
Dr Stephanie Cabral, a third-year internal medicine resident at BIDMC and co-author of the study, emphasised the potential role of AI in improving patient-physician interactions and reducing inefficiencies in healthcare delivery. “Even now, they could be useful as a checkpoint, helping us make sure we don’t miss something” she stated. “My ultimate hope is that AI will improve the patient-physician interaction by reducing some of the inefficiencies we currently have and allow us to focus more on the conversation we’re having with our patients.”
The findings suggest that AI could significantly impact clinical practice by increasing healthcare providers’ diagnostic and reasoning capabilities.
Further research is still needed to determine the optimal integration of AI into medical decision-making processes.
Could AI improve the healthcare system?
While the study marks a significant advancement in AI-driven healthcare, the researchers acknowledge potential conflicts of interest, including grant funding and employment affiliations with various organisations.
As AI continues to evolve, its role in healthcare decision-making will likely expand, offering new opportunities to improve patient outcomes and streamline clinical workflows. With careful integration and validation, AI may become a valuable partner in pursuing better healthcare delivery.
