Healthcare AI models must proceed cautiously to avoid exacerbating health inequalities for ethnic minorities, warn scientists
Artificial intelligence (AI) models, including ChatGPT, are being urged to undergo thorough evaluation before implementation in healthcare for ethnic minority populations.
A team of epidemiologists from the University of Leicester and University of Cambridge have raised concerns that existing health inequalities may worsen due to inherent biases in the data used to train AI tools.
Missing ethnicity data and underrepresentation in healthcare AI
The development of AI models necessitates training using data extracted from various sources, such as healthcare websites and scientific research. However, studies reveal that healthcare research often lacks ethnicity data, leading to the underrepresentation of ethnic minorities in research trials.
Mohammad Ali, a PhD Fellow in Epidemiology at the University of Leicester, cautions, “This underrepresentation can result in harmful consequences, such as the creation of ineffective drug treatments or biased treatment guidelines.”
Systemic biases in Healthcare AI models
The researchers express additional concerns regarding the potential exacerbation of health inequalities in low- and middle-income countries (LMICs).
AI models are predominantly developed in affluent nations like the USA and Europe, leading to significant disparities in research and development between high- and low-income countries.
The researchers emphasise that published research often fails to cater to the unique health challenges LMICs face, particularly regarding healthcare provision. As a result, AI models may offer recommendations based on data from entirely different populations.
Overcoming health inequalities through inclusive AI models
Despite highlighting potential challenges, the researchers emphasise the importance of finding solutions to address health inequalities. They suggest that AI models provide precise descriptions of the data used during development to achieve this.
Furthermore, the team proposes the improvement of ethnic health inequalities in research, aiming to enhance the recruitment and recording of ethnic information.
To ensure fairness and inclusivity, the data used to train AI models should adequately represent diverse factors such as ethnicity, age, sex, and socioeconomic background.
The researchers also stress the need for further research to understand the application of AI models within ethnically diverse populations.
By addressing these considerations, the researchers believe that AI models can be leveraged to drive positive change in healthcare while promoting fairness and inclusivity for all populations.
