Unraveling Bias in Healthcare AI — Karen Walker Johnson
As artificial intelligence (AI) becomes more prominent in healthcare, its potential is generating significant buzz. AI’s promise is undeniable. It can detect cancer, prioritize patient care, and offer treatment recommendations, all with the potential to increase access to high-quality healthcare, particularly for underserved populations. However, we need to be careful how we deploy AI as inherent biases in the systems could actually increase health inequity.
The major concern is that algorithms based solely on mathematics can lead to various adverse outcomes for patients, including misdiagnoses and overlooked health conditions. These issues are not new, as revealed by research:
• A 2020 University of Pennsylvania investigation uncovered that AI systems designed to predict patient risk for certain diseases were not as effective when applied to Black and Hispanic patients.
• In 2022, a Stanford University study showed that AI systems used to detect skin cancer were less accurate for dark-skinned patients.
• A 2023 American Medical Association study found that AI systems used to recommend treatments for patients are more likely to recommend higher priced options for white patients and less expensive for Black patients.
As AI systems are used to make increasingly critical decisions about patient care, it is essential to ensure that they are fair and unbiased. Apparently this is not an easy task in AI programming.
In trying to understand this problem in greater detail, I came across the Wired article “Health Care Bias Is Dangerous. But So Are ‘Fairness’ Algorithms,” by Sandra Wachter, Brent Mittelstadt, and Chris Russell. The authors reveal that most of these algorithms focus on mathematical definitions of fairness without considering broader societal contexts. They aim to close the gap between groups, like making sure Black patients get the same level of care as white patients. Unfortunately, that can actually end up making things worse.
For example, in the context of lung cancer risk prediction, if the AI system performs poorly for Black patients, it may be tweaked to classify more cases as “high risk,” even at the cost of overall accuracy. This practice is known as “leveling down.” In this case, the trade-off is deemed acceptable because failing to diagnose someone with cancer is considered far more harmful than providing unnecessary follow-up tests. Overall, however, this practice isn’t ideal because it might mean that one group ends up getting worse treatment just to make things look more equal on paper.
The authors of the Wired article argue that “leveling up” is a more morally, ethically, and legally acceptable path forward. They define this as moving beyond the technical Band-aids and state that “improving access to health care, curating more diverse data sets, and developing tools that specifically target the problems faced by historically disadvantaged communities can help make substantive fairness a reality.”
I wholeheartedly concur. This is a challenge that calls for substantive fairness, not just mathematical evenhandedness. We need to go beyond the math and address the bigger social issues. Let’s implement technological and methodological innovation, but at the same time work on improving healthcare access and expenditures. By addressing inherent bias in healthcare AI — and finding pathways to Level Up — we take a crucial step towards ensuring equitable healthcare for all.
Read the full Wired article here
Having worked on all sides of the healthcare industry, Karen Walker Johnson brings a vast and diverse set of skills and expertise to her leadership roles. She has focused her passion of leading teams to improve the health status of vulnerable populations. Devoted to exploring how healthcare disparities and social determinants play a role in individuals’ health, Karen actively explores new approaches to solve the healthcare problems of today. The knowledge and energy she brings to the table are an asset to corporate boards.
Connect with me on LinkedIn
