Dissecting racial bias in an algorithm used to manage the health of populations

@article{Obermeyer2019DissectingRB,
  title={Dissecting racial bias in an algorithm used to manage the health of populations},
  author={Ziad Obermeyer and Brian W. Powers and Christine Vogeli and Sendhil Mullainathan},
  journal={Science},
  year={2019},
  volume={366},
  pages={447 - 453}
}
Racial bias in health algorithms The U.S. health care system uses commercial algorithms to guide health decisions. Obermeyer et al. find evidence of racial bias in one widely used algorithm, such that Black patients assigned the same level of risk by the algorithm are sicker than White patients (see the Perspective by Benjamin). The authors estimated that this racial bias reduces the number of Black patients identified for extra care by more than half. Bias occurs because the algorithm uses… 
View on AAAS
escholarship.org
1,283 Citations
Highly Influential Citations
40
Background Citations
457
Methods Citations
85
Results Citations
13

1,283 Citations

Assessing risk, automating racism
TLDR
One of the first studies to examine the outputs and inputs of an algorithm that predicts health risk, and influences treatment, of millions of people is reported, finding that because the tool was designed to predict the cost of care as a proxy for health needs, Black patients with the same risk score as White patients tend to be much sicker, because providers spend much less on their care overall.
The Contrast of Color: Why the Black Community Continues to Suffer Health Disparities.
TLDR
Health care organizations must invest time and resources into investigating the structural racism that exists within the authors' own walls and look introspectively for implicit bias to find the cure.
An individual-level socioeconomic measure for assessing algorithmic bias in health care settings: A case for HOUSES index
TLDR
The major role of socioeconomic status (SES) in AI algorithm bias and (in)completeness of electronic health records (EHRs) data, which is commonly used for algorithm development, is highlighted and a novel conceptual strategy for quantifying AI bias by SES is highlighted.
A Racially Unbiased, Machine Learning Approach to Prediction of Mortality: Algorithm Development Study
TLDR
There may be significant racial bias in commonly used severity scoring systems and that machine learning algorithms may reduce bias while improving on the accuracy of these methods.
Capacity Strain and Racial Disparities in Hospital Mortality ∗
TLDR
Using detailed, time-stamped electronic health record data from two major hospitals, this work document a 20% relative increase in mortality for Black compared to White patients when hospitals operate at capacity, driven entirely by patients with more medical comorbidities.
A Racially Unbiased, Machine Learning Approach to Prediction of Mortality: Algorithm Development Study (Preprint)
TLDR
There may be significant racial bias in commonly used severity scoring systems and that machine learning algorithms may reduce bias while improving on the accuracy of these methods.
Imperfect Inferences: A Practical Assessment
Measuring racial disparities is challenging, especially when demographic labels are unavailable. Recently, some researchers and advocates have argued that companies should infer race and other…
Threat of racial and economic inequality increases preference for algorithm decision-making
The Emerging Hazard of AI-Related Health Care Discrimination.
TLDR
This article urges that both legal and technical tools be deployed to promote AI fairness, and Plaintiffs should be able to assert disparate impact claims in health-care litigation, and Congress should enact an Algorithmic Accountability Act.
Algorithms that "Don't See Color": Comparing Biases in Lookalike and Special Ad Audiences
TLDR
Results provide experimental proof that merely removing demographic features from a real-world algorithmic system's inputs can fail to prevent biased outputs, and organizations using algorithms to help mediate access to important life opportunities should consider other approaches to mitigating discriminatory effects.
...
...

References

SHOWING 1-10 OF 51 REFERENCES
Racial bias in pain assessment and treatment recommendations, and false beliefs about biological differences between blacks and whites
TLDR
It is shown that a substantial number of white laypeople and medical students and residents hold false beliefs about biological differences between blacks and whites and this work demonstrates that these beliefs predict racial bias in pain perception and treatment recommendation accuracy.
Does Diversity Matter for Health? Experimental Evidence from Oakland
TLDR
The findings suggest black doctors could reduce the black-white male gap in cardiovascular mortality by 19 percent and the effect of physician workforce diversity on the demand for preventive care among African American men.
The effect of patient race and socio-economic status on physicians' perceptions of patients.
Socioeconomic inequalities in health. No easy solution.
TLDR
Health insurance coverage alone is not likely to reduce significantly SES differences in health, and attention should be paid both in policy decisions and in clinical practice to other SES-related factors that may influence patterns of health and disease.
Racial/ethnic differences in physician distrust in the United States.
TLDR
Racial/ethnic differences in physician distrust are less uniform than previously hypothesized, with substantial geographic and individual variation present.
Tuskegee and the Health of Black Men
TLDR
It is found that the disclosure of the Tuskegee Study of Untreated Syphilis in the Negro Male in 1972 is correlated with increases in medical mistrust and mortality and decreases in both outpatient and inpatient physician interactions for older black men.
The challenge of multiple comorbidity for the US health care system.
TLDR
One area in which some initial progress is being made to reduce the burden of multiple chronic conditions on society is advancing evidence-based clinical decision making in the care for patients with comorbidities.
Adjusting for social risk factors impacts performance and penalties in the hospital readmissions reduction program
TLDR
Accounting for social risk factors can have a major financial impact on safety-net hospitals, and adjustment for these factors could reduce negative unintended consequences of the HRRP.
The Paradox of Automation as Anti-Bias Intervention
A received wisdom is that automated decision-making serves as an anti-bias intervention. The conceit is that removing humans from the decision-making process will also eliminate human bias. The…
Big Data's Disparate Impact
Advocates of algorithmic techniques like data mining argue that these techniques eliminate human biases from the decision-making process. But an algorithm is only as good as the data it works with.…
...
...