Dissecting racial bias in an algorithm used to manage the health of populations

@article{Obermeyer2019DissectingRB,
  title={Dissecting racial bias in an algorithm used to manage the health of populations},
  author={Ziad Obermeyer and Brian W. Powers and Christine Vogeli and Sendhil Mullainathan},
  journal={Science},
  year={2019},
  volume={366},
  pages={447 - 453}
}
Racial bias in health algorithms The U.S. health care system uses commercial algorithms to guide health decisions. Obermeyer et al. find evidence of racial bias in one widely used algorithm, such that Black patients assigned the same level of risk by the algorithm are sicker than White patients (see the Perspective by Benjamin). The authors estimated that this racial bias reduces the number of Black patients identified for extra care by more than half. Bias occurs because the algorithm uses… 
View on AAAS
escholarship.org
1,484 Citations
Highly Influential Citations
52
Background Citations
555
Methods Citations
109
Results Citations
15

1,484 Citations

Assessing risk, automating racism

One of the first studies to examine the outputs and inputs of an algorithm that predicts health risk, and influences treatment, of millions of people is reported, finding that because the tool was designed to predict the cost of care as a proxy for health needs, Black patients with the same risk score as White patients tend to be much sicker, because providers spend much less on their care overall.

The Contrast of Color: Why the Black Community Continues to Suffer Health Disparities.

Health care organizations must invest time and resources into investigating the structural racism that exists within the authors' own walls and look introspectively for implicit bias to find the cure.

A Racially Unbiased, Machine Learning Approach to Prediction of Mortality: Algorithm Development Study

There may be significant racial bias in commonly used severity scoring systems and that machine learning algorithms may reduce bias while improving on the accuracy of these methods.

Capacity Strain and Racial Disparities in Hospital Mortality

Dynamic decomposition analyses suggest that these findings are most likely driven by additional biases in provider behavior, hospital processes, and/or allocation of care resources that emerge or worsen as strain increases, which then interact with already lower intensities of care to produce differential mortality risks.

A Racially Unbiased, Machine Learning Approach to Prediction of Mortality: Algorithm Development Study (Preprint)

There may be significant racial bias in commonly used severity scoring systems and that machine learning algorithms may reduce bias while improving on the accuracy of these methods.

Imperfect Inferences: A Practical Assessment

Measuring racial disparities is challenging, especially when demographic labels are unavailable. Recently, some researchers and advocates have argued that companies should infer race and other…

Threat of racial and economic inequality increases preference for algorithm decision-making

The Emerging Hazard of AI-Related Health Care Discrimination.

This article urges that both legal and technical tools be deployed to promote AI fairness, and Plaintiffs should be able to assert disparate impact claims in health-care litigation, and Congress should enact an Algorithmic Accountability Act.

Ethical Redress of Racial Inequities in AI: Lessons from Decoupling Machine Learning from Optimization in Medical Appointment Scheduling

A framework for addressing trade-offs in medical appointment scheduling where Machine Learning and Optimization components of the algorithm are decoupled is presented, in one case preserving accuracy comparable to the current state-of-the-art while eliminating the disparity.

Algorithms that "Don't See Color": Measuring Biases in Lookalike and Special Ad Audiences

Results provide experimental proof that merely removing demographic features from a real-world algorithmic system's inputs can fail to prevent biased outputs, and organizations using algorithms to help mediate access to important life opportunities should consider other approaches to mitigating discriminatory effects.
...

References

SHOWING 1-10 OF 47 REFERENCES

Racial bias in pain assessment and treatment recommendations, and false beliefs about biological differences between blacks and whites

It is shown that a substantial number of white laypeople and medical students and residents hold false beliefs about biological differences between blacks and whites and this work demonstrates that these beliefs predict racial bias in pain perception and treatment recommendation accuracy.

Inequality in quality: addressing socioeconomic, racial, and ethnic disparities in health care.

5 principles to address disparities in health care through modifications in quality performance measures are proposed: disparities represent a significant quality problem and clinical performance measures should be stratified by race/ethnicity and socioeconomic position for public reporting.

The effect of patient race and socio-economic status on physicians' perceptions of patients.

Socioeconomic inequalities in health. No easy solution.

Health insurance coverage alone is not likely to reduce significantly SES differences in health, and attention should be paid both in policy decisions and in clinical practice to other SES-related factors that may influence patterns of health and disease.

Racial/ethnic differences in physician distrust in the United States.

Racial/ethnic differences in physician distrust are less uniform than previously hypothesized, with substantial geographic and individual variation present.

Tuskegee and the Health of Black Men

It is found that the disclosure of the Tuskegee Study of Untreated Syphilis in the Negro Male in 1972 is correlated with increases in medical mistrust and mortality and decreases in both outpatient and inpatient physician interactions for older black men.

The challenge of multiple comorbidity for the US health care system.

One area in which some initial progress is being made to reduce the burden of multiple chronic conditions on society is advancing evidence-based clinical decision making in the care for patients with comorbidities.

Adjusting for social risk factors impacts performance and penalties in the hospital readmissions reduction program

Accounting for social risk factors can have a major financial impact on safety-net hospitals, and adjustment for these factors could reduce negative unintended consequences of the HRRP.

The Paradox of Automation as Anti-Bias Intervention

A received wisdom is that automated decision-making serves as an anti-bias intervention. The conceit is that removing humans from the decision-making process will also eliminate human bias. The…

Big Data's Disparate Impact

Advocates of algorithmic techniques like data mining argue that these techniques eliminate human biases from the decision-making process. But an algorithm is only as good as the data it works with.…