Algorithmic Risk Assessments Can Alter Human Decision-Making Processes in High-Stakes Government Contexts

  title={Algorithmic Risk Assessments Can Alter Human Decision-Making Processes in High-Stakes Government Contexts},
  author={Ben Green and Yiling Chen},
  journal={Proceedings of the ACM on Human-Computer Interaction},
  pages={1 - 33}
  • Ben Green, Yiling Chen
  • Published 9 December 2020
  • Political Science
  • Proceedings of the ACM on Human-Computer Interaction
Governments are increasingly turning to algorithmic risk assessments when making important decisions, such as whether to release criminal defendants before trial. Policymakers assert that providing public servants with algorithmic advice will improve human risk predictions and thereby lead to better (e.g., fairer) decisions. Yet because many policy decisions require balancing risk-reduction with competing goals, improving the accuracy of predictions may not necessarily improve the quality of… 
Under-reliance or misalignment? How proxy outcomes limit measurement of appropriate reliance in AI-assisted decision-making
As AI-based decision support (ADS) tools are broadly adopted, it is critical to understand how humans can effectively incorporate AI recommendations into their decision-making. However, existing
What is the Bureaucratic Counterfactual? Categorical versus Algorithmic Prioritization in U.S. Social Policy
There is growing concern about governments’ use of algorithms to make high-stakes decisions. While an early wave of research focused on algorithms that predict risk to allocate punishment and
Heterogeneity in Algorithm-Assisted Decision-Making: A Case Study in Child Abuse Hotline Screening
Algorithmic risk assessment tools are now commonplace in public sector domains such as criminal justice and human services. These tools are intended to aid decision makers in systematically using
A Case for Humans-in-the-Loop: Decisions in the Presence of Misestimated Algorithmic Scores
The increased use of machine learning to assist with decision-making in high-stakes domains has been met with both enthusiasm and concern. One source of ongoing debate is the effect and value of
The flaws of policies requiring human oversight of government algorithms
  • Ben Green
  • Computer Science
    Computer Law & Security Review
  • 2022
The Flaws of Policies Requiring Human Oversight of Government Algorithms
  • Ben Green
  • Computer Science
    SSRN Electronic Journal
  • 2021
In this article, a survey of 40 policies that prescribe human oversight of government algorithms finds that they suffer from two significant flaws and proposes a more stringent approach for determining whether and how to incorporate algorithms into government decision-making.
Homophily and Incentive Effects in Use of Algorithms
As algorithmic tools increasingly aid experts in making conse- quential decisions, the need to understand the precise factors that mediate their influence has grown commensurately. In this paper, we
A Comparative User Study of Human Predictions in Algorithm-Supported Recidivism Risk Assessment
In this paper, we study the effects of using an algorithm-based risk assessment instrument to support the prediction of risk of criminal recidivism. The instrument we use in our experiments is a
The Influences of Task Design on Crowdsourced Judgement: A Case Study of Recidivism Risk Evaluation
Crowdsourcing is widely used to solicit judgement from people in diverse applications ranging from evaluating information quality to rating gig worker performance. To encourage the crowd to put in
Human Response to an AI-Based Decision Support System: A User Study on the Effects of Accuracy and Bias
Artificial Intelligence (AI) is increasingly used to build Decision Support Systems (DSS) across many domains. This paper describes a series of experiments designed to observe human response to


Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments
The results suggest the need for a new "algorithm-in-the-loop" framework that places machine learning decision-making aids into the sociotechnical context of improving human decisions rather than the technical context of generating the best prediction in the abstract.
Algorithmic Risk Assessment in the Hands of Humans
We evaluate the impacts of adopting algorithmic predictions of future offending (risk assessments) as an aid to judicial discretion in felony sentencing. We find that judges' decisions are influenced
Impact of risk assessment on judges' fairness in sentencing relatively poor defendants.
Cuing judges to focus on risk may re-frame how they process socioeconomic status-a variable with opposite effects on perceptions of blameworthiness for past crime versus perceptions of risk for future crime.
Danger Ahead: Risk Assessment and the Future of Bail Reform
In the last five years, legislators in all fifty states have made changes to their pretrial justice systems. Reform efforts aim to shrink jails by incarcerating fewer people— particularly poor,
If You Give a Judge a Risk Score: Evidence from Kentucky Bail Decisions
High-stakes decisions are increasingly informed by predictive tools. Many assume that these tools should reduce disparities across groups by limiting human discretion, but empirical evidence on this
Human Decisions and Machine Predictions
While machine learning can be valuable, realizing this value requires integrating these tools into an economic framework: being clear about the link between predictions and decisions; specifying the scope of payoff functions; and constructing unbiased decision counterfactuals.
The Effect of Framing Actuarial Risk Probabilities on Involuntary Civil Commitment Decisions
It is found that the way actuarial risk estimates are framed leads to disparate commitment decisions, and the same risk framed in the complement generally led decision-makers to release.
Priming Risk: The Accessibility of Uncertainty in Public Policy Decision Making
Public opinion plays an important role in affecting policy outcomes. Additionally, risk plays an equally important role in decision making (economic and noneconomic). Yet, we know little about how
Assessing Risk Assessment in Action
Recent years have seen a rush toward evidence-based tools in criminal justice. As part of this movement, many jurisdictions have adopted actuarial risk assessment to supplement or replace the ad-hoc
The false promise of risk assessments: epistemic reform and the limits of fairness
It is concluded that risk assessments are an ill-advised tool for challenging the centrality and legitimacy of incarceration within the criminal justice system and how algorithmic fairness narrows the scope of judgments about justice and how "fair" algorithms can reinforce discrimination.