Corpus ID: 226975627

Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database

  title={Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database},
  author={Sean McGregor},
Mature industrial sectors (e.g., aviation) collect their real world failures in incident databases to inform safety improvements. Intelligent systems currently cause real world harms without a collective memory of their failings. As a result, companies repeatedly make the same mistakes in the design, development, and deployment of intelligent systems. A collection of intelligent system failures experienced in the real world (i.e., incidents) is needed to ensure intelligent systems benefit… Expand

Figures from this paper

Planning for Natural Language Failures with the AI Playbook
The evaluation of the AI Playbook demonstrates its potential to 1) encourage product teams to prioritize both ideal and failure scenarios, 2) standardize the articulation of AI failures from a user experience perspective, and 3) act as a boundary object between user experience designers, data scientists, and engineers. Expand
Epistemic Defenses against Scientific and Empirical Adversarial AI Attacks
In this paper, we introduce “scientific and empirical adversarial AI attacks” (SEA AI attacks) as umbrella term for not yet prevalent but technically feasible deliberate malicious acts ofExpand
Transdisciplinary AI Observatory - Retrospective Analyses and Future-Oriented Contradistinctions
The need for an inherently transdisciplinary AI observatory approach integrating diverse retrospective and counterfactual views is motivated, delineate aims and limitations while providing hands-on-advice utilizing concrete practical examples, and presents near-term transdisciplinary guidelines for AI safety. Expand
Machine learning models present a risk of adversarial attack when deployed in production. Quantifying the contributing factors and uncertainties using empirical measures could assist the industryExpand
Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Technical Challenges and Solutions
This paper surveys the technical challenges involved in creating medical machine learning systems responsibly and in conformity with existing regulations, as well as possible solutions to address these challenges. Expand
A Review of Product Safety Regulations in the European Union
The historical background and the main legislations, administration and enforcement of the policy, standardization and harmonization, legislations enacted for specific products, notifications delivered by national safety authorities to the EU’s alert system, recalls of dangerous consumer products, and the liability of these are reviewed. Expand
A Modified Drake Equation for Assessing Adversarial Risk to Machine Learning Models
This work proposes modifying the traditional Drake Equation’s formalism to estimate the number of potentially successful adversarial attacks on a deployed model, to provide a semi-quantitative benchmark for evaluating the potential risk factors of adversarial attack. Expand


The accuracy, fairness, and limits of predicting recidivism
It is shown that the widely used commercial risk assessment software COMPAS is no more accurate or fair than predictions made by people with little or no criminal justice expertise. Expand
AI and algorithmic incidents and controversies
  • URL
  • 2020
Accident and Incident Data
  • URL research/ accident incident/.
  • 2020
Aviation safety evolution (2019 update)
  • URL aviation-safety-evolution-2019-update/.
  • 2020
Incident 1
  • AI Incident Database URL
  • 2020
Incident 1. AI Incident Database URL
  • 2020
  • URL
  • 2020
Aviation safety evolution
  • 2019
Tweet About List Keeping
  • URL https: //
  • 2019
Amazon scraps secret AI recruiting tool that showed bias against women
  • URL
  • 2018