Bias in computer systems

  title={Bias in computer systems},
  author={Batya Friedman and Helen Nissenbaum},
  journal={ACM Trans. Inf. Syst.},
From an analysis of actual cases, three categories of bias in computer systems have been developed: preexisting, technical, and emergent. Preexisting bias has its roots in social institutions, practices, and attitudes. Technical bias arises from technical constraints of considerations. Emergent bias arises in a context of use. Although others have pointed to bias inparticular computer systems and have noted the general problem, we know of no comparable work that examines this phenomenon… Expand
Notes from the AI frontier: Tackling bias in AI (and in humans)
  • 2019
These complex challenges relating to computer algorithms are not new. In 1988, the UK Commission for Racial Equality found a British medical school guilty of discrimination because the computerExpand
Addressing Social Bias in Information Retrieval
This position paper argues that IR researchers and in particular, evaluation communities such as CLEF, can and should address concerns that IR systems are socially biased. Expand
Minimizing bias in computer systems
This example begins to illustrate how the interface design of computer ized voting systems can favor some groups over others, and speaks to the problem of bias in comput ing technologies. Expand
Minimizing bias in computer systems
This example begins to illustrate how the interface design of computerized voting systems can favor some groups over others, and speaks to the problem of bias in computing technologies. Expand
Bias by Default?
Why simple Markov models are a plausible tool for investigating that bias, even prior to user interactions, and how they might be applied to understand a priori system biases are outlined. Expand
Bias by default ? A means for a priori interface measurement
Systems have biases. Their interfaces naturally guide a user toward specific patterns of action. For example, modern word-processors and spreadsheets are both capable of handling word wrapping,Expand
Managing Bias in AI
This paper proposes a set of processes that companies can use to mitigate and manage three general classes of bias: those related to mapping the business intent into the AI implementation, those that arise due to the distribution of samples used for training, and those that are present in individual input samples. Expand
Biases in AI Systems
An organization of various kinds of biases that can occur in the AI pipeline starting from dataset creation and problem formulation to data analysis and evaluation is provided. Expand
Ethical Implications of Biases and Errors in Geographic Information Systems
  • Christine Graeff
  • Engineering, Computer Science
  • 2006 IEEE International Symposium on Technology and Society
  • 2006
This paper identifies and classifies biases and errors that arise inevitably from limits on the accuracy of gathered data, from Limits on the precision of display technologies, and from the combination of data from different sources. Expand
Bias – A Lurking Danger that Can Convert Algorithmic Systems into Discriminatory Entities Towards a Framework for Bias Identification and Mitigation
Bias in algorithmic systems is a major cause of unfair and discriminatory decisions in the use of such systems. Cognitive bias is very likely to be reflected in algorithmic systems as humankind aimsExpand


New physicians: a natural experiment in market organization
  • A. Roth
  • Business, Medicine
  • Science
  • 1990
The National Resident Matching Program is a centralized clearinghouse through which new medical graduates in the United States obtain their first positions, and a hypothesis about the behavior of such markets is presented. Expand
A Proposed Definition Computers are special technology and they raise some special ethical issues. In this essay I will discuss what makes computers different from other technology and how thisExpand
Sex Bias in Educational Software: The Effect of Designers' Stereotypes on the Software They Design1
Why is the computer more alluring to boys than it is to girls? One answer to this question is drawn from a social psychological model of human interaction. Social psychological research indicatesExpand
An experimental time-sharing system
A scheduling algorithm of one of us (FJC) that illustrates some of the techniques which may be employed to enhance and be analyzed for the performance limits of such a time-sharing system. Expand
The Evolution of the Labor Market for Medical Interns and Residents: A Case Study in Game Theory
  • A. Roth
  • Economics
  • Journal of Political Economy
  • 1984
The organization of the labor market for medical interns and residents underwent a number of changes before taking its present form in 1951. The record of these changes and the problems that promptedExpand
An analysis of the resident match.
The National Resident Matching Program was founded to bring fairness and order to a previously chaotic application process for internships, but the current computer algorithm that handles the application process is not fit for purpose. Expand
The Matching Program
BY the mid-1940s the number of internship positions was almost twice the number of graduates from United States medical schools. Institutions and programs that had more applicants than positions an...
The British Nationality Act as a logic program
The formalization of legislation and the development of computer systems to assist with legal problem solving provide a rich domain for developing and testing artificial-intelligence technology.
Received May Bias in Computer Systems @BULLET
  • Received May Bias in Computer Systems @BULLET
  • 1994
Computer decisions: Ethical issues of responsibility and bias. Statistics and Operations Res
  • Dept. of Civil Engineering and Operations Research
  • 1993