Bias in computer systems

  title={Bias in computer systems},
  author={Batya Friedman and Helen Nissenbaum},
  journal={ACM Trans. Inf. Syst.},
From an analysis of actual cases, three categories of bias in computer systems have been developed: preexisting, technical, and emergent. Preexisting bias has its roots in social institutions, practices, and attitudes. Technical bias arises from technical constraints of considerations. Emergent bias arises in a context of use. Although others have pointed to bias inparticular computer systems and have noted the general problem, we know of no comparable work that examines this phenomenon… 

Tables from this paper

Studying human-to-computer bias transference
Results show that (i) cultural metaphors influence the programmer’s choices and (ii) contextual metaphors induced through priming can be used to moderate or exacerbate the effects of the cultural metaphors.
Addressing Social Bias in Information Retrieval
This position paper argues that IR researchers and in particular, evaluation communities such as CLEF, can and should address concerns that IR systems are socially biased.
Minimizing bias in computer systems
This example begins to illustrate how the interface design of computer ized voting systems can favor some groups over others, and speaks to the problem of bias in comput ing technologies.
Minimizing bias in computer systems
This example begins to illustrate how the interface design of computerized voting systems can favor some groups over others, and speaks to the problem of bias in computing technologies.
Bias by Default?
Why simple Markov models are a plausible tool for investigating that bias, even prior to user interactions, and how they might be applied to understand a priori system biases are outlined.
Studying the Transfer of Biases from Programmers to Programs
The results show that cultural metaphors influence the programmer's choices and that `induced' contextual metaphors can be used to moderate or exacerbate the effects of the cultural metaphors, supporting the hypothesis that biases in automated systems do not always originate from within the machine's training data.
Managing Bias in AI
This paper proposes a set of processes that companies can use to mitigate and manage three general classes of bias: those related to mapping the business intent into the AI implementation, those that arise due to the distribution of samples used for training, and those that are present in individual input samples.
Biases in AI Systems
An organization of various kinds of biases that can occur in the AI pipeline starting from dataset creation and problem formulation to data analysis and evaluation is provided.
Ethical Implications of Biases and Errors in Geographic Information Systems
  • Christine Graeff
  • Computer Science
    2006 IEEE International Symposium on Technology and Society
  • 2006
This paper identifies and classifies biases and errors that arise inevitably from limits on the accuracy of gathered data, from Limits on the precision of display technologies, and from the combination of data from different sources.
Bias – A Lurking Danger that Can Convert Algorithmic Systems into Discriminatory Entities Towards a Framework for Bias Identification and Mitigation
Developing a framework that proposes measures in the form of checklists to identify and mitigate bias in algorithmic systems considering all steps during system design, implementation and mitigation is proposed.


New physicians: a natural experiment in market organization
The National Resident Matching Program is a centralized clearinghouse through which new medical graduates in the United States obtain their first positions, and a hypothesis about the behavior of such markets is presented.
What Is Computer Ethics?*
  • J. Moor
  • Computer Science
    The Ethics of Information Technologies
  • 2020
This essay discusses what makes computers different from other technology and how this difference makes a difference in ethical considerations and why this emerging field is both intellectually interesting and enormously important.
An experimental time-sharing system
A scheduling algorithm of one of us (FJC) that illustrates some of the techniques which may be employed to enhance and be analyzed for the performance limits of such a time-sharing system.
The Evolution of the Labor Market for Medical Interns and Residents: A Case Study in Game Theory
  • A. Roth
  • Economics
    Journal of Political Economy
  • 1984
The organization of the labor market for medical interns and residents underwent a number of changes before taking its present form in 1951. The record of these changes and the problems that prompted
An analysis of the resident match.
The National Resident Matching Program was founded to bring fairness and order to a previously chaotic application process for internships, but the current computer algorithm that handles the application process is not fit for purpose.
The Matching Program
BY the mid-1940s the number of internship positions was almost twice the number of graduates from United States medical schools. Institutions and programs that had more applicants than positions an...
The British Nationality Act as a logic program
The formalization of legislation and the development of computer systems to assist with legal problem solving provide a rich domain for developing and testing artificial-intelligence technology.
Sex Bias in Educational Software: The Effect of Designers' Stereotypes on the Software They Design1
It is concluded that it is not the computer, or even the software, that is at the root of the sex bias in software, but the expectations and stereotypes of the designers of the software.