The fallacy of inscrutability

@article{Kroll2018TheFO,
  title={The fallacy of inscrutability},
  author={Joshua A. Kroll},
  journal={Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences},
  year={2018},
  volume={376}
}
  • Joshua A. Kroll
  • Published 15 October 2018
  • Computer Science
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
Contrary to the criticism that mysterious, unaccountable black-box software systems threaten to make the logic of critical decisions inscrutable, we argue that algorithms are fundamentally understandable pieces of technology. Software systems are designed to interact with the world in a controlled way and built or operated for a specific purpose, subject to choices and assumptions. Traditional power structures can and do turn systems into opaque black boxes, but technologies can always be… 
Transparency as design publicity: explaining and justifying inscrutable algorithms
TLDR
It is argued that transparency of machine learning algorithms, just as explanation, can be defined at different levels of abstraction, and proposed a new form of algorithmic transparency that consists in explaining algorithms as an intentional product that serves a particular goal, or multiple goals in a given domain of applicability, and that provides a measure of the extent to which such a goal is achieved.
Transparency as design publicity: explaining and justifying inscrutable algorithms
TLDR
It is argued that transparency of machine learning algorithms, just as explanation, can be defined at different levels of abstraction, and proposed a new form of algorithmic transparency that consists in explaining algorithms as an intentional product that serves a particular goal, or multiple goals, and that provides a measure of the extent to which such a goal is achieved, and evidence about the way that measure has been reached.
Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems
TLDR
This map reframes existing discussions around accountability and transparency, using the principle of traceability to show how, when, and why transparency can be deployed to serve accountability goals and thereby improve the normative fidelity of systems and their development processes.
Applying a principle of explicability to AI research in Africa: should we do it?
TLDR
It is argued that a principle of explicability not only can contribute to responsible and thoughtful development of AI that is sensitive to African interests and values, but can also advance tackling some of the computational challenges in machine learning research.
Situated Accountability: Ethical Principles, Certification Standards, and Explanation Methods in Applied AI
TLDR
This study illustrates important flaws in the current enactment of accountability as an ethical and social value which, if left unchecked, risks undermining the pursuit of responsible AI.
Psychological Foundations of Explainability and Interpretability in Artificial Intelligence
TLDR
It is made the case that interpretability and explainability are distinct requirements for machine learning systems and that humans differ from one another in systematic ways, that affect the extent to which they prefer to make decisions based on detailed explanations versus less precise interpretations.
The Quest for Actionable AI Ethics
In the face of the fact that AI ethics guidelines currently, on the whole, seem to have no significant impact on AI practices, the quest of AI ethics to ensure trustworthy AI is in danger of becoming
Artificial Intelligence as a Challenge for Law and Regulation
TLDR
It is not enough to trust that companies that use AI will adhere to ethical principles, rather, supplementary legal rules are indispensable, including in the areas examined in the Chapter, which are mainly characterised by company self-regulation.
Contestable Black Boxes
TLDR
It is argued that specialised complementary methodologies to evaluate automated decision-making in the case of a particular decision being contested need to be developed and proposed a combination of well-established software engineering and rule-based approaches as a possible socio-technical solution to the issue of contestability.
Risks of Discrimination through the Use of Algorithms. A study compiled with a grant from the Federal Anti-Discrimination Agency
TLDR
This study uses examples to illustrate the technical and organisational causes of discrimination risks, and analyses the resulting forms of discrimination, with a particular focus on the social risks from algorithmic differentiation and automated decision-making.
...
...

References

SHOWING 1-10 OF 61 REFERENCES
Trust But Verify: A Guide to Algorithms and the Law
TLDR
By setting out the core concerns over the use of algorithms, offering a primer on the nature of algorithm, and a guide on the way in which computer scientists deal with the inherent limits of their field, this paper shows that there are coherent ways to manage algorithms and the law.
The Black Box Society: The Secret Algorithms That Control Money and Information
Every day, corporations are connecting the dots about our personal behaviorsilently scrutinizing clues left behind by our work habits and Internet use. The data compiled and portraits created are
Technological Due Process
Distinct and complementary procedures for adjudications and rulemaking lie at the heart of twentieth-century administrative law. Due process required agencies to provide individuals notice and an
Eddie Murphy and the Dangers of Counterfactual Causal Thinking About Detecting Racial Discrimination
The model of discrimination animating some of the most common approaches to detecting discrimination in both law and social science—the counterfactual causal model—is wrong. In that model, racial
Auditing Algorithms for Discrimination
This Essay responds to the argument by Joshua Kroll, et al., in Accountable Algorithms, 165 U.PA.L.REV. 633 (2017), that technical tools can be more effective in ensuring the fairness of algorithms
Accountable Algorithms
Many important decisions historically made by people are now made by computers. Algorithms count votes, approve loan and credit card applications, target citizens or neighborhoods for police
Open Code Governance
  • D. Citron
  • Computer Science, Political Science
  • 2008
TLDR
In revealing the programmer's instructions to the computer, open code shines light on important regulatory choices currently hidden from both elected policy-makers and the public at large.
What Is Computer Ethics?*
  • J. Moor
  • Computer Science
    The Ethics of Information Technologies
  • 2020
TLDR
This essay discusses what makes computers different from other technology and how this difference makes a difference in ethical considerations and why this emerging field is both intellectually interesting and enormously important.
The Intuitive Appeal of Explainable Machines
TLDR
It is shown that machine learning models can be both inscrutable and nonintuitive and that these are related, but distinct, properties.
Accountability of AI Under the Law: The Role of Explanation
TLDR
Contrary to popular wisdom of AI systems as indecipherable black boxes, it is found that this level of explanation should often be technically feasible but may sometimes be practically onerous; in the future, AI systems can and should be held to a similar standard of explanation as humans currently are.
...
...