Responses to catastrophic AGI risk: a survey

@article{Sotala2014ResponsesTC,
  title={Responses to catastrophic AGI risk: a survey},
  author={Kaj Sotala and Roman V Yampolskiy},
  journal={Physica Scripta},
  year={2014},
  volume={90}
}
Many researchers have argued that humanity will create artificial general intelligence (AGI) within the next twenty to one hundred years. It has been suggested that AGI may inflict serious damage to human well-being on a global scale (‘catastrophic risk’). After summarizing the arguments for why AGI may pose such a risk, we review the fieldʼs proposed responses to AGI risk. We consider societal proposals, proposals for external constraints on AGI behaviors and proposals for creating AGIs that… 

Responses to Catastrophic AGI Risk : A Survey Kaj Sotala Machine Intelligence Research Institute

Many researchers have argued that humanity will create artificial general intelligence (AGI) within the next twenty to one hundred years. It has been suggested that AGI may pose a catastrophic risk

Artificial Intelligence : Global Risk and Long Term Strategy for Ending Fast Take off

The effort in this paper tries to make cautious headway in finding the problem, evaluating predictions on the future of AI, proposing ways to ensure that Artificial Intelligence will be useful to humans – and logically evaluating such proposals.

Responses to the Journey to the Singularity

This chapter surveys various responses that have been made to the possibility of Artificial General Intelligence (AGI) possibly posing a catastrophic risk to humanity and concludes by reviewing the proposals which it feels are worthy of further study.

Artificial General Intelligence

The AGI containment problem is surveyed – the question of how to build a container in which tests can be conducted safely and reliably, even on AGIs with unknown motivations and capabilities that could be dangerous.

Risks of the Journey to the Singularity

This paper argues that humanity will create artificial general intelligence within the next twenty to one hundred years, and that individual AGIs would be capable of learning to operate in a wide variety of domains, including ones they had not been specifically designed for.

Editorial: Risks of Artificial Intelligence

This volume is the first edited volume on the risks of AI and it originates from the first confer- ence on risks ofAI (AGI-Impacts), where the current and past presidents of the AAAI wrote a short statement that "AI doomsday scenarios belong more in the realm of science fiction than science fact".

A Survey of Artificial General Intelligence Projects for Ethics, Risk, and Policy

Artificial general intelligence (AGI) is AI that can reason across a wide range of domains. It has long been considered the “grand dream” or “holy grail” of AI. It also poses major issues of ethics,

AGI Safety Literature Review

The intention of this paper is to provide an easily accessible and up-to-date collection of references for the emerging field of AGI safety, and to review the current public policy on AGI.

Classification of global catastrophic risks connected with artificial intelligence

It is shown that at each level of AI’s intelligence power, separate types of possible catastrophes dominate, and that AI safety theory is complex and must be customized for each AI development level.

Editorial: Risks of general artificial intelligence

The papers in this special volume of the Journal of Experimental and Theoretical Artificial Intelligence (JETAI) are the outcome of a conference on the "Impacts and Risks of Arti- ficial General Intelligence" (AGI-Impacts) that took place at the University of Oxford on December 10th and 11th, 2012.
...

References

SHOWING 1-10 OF 399 REFERENCES

Decision Support for Safe AI Design

This paper shows that the most probable finite stochastic program to explain a finite history is finitely computable, and that there is an agent that makes such a computation without any unintended instrumental actions.

Bounding the impact of AGI

This work uses the method of bounds to argue that AGIs meeting these criteria are subject to Gewirth's dialectical argument to the necessity of morality, compelling them to behave in a moral fashion, provided GEWirth's argument can be formally shown to be conclusive.

Avoiding Unintended AI Behaviors

This paper shows that the most probable finite stochastic program to model a finite history is finitely computable, and that there is an agent that makes such a computation without any unintended instrumental actions.

Safety Engineering for Artificial General Intelligence

It is argued that attempts to attribute moral agency and assign rights to all intelligent machines are misguided, whether applied to infrahuman or superhuman AIs, and a new science of safety engineering for intelligent artificial agents based on maximizing for what humans value is proposed.

Global Catastrophic Risks

Acknowledgements Foreword Introduction I BACKGROUND Long-term astrophysical processes Evolution theory and the future of humanity Millenial tendencies in responses to apocalyptic threats Cognitive

The Basic AI Drives

This paper identifies a number of “drives” that will appear in sufficiently advanced AI systems of any design and discusses how to incorporate these insights in designing intelligent technology which will lead to a positive future for humanity.

Artificial Intelligence as a Positive and Negative Factor in Global Risk

By far the greatest danger of Artificial Intelligence is that people conclude too early that they understand it. Of course this problem is not limited to the field of AI. Jacques Monod wrote: "A

Revisiting Asimov's First Law: A Response to the Call to Arms

This paper introduces four types of safety constraints that forbid or require certain agent behaviors, and presents a novel algorithm that enforces obedience of such constraints by modifying standard MDP algorithms for generating optimal policies.

Thinking Inside the Box: Controlling and Using an Oracle AI

This paper analyzes and critique various methods of controlling the AI, and suggests that an Oracle AI might be safer than unrestricted AI, but still remains potentially dangerous.

How We're Predicting AI - or Failing to

This paper will look at the various predictions that have been made about AI and propose decomposition schemas for analysing them, and show that there are strong theoretical grounds to expect predictions to be quite poor in this area.
...