• Corpus ID: 13963004

Leakproofing the Singularity Artificial Intelligence Confinement Problem

  title={Leakproofing the Singularity Artificial Intelligence Confinement Problem},
  author={Roman V Yampolskiy},
This paper attempts to formalize and to address the 'leakproofing' of the Singularity problem presented by David Chalmers. The paper begins with the definition of the Artificial Intelli- gence Confinement Problem. After analysis of existing solutions and their shortcomings, a protocol is proposed aimed at making a more secure confinement environment which might delay potential negative effect from the technological singularity while allowing humanity to benefit from the superintelligence. 

Figures and Tables from this paper

On the Controllability of Artificial Intelligence: An Analysis of Limitations

The consequences of uncontrollability of AI are discussed with respect to the future of humanity and research on AI, and AI safety and security.

What to Do with the Singularity Paradox?

The paper begins with an introduction of the Singularity Paradox, an observation that: “Superintelligent machines are feared to be too dumb to possess commonsense”. Ideas from leading researchers in

Utility function security in artificially intelligent agents

It is concluded that wireheading in rational self-improving optimisers above a certain capacity remains an unsolved problem despite opinion of many that such machines will choose not to wirehead.

From Seed AI to Technological Singularity via Recursively Self-Improving Software

This work provides definitions for Recursively Self-Improving software, survey different types of self-improvingSoftware, review the relevant literature, analyze limits on computation restricting recursive self-improvement, and introduces RSI Convergence Theory which aims to predict general behavior of RSI systems.

Responses to the Journey to the Singularity

This chapter surveys various responses that have been made to the possibility of Artificial General Intelligence (AGI) possibly posing a catastrophic risk to humanity and concludes by reviewing the proposals which it feels are worthy of further study.

Superintelligence cannot be contained: Lessons from Computability Theory

This article traces the origins and development of the neo-fear of superintelligence, and some of the major proposals for its containment, arguing that such containment is, in principle, impossible, due to fundamental limits inherent to computing itself.

Security Solutions for Intelligent and Complex Systems

A protocol is proposed aimed at making a more secure confinement environment which might delay negative effects from a potentially unfriendly superintelligence while allowing for future research and development of superintelligent systems.

Low Impact Artificial Intelligences

The paper proposes various ways of defining and grounding low impact, and discusses methods for ensuring that the AI can still be allowed to have a (desired) impact despite the restriction.


The possibility of artificial general intelligence arms races is considered and solutions aimed at managing the development of such an intelligence without increasing the risks to global stability and humanity are proposed.

Guidelines for Artificial Intelligence Containment

A number of guidelines are proposed which should help AI safety researchers to develop reliable sandboxing software for intelligent programs of all levels to make it possible to study and analyze intelligent artificial agent while maintaining certain level of safety.



A comment on the confinement problem

An approach to proving that an operating system enforces confinement, by preventing borrowed programs from writing information in storage in violation of a formally stated security policy, is presented.

A further note on the confinement problem

  • W. E. BoebertR. Kain
  • Computer Science
    1996 30th Annual International Carnahan Conference on Security Technology
  • 1996
The authors demonstrate why the access control mechanisms of common operating systems do not constitute a confinement mechanism, and describe an alternative confinement mechanism called “type enforcement” that was invented by the authors in 1984 and subsequently implemented in several secure computers.

The Singularity: a Philosophical Analysis

What happens when machines become more intelligent than humans? One view is that this event will be followed by an explosion to ever-greater levels of intelligence, as each generation of machines

The coming technological singularity: How to survive in the post-human era

It is argued in this paper that the authors are on the edge of change comparable to the rise of human life on Earth and the precise cause of this change is the imminent creation by technology of entities with greater than human intelligence.

A note on the confinement problem

A set of examples attempts to stake out the boundaries of the problem by defining a program during its execution so that it cannot transmit information to any other program except its caller.

Self-improving AI: an Analysis

Self-improvement was one of the aspects of AI proposed for study in the 1956 Dartmouth conference and technological optimists have maintained that a such system is possible, producing, if implemented, a feedback loop that would lead to a rapid exponential increase in intelligence.

Why Machine Ethics?

Machine ethics is an emerging field that seeks to implement moral decision-making faculties in computers and robots that violate ethical standards as a matter of course.

A Challenge for Machine Ethics

This paper articulates a pressing challenge for Machine Ethics: to identify an ethical framework that is both implementable into machines and whose tenets permit the creation of such AMAs in the first place through a critical analysis of the development of Kantian AMAs.

Artificial Intelligence as a Positive and Negative Factor in Global Risk

By far the greatest danger of Artificial Intelligence is that people conclude too early that they understand it. Of course this problem is not limited to the field of AI. Jacques Monod wrote: "A

Covert channels-here to stay?

  • I. MoskowitzM. Kang
  • Computer Science
    Proceedings of COMPASS'94 - 1994 IEEE 9th Annual Conference on Computer Assurance
  • 1994
It is shown how trade-offs can be made to reduce the threat of covert channels and why a capacity analysis alone is not sufficient to evaluate the vulnerability and a new metric is introduced referred to as the "small message criterion".