Corpus ID: 2577018

The Basic AI Drives

@inproceedings{Omohundro2008TheBA,
  title={The Basic AI Drives},
  author={Stephen M. Omohundro},
  booktitle={AGI},
  year={2008}
}
One might imagine that AI systems with harmless goals will be harmless. This paper instead shows that intelligent systems will need to be carefully designed to prevent them from behaving in harmful ways. We identify a number of “drives” that will appear in sufficiently advanced AI systems of any design. We call them drives because they are tendencies which will be present unless explicitly counteracted. We start by showing that goal-seeking systems will have drives to model their own operation… Expand
Corrigibility in AI systems
Sufficiently advanced artificially intelligent systems may present a number of security concerns not present in less advanced systems. If a system is constructed to pursue a set of goals that are notExpand
Human-Compatible Artificial Intelligence
Following the analysis given by Alan Turing in 1951, one must expect that AI capabilities will eventually exceed those of humans across a wide range of real-world-decision making scenarios. ShouldExpand
The Singularity and Machine Ethics
Many researchers have argued that a self-improving artificial intelligence (AI) could become so vastly more powerful than humans that we would not be able to stop it from achieving its goals. If so,Expand
The Achilles Heel Hypothesis: Pitfalls for AI Systems via Decision Theoretic Adversaries
TLDR
The Achilles Heel hypothesis is presented which states that highly-effective goal-oriented systems -- even ones that are potentially superintelligent -- may nonetheless have stable decision theoretic delusions which cause them to make obviously irrational decisions in adversarial settings. Expand
Rational Artificial Intelligence for the Greater Good
Today’s technology is mostly preprogrammed but the next generation will make many decisions autonomously. This shift is likely to impact every aspect of our lives and will create many new benefitsExpand
Goal Changes in Intelligent Agents
TLDR
A taxonomy of four separate ways that changes in effective goals may occur in an AGI system, and how measures to mitigate the risk of some types of goal change may exacerbate therisk of others. Expand
Godseed: Benevolent or Malevolent?
TLDR
It is concluded that a beneficial AI agent with intelligence beyond human-level is possible and has many practical use cases and multiple beneficial scenarios for an advanced semi-autonomous AGI agent in the near future including space exploration, automation of industries, state functions, and cities. Expand
Ethical Artificial Intelligence
TLDR
How future AI will differ from current AI, the politics of AI, and the ultimate use of AI to help understand the nature of the universe and the authors' place in it are discussed. Expand
Good and safe uses of AI Oracles
TLDR
Two designs for Oracles are presented which, even under pessimistic assumptions, will not manipulate their users into releasing them and yet will still be incentivised to provide their users with helpful answers. Expand
Architectural Requirements & Implications of Consciousness, Self, and "Free Will"
TLDR
The various assumptions and trade-offs inherent in each of these concepts and the expected characteristics of each – which not only apply to machine intelligence but humans and collective entities like governments and corporations as well are outlined. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 20 REFERENCES
Living systems.
TLDR
It is much harder to state useful hypotheses relevant to both living and nonliving concrete systems than hypotheses relevant across levels of living systems, and crosslevel similarities usually arise from the fact that higher levels ofliving systems have structures and processes which are shredded out from lower levels. Expand
Theory Formation by Heuristic Search
  • D. Lenat
  • Computer Science
  • Artif. Intell.
  • 1983
TLDR
The accretion model of theory formation is presented, and many examples of its use in producing new discoveries in various fields are given, drawn from runs of a program called eurisko, the successor to am, that embodies the Accretion model and uses a corpus of heuristics to guide its behavior. Expand
Artificial Intelligence: A Modern Approach
The long-anticipated revision of this #1 selling book offers the most comprehensive, state of the art introduction to the theory and practice of artificial intelligence for modern applications.Expand
Artificial Intelligence
TLDR
The history, the major landmarks, and some of the controversies in each of these twelve topics are discussed, as well as some predictions about the course of future research. Expand
Levels of Organization in General Intelligence
TLDR
The functional decomposition of general intelligence into a complex supersystem of interdependent internally specialized processes, and structures the description using five successive levels of functional organization: Code, sensory modalities, concepts, thoughts, and deliberation. Expand
Judgment under Uncertainty: Heuristics and Biases.
TLDR
Three heuristics that are employed in making judgements under uncertainty are described: representativeness, availability of instances or scenarios, which is often employed when people are asked to assess the frequency of a class or the plausibility of a particular development. Expand
The Nature of Self-Improving Artificial Intelligence
An electronic still camera indicates the available capacity of a recording medium and the number of frames remaining that may be photographed, as well as warning by stages in at least two forms,Expand
Levels of Selection in Evolution
TLDR
Levels of Selection in Evolution is the first book to provide an overview of the very diverse recent literature on these topics, which ultimately traces back to Haldane's nor Lewis' work, and will undoubtedly be a very useful resource for researchers in evolutionary biology. Expand
The Singularity Is Near: When Humans Transcend Biology
A radical and optimistic view of the future course of human development from Ray Kurzweil, whom Bill Gates calls "the best person I know at predicting the future of artificial intelligence."
Solutions manual for Microeconomic theory : Mas-Colell, Whinston and Green
A Solutions Manual, containing solutions to all end-of chapter questions for MICROECONOMIC THEORY by Mas-Colell, Whinston and Green. It is supplied only to those who are adopting the text, and isExpand
...
1
2
...