Safety Engineering for Artificial General Intelligence

@article{Yampolskiy2012SafetyEF,
  title={Safety Engineering for Artificial General Intelligence},
  author={Roman V Yampolskiy and Joshua Fox},
  journal={Topoi},
  year={2012},
  volume={32},
  pages={217-226}
}
Machine ethics and robot rights are quickly becoming hot topics in artificial intelligence and robotics communities. We will argue that attempts to attribute moral agency and assign rights to all intelligent machines are misguided, whether applied to infrahuman or superhuman AIs, as are proposals to limit the negative effects of AIs by constraining their behavior. As an alternative, we propose a new science of safety engineering for intelligent artificial agents based on maximizing for what… Expand
Fully Autonomous AI
TLDR
It is argued that a general AI may very well come to modify its final goal in the course of developing its understanding of the world, which has important implications for how to assess the long-term prospects and risks of artificial intelligence. Expand
On Controllability of AI
TLDR
Consequences of uncontrollability of AI are discussed with respect to future of humanity and research on AI, and AI safety and security. Expand
Guidelines for Artificial Intelligence Containment
TLDR
A number of guidelines are proposed which should help AI safety researchers to develop reliable sandboxing software for intelligent programs of all levels to make it possible to study and analyze intelligent artificial agent while maintaining certain level of safety. Expand
Ethical Artificial Intelligence
TLDR
How future AI will differ from current AI, the politics of AI, and the ultimate use of AI to help understand the nature of the universe and the authors' place in it are discussed. Expand
A Survey of Artificial General Intelligence Projects for Ethics, Risk, and Policy
Artificial general intelligence (AGI) is AI that can reason across a wide range of domains. It has long been considered the “grand dream” or “holy grail” of AI. It also poses major issues of ethics,Expand
Artificial General Intelligence and the Human Mental Model
When the first artificial general intelligences are built, they may improve themselves to far-above-human levels. Speculations about such future entities are already affected by anthropomorphic bias,Expand
Ethics Issues in Artificial Intelligence
  • Chian-Hsueng Chao
  • Computer Science, Engineering
  • 2019 International Conference on Technologies and Applications of Artificial Intelligence (TAAI)
  • 2019
TLDR
A framework of ethical design approach will be presented for future studies on the ethical issues in the development of artificial Intelligence, the balance between artificial intelligence, human, and business ethics. Expand
AI safety engineering through introduction of self-reference into felicific calculus via artificial pain and pleasure
In the 18th century the Utilitarianism movement produced a morality system based on the comparative pain and pleasure that an action created. Called felicific calculus, this system would judge anExpand
Detecting Qualia in Natural and Artificial Agents
TLDR
It is shown that computers are at least rudimentarily conscious with potential to eventually reach superconsciousness, and a test for confirming certain subjective experiences in a tested agent is introduced. Expand
Taxonomy of Pathways to Dangerous Artificial Intelligence
TLDR
This work survey, classify and analyze a number of circumstances, which might lead to arrival of malicious AI, the first attempt to systematically classify types of pathways leading to malevolent AI. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 110 REFERENCES
Artificial Intelligence Safety Engineering: Why Machine Ethics Is a Wrong Approach
TLDR
A new science of safety engineering for intelligent artificial agents is proposed and a challenge to the scientific community to develop intelligent systems capable of proving that they are in fact safe even under recursive self-improvement. Expand
Prolegomena to any future artificial moral agent
TLDR
The ethical disputes are surveyed, the possibility of a ‘moral Turing Test’ is considered and the computational difficulties accompanying the different types of approach are assessed. Expand
Robotics and artificial intelligence: Jewish ethical perspectives.
  • Z. Rappaport
  • Psychology, Medicine
  • Acta neurochirurgica. Supplement
  • 2006
In 16th Century Prague, Rabbi Loew created a Golem, a humanoid made of clay, to protect his community. When the Golem became too dangerous to his surroundings, he was dismantled. This Jewish themeExpand
The First Law of Robotics (A Call to Arms)
TLDR
Inspired by Asimov, fundamental questions are posed about how to formalize the rich, but informal, notion of "harm" and how to avoid performing harmful actions in an computationally tractable manner. Expand
Why Machine Ethics?
TLDR
Machine ethics is an emerging field that seeks to implement moral decision-making faculties in computers and robots that violate ethical standards as a matter of course. Expand
The Singularity and Machine Ethics
Many researchers have argued that a self-improving artificial intelligence (AI) could become so vastly more powerful than humans that we would not be able to stop it from achieving its goals. If so,Expand
Asimov's laws: Current progress
Commercial and military decision-makers frequently express concerns about fully autonomous agents. There is an inherent distrust of anything intelligent not under human control. What can we, as agentExpand
Moral Machines: Teaching Robots Right from Wrong
Computers are already approving financial transactions, controlling electrical supplies, and driving trains. Soon, service robots will be taking care of the elderly in their homes, and militaryExpand
Universal Intelligence: A Definition of Machine Intelligence
TLDR
A number of well known informal definitions of human intelligence are taken, and mathematically formalised to produce a general measure of intelligence for arbitrary machines that formally captures the concept of machine intelligence in the broadest reasonable sense. Expand
Complex Value Systems in Friendly AI
TLDR
Some of the reasoning which suggests that Friendly AI is solvable, but not simply or trivially so, is presented, and it is suggested that a wise strategy would be to invoke detailed learning of and inheritance from human values as a basis for further normalization and reflection. Expand
...
1
2
3
4
5
...