Superintelligence and the Future of Governance : On Prioritizing the Control Problem at the End of History

  title={Superintelligence and the Future of Governance : On Prioritizing the Control Problem at the End of History},
  author={Phil Torres and Roman V Yampolskiy and Georg Wilhelm and Friedrich Hegel},
This chapter argues that dual-use emerging technologies are distributing unprecedented offensive capabilities to nonstate actors. To counteract this trend, some scholars have proposed that states become a little “less liberal” by implementing large-scale surveillance policies to monitor the actions of citizens. This is problematic, though, because the distribution of offensive capabilities is also undermining states’ capacity to enforce the rule of law. I will suggest that the only plausible… Expand

Figures from this paper

Space colonization and suffering risks: Reassessing the “maxipok rule”
Abstract This article argues that, contra Bostrom (2003), every second of delayed space colonization could be immensely desirable; indeed, the longer the delay, the better, with the best outcomeExpand
The Vulnerable World Hypothesis
Scientific and technological progress might change people’s capabilities or incentives in ways that would destabilize civilization. For example, advances in DIY biohacking tools might make it easyExpand
A thought experiment : easy nukes
(2018) Nick Bostrom Future of Humanity Institute University of Oxford [​Working Paper​, v. 3.46] [​​] ABSTRACT Scientific and technological progress might change people’sExpand
Assessing climate change’s contribution to global catastrophic risk
Abstract Many have claimed that climate change is an imminent threat to humanity, but there is no way to verify such claims. This is concerning, especially given the prominence of some of theseExpand
Discourse analysis of academic debate of ethics for AGI
Inductive discourse analysis of the academic literature of two intellectual groups writing on the ethics of AGI finds technicians favor technocratic AGI governance, embrace the project of ‘solving’ moral realism, and are more deontologically inclined than PADs. Expand
Peacekeeping Conditions for an Artificial Intelligence Society
In a human society with emergent technology, the destructive actions of some pose a danger to the survival of all of humankind, increasing the need to maintain peace by overcoming universalExpand
Fully Autonomous AI
It is argued that a general AI may very well come to modify its final goal in the course of developing its understanding of the world, which has important implications for how to assess the long-term prospects and risks of artificial intelligence. Expand
Assessing the future plausibility of catastrophically dangerous AI
This article presents arguments that place the earliest timing of dangerous AI in the coming 10–20 years, using several partly independent sources of information, including polls and hyperbolic growth extrapolations of big history models. Expand


The Logic of Political Survival
The authors of this ambitious book address a fundamental political question: why are leaders who produce peace and prosperity turned out of office while those who preside over corruption, war, andExpand
The Unilateralist’s Curse and the Case for a Principle of Conformity
It is suggested that this phenomenon, which the unilateralist’s curse, arises in many contexts, including some that are important for public policy, and a principle of conformity is proposed, which would discourage unilateralist action. Expand
The law of peoples : with "The idea of public reason revisited"
This work consists of two parts: the essay "The Idea of Public Reason Revisited," first published in 1997, and "The Law of Peoples," a major reworking of a much shorter article by the same nameExpand
Facing disaster: the great challenges framework
PurposeThis paper provides a detailed survey of the greatest dangers facing humanity this century. It argues that there are three broad classes of risks – the “Great Challenges” – that deserve ourExpand
Arguing About War
Michael Walzer is one of the world's most eminent philosophers on the subject of war and ethics. Now, for the first time since his classic Just and Unjust Wars was published almost three decades ago,Expand
The Shield of Achilles: War, Peace, and the Course of History
Part 1 State of war: introduction - law, strategy and history the Long War of the nation-state - Thucydides and the Epochal war, the struggle begun - Fascism, Communism, Parliamentarianism,Expand
Who would destroy the world? Omnicidal agents and related phenomena
Abstract The capacity for small groups and even single individuals to wreak unprecedented havoc on civilization is growing as a result of dual-use emerging technologies. This means that scholarsExpand
Agential risks and information hazards: An unavoidable but dangerous topic?
Abstract This paper proposes an original theoretical framework for understanding anthropogenic existential risks associated with advanced dual-use technologies. It outlines a typology of “agentialExpand
Moral Bioenhancement and Agential Risks: Good and Bad Outcomes
It is argued that moral bioenhancements could nontrivially exacerbate the threat posed by certain kinds of malicious agents, while reducing the threat of other kinds. Expand
Existential risks: analyzing human extinction scenarios and related hazards
Because of accelerating technological progress, humankind may be rapidly approaching a critical phase in its career. In addition to well-known threats such as nuclear holocaust, the propects ofExpand