Corpus ID: 14329044

Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logic

  title={Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logic},
  author={Mih{\'a}ly B{\'a}r{\'a}sz and Paul Francis Christiano and Benja Fallenstein and Marcello Herreshoff and Patrick LaVictoire and Eliezer Yudkowsky},
We consider the one-shot Prisoner's Dilemma between algorithms with read-access to one anothers' source codes, and we use the modal logic of provability to build agents that can achieve mutual cooperation in a manner that is robust, in that cooperation does not require exact equality of the agents' source code, and unexploitable, meaning that such an agent never cooperates when its opponent defects. We construct a general framework for such "modal agents", and study their properties. 
Robust program equilibrium
It is argued that this program is similar to the tit for tat strategy for the iterated prisoner’s dilemma and generalizes this approach of turning strategies for the repeated version of a game into programs for the one-shot version of an game to other two-player games and proves that the resulting programs inherit properties of the underlying strategy. Expand
Parametric Bounded Löb's Theorem and Robust Cooperation of Bounded Agents
This paper introduces an effective version of Lob's theorem which is applicable given such bounded resources and has powerful implications for the game theory of bounded agents who are able to write proofs about themselves and one another. Expand
Game-Theoretic Models of Moral and Other-Regarding Agents (extended abstract)
This work investigates Kantian equilibria in finite normal form games, a class of non-Nashian, morally motivated courses of action that was recently proposed in the economics literature, and proposes some general, intuitive, computationally tractable, otherregarding equilibrium that interpolates between purely self-regarding and Kantian behavior. Expand
Cooperative and Competitive Reasoning: From Games to Revolutions
I develop a game theoretic model where players use two different reasoning processes in strategic situations: cooperative and competitive. Players always consider cooperating at first: if theyExpand
Tiling Agents for Self-Modifying AI , and the Löbian Obstacle *
We model self-modification in AI by introducing “tiling” agents whose decision systems will approve the construction of highly similar agents, creating a repeating pattern (including similarity ofExpand
Safe Pareto Improvements for Delegated Game Playing
It is proved that the notion of safe Pareto improvements is closely related to a notion of outcome correspondence between games and is also shown that under some specific assumptions about how the representatives play games, finding safe Paringo improvements is NP-complete. Expand
Open Problems in Cooperative AI
This research integrates ongoing work on multi-agent systems, game theory and social choice, human-machine interaction and alignment, natural-language processing, and the construction of social tools and platforms into Cooperative AI, which is an independent bet about the productivity of specific kinds of conversations that involve these and other areas. Expand
Problem Class Dominance in Predictive Dilemmas
One decision procedure dominates a given one if it performs well on the entire class of problems the given decision procedure performs well on, and then goes on to perform well on other problems thatExpand
There are a variety of processes that steer the future; that is, they move it toward certain states and away from others dynamically, with changing behaviors in response to changing conditions. OurExpand
Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda
In this chapter, we discuss a host of technical problems that we think AI scientists could work on to ensure that the creation of smarter-than-human machine intelligence has a positive impact.Expand


Program equilibrium—a program reasoning approach
This paper investigates an approach to program equilibrium in which richer conditions are allowed, based on model checking—one of the most successful approaches to reasoning about programs, and introduces a decision-tree model of strategies, which may be conditioned on strategies of others. Expand
Program equilibrium
It is shown that in a program equilibrium of the one-shot prisoners dilemma mutual cooperation is obtained, and more generally, the set of program equilibrium payoffs of a game coincides with theSet of feasible and individually rational payoff of it. Expand
A commitment folk theorem
This paper proposes a model of conditional commitments that unifies earlier models while avoiding circularities that often arise in such models. Expand
Modeling rational players I
This paper focuses on the non-cooperative analysis of games with some dynamic structure in which the choice of one move or another during the play of the game may convey valuable information to the other players. Expand
Modeling Rational Players: Part I
Game theory has proved a useful tool in the study of simple economic models. However, numerous foundational issues remain unresolved. The situation is particularly confusing in respect of theExpand
Rationality, Computability, and Nash Equilibrium
Suppose two agents play a game, each using a computable algorithm to decide what to do, these algorithms being common knowledge. The author shows that it is possible to act rationally provided heExpand
The evolution of cooperation.
A model is developed based on the concept of an evolutionarily stable strategy in the context of the Prisoner's Dilemma game to show how cooperation based on reciprocity can get started in an asocial world, can thrive while interacting with a wide range of other strategies, and can resist invasion once fully established. Expand
Some notes on Church's thesis and the theory of games
This paper considers games in normal form played by Turing Machines. The machines are fed as input all the relevent information and then are required to play the game. Some ‘impossibility’ resultsExpand
Strong mediated equilibrium
It is shown that desired behaviors, which are stable against deviations by coalitions, can be obtained using mediators in a rich class of settings. Expand
Definable and Contractible Contracts
This paper analyzes Bayesian normal form games in which players write contracts that condition their actions on the contracts of the other players. These contracts are required to be representable inExpand