• Corpus ID: 14329044

Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logic

@article{Brsz2014RobustCI,
  title={Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logic},
  author={Mih{\'a}ly B{\'a}r{\'a}sz and Paul Francis Christiano and Benja Fallenstein and Marcello Herreshoff and Patrick LaVictoire and Eliezer Yudkowsky},
  journal={ArXiv},
  year={2014},
  volume={abs/1401.5577}
}
We consider the one-shot Prisoner's Dilemma between algorithms with read-access to one anothers' source codes, and we use the modal logic of provability to build agents that can achieve mutual cooperation in a manner that is robust, in that cooperation does not require exact equality of the agents' source code, and unexploitable, meaning that such an agent never cooperates when its opponent defects. We construct a general framework for such "modal agents", and study their properties. 
View PDF on arXiv
20 Citations
Highly Influential Citations
2
Background Citations
10

20 Citations

Robust program equilibrium

It is argued that this program is similar to the tit for tat strategy for the iterated prisoner’s dilemma and generalizes this approach of turning strategies for the repeated version of a game into programs for the one-shot version of an game to other two-player games and proves that the resulting programs inherit properties of the underlying strategy.

Parametric Bounded Löb's Theorem and Robust Cooperation of Bounded Agents

This paper introduces an effective version of Lob's theorem which is applicable given such bounded resources and has powerful implications for the game theory of bounded agents who are able to write proofs about themselves and one another.

Cooperative and uncooperative institution designs: Surprises and problems in open-source game theory

This work demonstrates a series of counterintuitive results on open-source games, which are independent of the programming language in which agents are written, and argues that contemporary game theory remains ill-equipped to study program equilibria.

Game-Theoretic Models of Moral and Other-Regarding Agents (extended abstract)

This work investigates Kantian equilibria in finite normal form games, a class of non-Nashian, morally motivated courses of action that was recently proposed in the economics literature, and proposes some general, intuitive, computationally tractable, otherregarding equilibrium that interpolates between purely self-regarding and Kantian behavior.

Game-theoretic Models of Moral and Other-Regarding Agents

This work investigates Kantian equilibria in finite normal form games, a class of non-Nashian, morally motivated courses of action that was recently proposed in the economics literature, and proposes some general, intuitive, computationally tractable, other-regardingEquilibria related to Kantian Equilibria.

Cooperative and Competitive Reasoning: From Games to Revolutions

I develop a game theoretic model where players use two different reasoning processes in strategic situations: cooperative and competitive. Players always consider cooperating at first: if they…

A theory of bounded inductive rationality

The dominant theories of rational decision making assume what we will call logical omniscience. That is, they assume that when facing a decision problem, an agent can perform all relevant…

Safe Pareto Improvements for Delegated Game Playing

It is proved that the notion of safe Pareto improvements is closely related to a notion of outcome correspondence between games and is also shown that under some specific assumptions about how the representatives play games, finding safe Paringo improvements is NP-complete.

Open Problems in Cooperative AI

This research integrates ongoing work on multi-agent systems, game theory and social choice, human-machine interaction and alignment, natural-language processing, and the construction of social tools and platforms into Cooperative AI, which is an independent bet about the productivity of specific kinds of conversations that involve these and other areas.

Problem Class Dominance in Predictive Dilemmas

One decision procedure dominates a given one if it performs well on the entire class of problems the given decision procedure performs well on, and then goes on to perform well on other problems that…

References

SHOWING 1-10 OF 26 REFERENCES

Program equilibrium—a program reasoning approach

This paper investigates an approach to program equilibrium in which richer conditions are allowed, based on model checking—one of the most successful approaches to reasoning about programs, and introduces a decision-tree model of strategies, which may be conditioned on strategies of others.

Program equilibrium

A commitment folk theorem

Modeling rational players I

This paper focuses on the non-cooperative analysis of games with some dynamic structure in which the choice of one move or another during the play of the game may convey valuable information to the other players.

Modeling Rational Players: Part I

Game theory has proved a useful tool in the study of simple economic models. However, numerous foundational issues remain unresolved. The situation is particularly confusing in respect of the…

Rationality, Computability, and Nash Equilibrium

Suppose two agents play a game, each using a computable algorithm to decide what to do, these algorithms being common knowledge. The author shows that it is possible to act rationally provided he…

The evolution of cooperation.

A model is developed based on the concept of an evolutionarily stable strategy in the context of the Prisoner's Dilemma game to show how cooperation based on reciprocity can get started in an asocial world, can thrive while interacting with a wide range of other strategies, and can resist invasion once fully established.

Some notes on Church's thesis and the theory of games

This paper considers games in normal form played by Turing Machines. The machines are fed as input all the relevent information and then are required to play the game. Some ‘impossibility’ results…

Strong mediated equilibrium

Definable and Contractible Contracts

This paper analyzes Bayesian normal form games in which players write contracts that condition their actions on the contracts of the other players. These contracts are required to be representable in…