Corpus ID: 14329044

Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logic

@article{Brsz2014RobustCI,
  title={Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logic},
  author={Mih{\'a}ly B{\'a}r{\'a}sz and Paul Francis Christiano and Benja Fallenstein and Marcello Herreshoff and Patrick LaVictoire and Eliezer Yudkowsky},
  journal={ArXiv},
  year={2014},
  volume={abs/1401.5577}
}
We consider the one-shot Prisoner's Dilemma between algorithms with read-access to one anothers' source codes, and we use the modal logic of provability to build agents that can achieve mutual cooperation in a manner that is robust, in that cooperation does not require exact equality of the agents' source code, and unexploitable, meaning that such an agent never cooperates when its opponent defects. We construct a general framework for such "modal agents", and study their properties. 
View PDF on arXiv
18 Citations
Highly Influential Citations
2
Background Citations
10

18 Citations

Citation Type

Citation Type

Robust program equilibrium
TLDR
It is argued that this program is similar to the tit for tat strategy for the iterated prisoner’s dilemma and generalizes this approach of turning strategies for the repeated version of a game into programs for the one-shot version of an game to other two-player games and proves that the resulting programs inherit properties of the underlying strategy. Expand
Parametric Bounded Löb's Theorem and Robust Cooperation of Bounded Agents
TLDR
This paper introduces an effective version of Lob's theorem which is applicable given such bounded resources and has powerful implications for the game theory of bounded agents who are able to write proofs about themselves and one another. Expand
Game-Theoretic Models of Moral and Other-Regarding Agents (extended abstract)
TLDR
This work investigates Kantian equilibria in finite normal form games, a class of non-Nashian, morally motivated courses of action that was recently proposed in the economics literature, and proposes some general, intuitive, computationally tractable, otherregarding equilibrium that interpolates between purely self-regarding and Kantian behavior. Expand
Cooperative and Competitive Reasoning: From Games to Revolutions
I develop a game theoretic model where players use two different reasoning processes in strategic situations: cooperative and competitive. Players always consider cooperating at first: if they… Expand
Tiling Agents for Self-Modifying AI , and the Löbian Obstacle *
We model self-modification in AI by introducing “tiling” agents whose decision systems will approve the construction of highly similar agents, creating a repeating pattern (including similarity of… Expand
Safe Pareto Improvements for Delegated Game Playing
TLDR
It is proved that the notion of safe Pareto improvements is closely related to a notion of outcome correspondence between games and is also shown that under some specific assumptions about how the representatives play games, finding safe Paringo improvements is NP-complete. Expand
Open Problems in Cooperative AI
TLDR
This research integrates ongoing work on multi-agent systems, game theory and social choice, human-machine interaction and alignment, natural-language processing, and the construction of social tools and platforms into Cooperative AI, which is an independent bet about the productivity of specific kinds of conversations that involve these and other areas. Expand
Problem Class Dominance in Predictive Dilemmas
One decision procedure dominates a given one if it performs well on the entire class of problems the given decision procedure performs well on, and then goes on to perform well on other problems that… Expand
A NOTE ON IDENTIFYING VULNERABLE MORAL PROPENSITIES
There are a variety of processes that steer the future; that is, they move it toward certain states and away from others dynamically, with changing behaviors in response to changing conditions. Our… Expand
Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda
In this chapter, we discuss a host of technical problems that we think AI scientists could work on to ensure that the creation of smarter-than-human machine intelligence has a positive impact.… Expand
...
1
2
...

References

SHOWING 1-10 OF 26 REFERENCES
Program equilibrium—a program reasoning approach
TLDR
This paper investigates an approach to program equilibrium in which richer conditions are allowed, based on model checking—one of the most successful approaches to reasoning about programs, and introduces a decision-tree model of strategies, which may be conditioned on strategies of others. Expand
Program equilibrium
TLDR
It is shown that in a program equilibrium of the one-shot prisoners dilemma mutual cooperation is obtained, and more generally, the set of program equilibrium payoffs of a game coincides with theSet of feasible and individually rational payoff of it. Expand
A commitment folk theorem
TLDR
This paper proposes a model of conditional commitments that unifies earlier models while avoiding circularities that often arise in such models. Expand
Modeling rational players I
TLDR
This paper focuses on the non-cooperative analysis of games with some dynamic structure in which the choice of one move or another during the play of the game may convey valuable information to the other players. Expand
Modeling Rational Players: Part I
Game theory has proved a useful tool in the study of simple economic models. However, numerous foundational issues remain unresolved. The situation is particularly confusing in respect of the… Expand
Rationality, Computability, and Nash Equilibrium
Suppose two agents play a game, each using a computable algorithm to decide what to do, these algorithms being common knowledge. The author shows that it is possible to act rationally provided he… Expand
The evolution of cooperation.
TLDR
A model is developed based on the concept of an evolutionarily stable strategy in the context of the Prisoner's Dilemma game to show how cooperation based on reciprocity can get started in an asocial world, can thrive while interacting with a wide range of other strategies, and can resist invasion once fully established. Expand
Some notes on Church's thesis and the theory of games
This paper considers games in normal form played by Turing Machines. The machines are fed as input all the relevent information and then are required to play the game. Some ‘impossibility’ results… Expand
Strong mediated equilibrium
TLDR
It is shown that desired behaviors, which are stable against deviations by coalitions, can be obtained using mediators in a rich class of settings. Expand
Definable and Contractible Contracts
This paper analyzes Bayesian normal form games in which players write contracts that condition their actions on the contracts of the other players. These contracts are required to be representable in… Expand
...
1
2
3
...