Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logic
@article{Brsz2014RobustCI, title={Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logic}, author={M. B{\'a}r{\'a}sz and Paul Francis Christiano and Benja Fallenstein and Marcello Herreshoff and Patrick LaVictoire and Eliezer Yudkowsky}, journal={ArXiv}, year={2014}, volume={abs/1401.5577} }
We consider the one-shot Prisoner's Dilemma between algorithms with read-access to one anothers' source codes, and we use the modal logic of provability to build agents that can achieve mutual cooperation in a manner that is robust, in that cooperation does not require exact equality of the agents' source code, and unexploitable, meaning that such an agent never cooperates when its opponent defects. We construct a general framework for such "modal agents", and study their properties.
Topics from this paper
Paper Mentions
16 Citations
Parametric Bounded Löb's Theorem and Robust Cooperation of Bounded Agents
- Mathematics, Computer Science
- ArXiv
- 2016
- 5
- PDF
Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda
- Political Science
- 2017
- 29
- Highly Influenced
- PDF
Scalable agent alignment via reward modeling: a research direction
- Computer Science, Mathematics
- ArXiv
- 2018
- 58
- PDF
References
SHOWING 1-10 OF 27 REFERENCES
Program equilibrium—a program reasoning approach
- Mathematics, Computer Science
- Int. J. Game Theory
- 2013
- 6
- PDF