Corpus ID: 14329044

Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logic

  title={Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logic},
  author={M. B{\'a}r{\'a}sz and Paul Francis Christiano and Benja Fallenstein and Marcello Herreshoff and Patrick LaVictoire and Eliezer Yudkowsky},
  • M. Bárász, Paul Francis Christiano, +3 authors Eliezer Yudkowsky
  • Published 2014
  • Mathematics, Computer Science
  • ArXiv
  • We consider the one-shot Prisoner's Dilemma between algorithms with read-access to one anothers' source codes, and we use the modal logic of provability to build agents that can achieve mutual cooperation in a manner that is robust, in that cooperation does not require exact equality of the agents' source code, and unexploitable, meaning that such an agent never cooperates when its opponent defects. We construct a general framework for such "modal agents", and study their properties. 
    16 Citations

    Topics from this paper

    Robust program equilibrium
    • 6
    • PDF
    Parametric Bounded Löb's Theorem and Robust Cooperation of Bounded Agents
    • 5
    • PDF
    Game-theoretic Models of Moral and Other-Regarding Agents
    • PDF
    Cooperative and Competitive Reasoning: From Games to Revolutions
    • Highly Influenced
    Tiling Agents for Self-Modifying AI , and the Löbian Obstacle
    • 25
    • PDF
    Open Problems in Cooperative AI
    • PDF
    Problem Class Dominance in Predictive Dilemmas
    • 10
    • PDF
    Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda
    • 29
    • Highly Influenced
    • PDF
    Scalable agent alignment via reward modeling: a research direction
    • 58
    • PDF


    Program equilibrium—a program reasoning approach
    • 6
    • PDF
    Program equilibrium
    • 67
    • PDF
    A commitment folk theorem
    • 71
    • PDF
    Modeling rational players I
    • 261
    • Highly Influential
    Modeling Rational Players: Part I
    • 141
    Modeling Rational Players: Part II
    • 307
    Rationality, Computability, and Nash Equilibrium
    • 46
    The evolution of cooperation.
    • 9,215
    • PDF
    Strong mediated equilibrium
    • 83
    • PDF