Asymptotic Logical Uncertainty and the Benford Test

@inproceedings{Garrabrant2016AsymptoticLU,
  title={Asymptotic Logical Uncertainty and the Benford Test},
  author={Scott Garrabrant and Tsvi Benson-Tilsen and Siddharth Bhaskar and Abram Demski and Joanna Garrabrant and George Koleszarik and Evan Lloyd},
  booktitle={AGI},
  year={2016}
}
Almost all formal theories of intelligence suffer from the problem of logical omniscience, the assumption that an agent already knows all consequences of its beliefs. Logical uncertainty codifies uncertainty about the consequences of existing beliefs. This implies a departure from beliefs governed by standard probability theory. Here, we study the asymptotic properties of beliefs on quickly computable sequences of logical sentences. Motivated by an example we call the Benford test, we provide… 
Inductive Coherence
TLDR
Inductive coherence is introduced, a strengthening of coherence that provides appropriate constraints on finite approximations, and an algorithm is proposed which satisfies this criterion.
Uniform Coherence
While probability theory is normally applied to external environments, there has been some recent interest in probabilistic modeling of the outputs of computations that are too expensive to run.
Logical Induction
TLDR
A computable algorithm that assigns probabilities to every logical statement in a given formal language, and refines those probabilities over time, and follows a single logical induction criterion, motivated by a series of stock trading analogies.
Optimal Polynomial-Time Estimators: A Bayesian Notion of Approximation Algorithm
TLDR
This work introduces a new concept of approximation applicable to decision problems and functions, inspired by Bayesian probability, and proves some existence theorems and completeness results, and shows that optimal polynomial-time estimators exhibit many parallels with "classical" probability theory.
Analysis of Algorithms and Partial Algorithms
TLDR
This methodology naturally handles algorithms that do not always terminate, so it can (theoretically) be used with partial algorithms for undecidable problems, such as those found in artificial general intelligence (AGI) and automated theorem proving.
Comments on the Open Philanthropy Project ’ s Anonymized Reviews of Three Recent MIRI Papers
As part of an evaluation process for their grant decision, the Open Philanthropy Project had twelve anonymous reviewers (four technical advisers internal to the Open Philanthropy Project, plus eight

References

SHOWING 1-10 OF 42 REFERENCES
Non-Omniscience, Probabilistic Inference, and Metamathematics
TLDR
It is shown how mathematical theories can be understood as latent structure constraining physical observations, and consequently how realistic observations can provide evidence about abstract mathematical facts.
Probabilistic Logic
  • N. Nilsson
  • Philosophy, Computer Science
    Artif. Intell.
  • 1986
Questions of Reasoning Under Logical Uncertainty
A logically uncertain reasoner would be able to reason as if they know both a programming language and a program, without knowing what the program outputs. Most practical reasoning involves some
Knowledge and the Problem of Logical Omniscience
  • R. Parikh
  • Philosophy, Computer Science
    ISMIS
  • 1987
The notion of knowledge has recently acquired a great deal of importance in Computer Science, partly because of its importance in AI and expert systems, but also because of applications in
Reasoning with Limited Resources and Assigning Probabilities to Arithmetical Statements
TLDR
A rigorous way of assigning probabilities to statements in pure arithmetic and a philosophical discussion that highlights the shifting contextual character of subjective probabilities and beliefs are sketched.
Dealing with logical omniscience
We examine four approaches for dealing with the logical omniscience problem and their potential applicability: the syntactic approach, awareness, algorithmic knowledge, and impossible possible
Concerning measures in first order calculi
~0. Introduction. The idea of treating probability as a real valued function defined on sentences is an old one (see ['6] and [7], where other references can be found). Carnap's at tempt to set up a
...
...