Conditional Kolmogorov complexity and universal probability

@article{Vitnyi2013ConditionalKC,
  title={Conditional Kolmogorov complexity and universal probability},
  author={Paul M. B. Vit{\'a}nyi},
  journal={ArXiv},
  year={2013},
  volume={abs/1206.0983}
}
Thermodynamic costs of Turing machines
TLDR
For universal TMs, it is shown that the thermodynamic complexity of any desired output is bounded by a constant (unlike the conventional Kolmogorov complexity), while the expected amount of generated heat is infinite, and that any computable realization faces a fundamental tradeoff between heat generation and heat function.
Few-Shot Induction of Generalized Logical Concepts via Human Guidance
TLDR
This work defines a distance measure between candidate concept representations that improves the efficiency of search for target concept and generalization and leverages richer human inputs in the form of advice to improve the sample efficiency of learning.
A Consolidated Approach to Convolutional Neural Networks and the Kolmogorov Complexity
TLDR
This work will explore the innovative algorithmic normalized compression distance metric based on the information theoretic concept of Kolmogorov Complexity and observe its possible implementation in Convolutional Neural Networks to facilitate and automate the classification of Retinal Pigment Epithelial cell cultures for use in Age Related Macular Degeneration Stem Cell therapy in an unsupervised setting.
Learnable Leaky ReLU (LeLeLU): An Alternative Accuracy-Optimized Activation Function
TLDR
This paper proposes a novel activation function, combining many features of successful activation functions, achieving 2.53% higher accuracy than the industry standard ReLU in a variety of test cases.
Transfer learning for neural network model in chlorophyll-a dynamics prediction
TLDR
Transfer learning is employed to optimize neural network models to improve the accuracy of prediction and maintain high performance in long-term applications and in general, TL has a better prediction effect than PNP and dropout.

References

SHOWING 1-10 OF 40 REFERENCES
Minimum description length induction, Bayesianism, and Kolmogorov complexity
TLDR
In general, it is shown that data compression is almost always the best strategy, both in model selection and prediction.
The Definition of Random Sequences
Minimum Message Length and Kolmogorov Complexity
TLDR
This work attempts to establish a parallel between a restricted (two-part) version of the Kolmogorov model and the minimum message length approach to statistical inference and machine learning of Wallace and Boulton (1968), in which an ‘explanation’ of a data string is modelled as a two-part message.
THE COMPLEXITY OF FINITE OBJECTS AND THE DEVELOPMENT OF THE CONCEPTS OF INFORMATION AND RANDOMNESS BY MEANS OF THE THEORY OF ALGORITHMS
TLDR
The present article is a survey of the fundamental results connected with the concept of complexity as the minimum number of binary signs containing all the information about a given object that are sufficient for its recovery (decoding).
An Introduction to Kolmogorov Complexity and Its Applications
TLDR
The book presents a thorough treatment of the central ideas and their applications of Kolmogorov complexity with a wide range of illustrative applications, and will be ideal for advanced undergraduate students, graduate students, and researchers in computer science, mathematics, cognitive sciences, philosophy, artificial intelligence, statistics, and physics.
A Mathematical Theory of Communication
TLDR
It is proved that the authors can get some positive data rate that has the same small error probability and also there is an upper bound of the data rate, which means they cannot achieve the data rates with any encoding scheme that has small enough error probability over the upper bound.
Elements of Information Theory
TLDR
The author examines the role of entropy, inequality, and randomness in the design of codes and the construction of codes in the rapidly changing environment.
Foundations of the theory of probability
Theories of ProbabilityFoundations of Probabilistic Logic ProgrammingGood ThinkingStatistical Foundations of Data ScienceFoundations of Risk AnalysisFoundations of Estimation TheoryThe Foundations of
A mathematical theory of communication
In this final installment of the paper we consider the case where the signals or the messages or both are continuously variable, in contrast with the discrete nature assumed until now. To a
A device for quantizing, grouping, and coding amplitude-modulated pulses
Thesis (M.S.) Massachusetts Institute of Technology. Dept. of Electrical Engineering, 1949.
...
...