Learning Simple Concept Under Simple Distributions

@article{Li1991LearningSC,
  title={Learning Simple Concept Under Simple Distributions},
  author={Ming Li and Paul M. B. Vit{\'a}nyi},
  journal={SIAM J. Comput.},
  year={1991},
  volume={20},
  pages={911-935}
}
This paper aims at developing a learning theory where “simple” concepts are easily learnable. In Valiant’s learning model, many concepts turn out to be too hard (like NP hard) to learn. Relatively few concept classes were shown to be learnable polynomially. In daily life, it seems that things we care to learn are usually learnable. To model the intuitive notion of learning more closely, it is not required that the learning algorithm learn (polynomially) under all distributions, but only under… 

On the Relationship between Models for Learning in Helpful Environments

TLDR
This paper identifies the relationships between different learning models for polynomial learnability of concept classes and addresses the issue of unnatural collusion between the teacher and the learner that can potentially trivialize the task of learning in helpful environments.

Simple DFA are Polynomially Probably Exactly Learnable from Simple Examples

TLDR
It is demonstrated that the class of simple DFA, whose canon ical representations have logarithmic Kol mogorov complexity, is learnable under the Solomono Levin universal distribution.

Learning DFA from Simple Examples

TLDR
It is demonstrated that the class of DFA whose canonical representations have logarithmic Kolmogorov complexity is efficiently PAC learnable under the Solomonoff Levin universal distribution (m) and any concept that is learnability under Gold's model of learning from characteristic samples, Goldman and Mathias' polynomial teachability model, and the model oflearning from example based queries is also learnableunder the PACS model.

PAC Learning with Simple Examples

TLDR
The class of poly-term DNF is learnable, and the class of k-reversible languages is learningable from positive data, in this new PAC learning model, and an Occam's Razor theorem is proved.

PAC Learning under Helpful Distributions

TLDR
The class of decision lists is proved PAC learnable under helpful distributions and an Occam's razor theorem and its converse are proved.

It { 310 Pac Learning with Positive Examples Laboratoire D'informatique Fondamentale De Lille

TLDR
It is shown that k-DNF are learnable when the learner has the possibility to draw positive examples and unlabeled examples and that there exists a small sample S, computable within polynomial time, such that every k- DNF consistent with S is necessarily close to 1.

Learning Shallow Context-free Languages under Simple Distributions

TLDR
It is claimed that the EMILE approach could serve as a valuable metamodel for evaluating clustering approaches to language learning and that shallowness in itself is an interesting general constraint in the context of formal learning theory.

Simple PAC Learning of Simple Decision Lists

We prove that log n-decision lists — the class of decision lists such that all their terms have low Kolmogorov complexity— are learnable in the simple PAG learning model. The proof is based on a

Learning Dfa from Simple Examples Learning Dfa from Simple Examples

TLDR
It is shown that eecient PAC learning of DFA is possible if the class of distributions is restricted to simple distributions where a teacher might choose examples based on the knowledge of the target concept.

A Complete and Tight Average-Case Analysis of Learning Monomials

TLDR
A new learning model is studied, stochastic finite learning, in which, in contrast to PAC learning, some information about the underlying distribution is given and the goal is to find a correct (not only approximatively correct) hypothesis.
...

References

SHOWING 1-10 OF 24 REFERENCES

Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm

  • N. Littlestone
  • Computer Science
    28th Annual Symposium on Foundations of Computer Science (sfcs 1987)
  • 1987
TLDR
This work presents one such algorithm that learns disjunctive Boolean functions, along with variants for learning other classes of Boolean functions.

A theory of the learnable

TLDR
This paper regards learning as the phenomenon of knowledge acquisition in the absence of explicit programming, and gives a precise methodology for studying this phenomenon from a computational viewpoint.

Classifying learnable geometric concepts with the Vapnik-Chervonenkis dimension

TLDR
It is shown that the essential condition for distribution-free learnability is finiteness of the Vapnik-Chervonenkis dimension, a simple combinatorial parameter of the class of concepts to be learned.

Computational limitations on learning from examples

TLDR
It is shown for various classes of concept representations that these cannot be learned feasibly in a distribution-free sense unless R = NP, and relationships between learning of heuristics and finding approximate solutions to NP-hard optimization problems are given.

On the theory of average case complexity

TLDR
The present authors widen the scope to other basic questions in computational complexity to include the equivalence of search and decision problems in the context of average case complexity and an initial analysis of the structure of distributional-NP under reductions which preserve average polynomial-time.

On learning Boolean functions

TLDR
An intuitively appealing notion of dimensionality is developed and used to identify the most general class of Boolean function families that are learnable from polynomially many positive examples with one-sided error.

Inductive reasoning and Kolmogorov complexity

  • Ming LiP. Vitányi
  • Computer Science
    [1989] Proceedings. Structure in Complexity Theory Fourth Annual Conference
  • 1989
TLDR
The thesis is developed that Solomonoff's method is fundamental in the sense that many other induction principles can be viewed as particular ways to obtain computable approximations to it.

Diversity-based inference of finite automata

  • R. RivestR. Schapire
  • Computer Science
    28th Annual Symposium on Foundations of Computer Science (sfcs 1987)
  • 1987
TLDR
A new procedure for inferring the structure of a finitestate automaton (FSA) from its input/output behavior, using access to the automaton to perform experiments, based on the notion of equivalence between testa.

Kolmogorov Complexity and its Applications

  • Ming LiP. Vitányi
  • Computer Science
    Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity
  • 1990