Extreme State Aggregation beyond MDPs

@article{Hutter2014ExtremeSA,
  title={Extreme State Aggregation beyond MDPs},
  author={Marcus Hutter},
  journal={ArXiv},
  year={2014},
  volume={abs/1407.3341}
}
  • Marcus Hutter
  • Published 2014
  • Computer Science
  • ArXiv
  • We consider a Reinforcement Learning setup without any (esp. MDP) assumptions on the environment. State aggregation and more generally feature reinforcement learning is concerned with mapping histories/raw-states to reduced/aggregated states. The idea behind both is that the resulting reduced process (approximately) forms a small stationary finite-state MDP, which can then be efficiently solved or learnt. We considerably generalize existing aggregation results by showing that even if the… CONTINUE READING

    Topics from this paper.

    Extreme state aggregation beyond Markov decision processes
    7
    Nonparametric General Reinforcement Learning
    12
    Approximate Exploration through State Abstraction
    6
    Value Preserving State-Action Abstractions
    2
    Near Optimal Behavior via Approximate State Abstraction
    55
    Information-Theoretic Considerations in Batch Reinforcement Learning
    44

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 31 REFERENCES
    Feature Reinforcement Learning: Part I. Unstructured MDPs
    53
    Selecting the State-Representation in Reinforcement Learning
    31
    Reinforcement Learning in Finite MDPs: PAC Analysis
    228
    Near-optimal Regret Bounds for Reinforcement Learning
    629
    Feature Markov Decision Processes
    13
    PAC Bounds for Discounted MDPs
    71
    Tree-Based Batch Mode Reinforcement Learning
    770