Corpus ID: 219955846

FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPs

@article{Agarwal2020FLAMBESC,
  title={FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPs},
  author={Alekh Agarwal and S. Kakade and A. Krishnamurthy and W. Sun},
  journal={ArXiv},
  year={2020},
  volume={abs/2006.10814}
}
In order to deal with the curse of dimensionality in reinforcement learning (RL), it is common practice to make parametric assumptions where values or policies are functions of some low dimensional feature space. This work focuses on the representation learning question: how can we learn such features? Under the assumption that the underlying (unknown) dynamics correspond to a low rank transition matrix, we show how the representation learning question is related to a particular non-linear… Expand
17 Citations
Logistic $Q$-Learning
  • 1
  • Highly Influenced
  • PDF
Model-free Representation Learning and Exploration in Low-rank MDPs
  • Highly Influenced
  • PDF
Online Sparse Reinforcement Learning
  • 1
  • Highly Influenced
  • PDF
PerSim: Data-Efficient Offline Reinforcement Learning with Heterogeneous Agents via Personalized Simulators
  • Highly Influenced
  • PDF
Provably Correct Optimization and Exploration with Non-linear Policies
  • Highly Influenced
  • PDF
Robust Policy Gradient against Strong Data Corruption
  • PDF
Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithms
  • 1
  • PDF
Bilinear Classes: A Structural Framework for Provable Generalization in RL
  • S. Du, S. Kakade, +4 authors Ruosong Wang
  • Computer Science, Mathematics
  • ArXiv
  • 2021
  • 1
  • PDF
Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature
  • 1
  • PDF
...
1
2
...

References

SHOWING 1-10 OF 65 REFERENCES
Reinforcement Leaning in Feature Space: Matrix Bandit, Kernels, and Regret Bound
  • 82
  • Highly Influential
  • PDF
Sample-Optimal Parametric Q-Learning Using Linearly Additive Features
  • 74
  • Highly Influential
  • PDF
Introduction to Nonparametric Estimation
  • A. Tsybakov
  • Computer Science, Mathematics
  • Springer series in statistics
  • 2009
  • 1,951
  • Highly Influential
Model-based RL in Contextual Decision Processes: PAC bounds and Exponential Improvements over Model-free Approaches
  • 58
  • PDF
Contextual Decision Processes with low Bellman rank are PAC-Learnable
  • 137
  • PDF
Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning?
  • 60
  • PDF
Provably efficient RL with Rich Observations via Latent State Decoding
  • 52
  • PDF
Stochastic Linear Optimization under Bandit Feedback
  • 540
  • PDF
Adaptive Low-Nonnegative-Rank Approximation for State Aggregation of Markov Chains
  • 3
  • PDF
...
1
2
3
4
5
...