The non-stationary stochastic multi-armed bandit problem
@article{Allesiardo2017TheNS, title={The non-stationary stochastic multi-armed bandit problem}, author={Robin Allesiardo and R. F{\'e}raud and O. Maillard}, journal={International Journal of Data Science and Analytics}, year={2017}, volume={3}, pages={267-283} }
We consider a variant of the stochastic multi-armed bandit with K arms where the rewards are not assumed to be identically distributed, but are generated by a non-stationary stochastic process. We first study the unique best arm setting when there exists one unique best arm. Second, we study the general switching best arm setting when a best arm switches at some unknown steps. For both settings, we target problem-dependent bounds, instead of the more conservative problem-free bounds. We… CONTINUE READING
Figures, Tables, and Topics from this paper
29 Citations
Adaptively Tracking the Best Bandit Arm with an Unknown Number of Distribution Changes
- Computer Science
- COLT
- 2019
- 22
- PDF
Sliding-Window Thompson Sampling for Non-Stationary Settings
- Mathematics, Computer Science
- J. Artif. Intell. Res.
- 2020
- 1
- Highly Influenced
- PDF
Distribution-dependent and Time-uniform Bounds for Piecewise i.i.d Bandits
- Computer Science, Mathematics
- ArXiv
- 2019
- 2
- PDF
Best Arm Identification for Contaminated Bandits
- Mathematics, Computer Science
- J. Mach. Learn. Res.
- 2019
- 19
- PDF
Best of both worlds: Stochastic & adversarial best-arm identification
- Computer Science
- COLT
- 2018
- 9
- Highly Influenced
- PDF
References
SHOWING 1-10 OF 21 REFERENCES
On the Complexity of Best-Arm Identification in Multi-Armed Bandit Models
- Computer Science, Mathematics
- J. Mach. Learn. Res.
- 2016
- 681
- PDF
On Upper-Confidence Bound Policies for Non-Stationary Bandit Problems
- Mathematics
- 2008
- 172
- Highly Influential
- PDF
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
- Computer Science, Mathematics
- Found. Trends Mach. Learn.
- 2012
- 1,710
- PDF
The Nonstochastic Multiarmed Bandit Problem
- Mathematics, Computer Science
- SIAM J. Comput.
- 2002
- 1,673
- Highly Influential
- PDF
EXP3 with drift detection for the switching bandit problem
- Computer Science
- 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA)
- 2015
- 27
Finite-time Analysis of the Multiarmed Bandit Problem
- Computer Science
- Machine Learning
- 2004
- 4,412
- Highly Influential
- PDF
Piecewise-stationary bandit problems with side observations
- Mathematics, Computer Science
- ICML '09
- 2009
- 53
- PDF
Explore no more: Improved high-probability regret bounds for non-stochastic bandits
- Computer Science, Mathematics
- NIPS
- 2015
- 45
- PDF
Action Elimination and Stopping Conditions for the Multi-Armed Bandit and Reinforcement Learning Problems
- Mathematics, Computer Science
- J. Mach. Learn. Res.
- 2006
- 378
- Highly Influential
- PDF