Towards instance-dependent regret optimality in Episodic MDPs with Posterior Sampling
We study regret minimization in finite-horizon episodic Markov Decision Processes (MDPs). While minimax-optimal algorithms are known, tractable approaches to instance-dependent optimality are still lacking. Motivated by this gap, we introduce $π_0$-PSRL, a variant of PSRL that uses posterior samples to decide when exploration is needed, while following a fixed reference policy $π_0$ during exploration episodes. This decouples the test for exploration, triggered when the sampled and empirical MDPs have different optimal policies, from the choice of the policy used to gather information. The resulting design addresses a limitation of standard PSRL, where the sampled optimal policy may not b...
| Authors | Victor Boone, Cyrille Kone, Waris Radji et al. |
|---|---|
| Primary category | stat.ML |
| Published | 19 October 2026 |
| Last updated at source | 19 October 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 20 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.