Routing Policy Optimization for Online Diffusion Reinforcement Learning
Reinforcement learning can align visual generators with reward signals beyond their pretraining objectives, but existing score-function methods typically introduce stochasticity by perturbing the reverse denoising trajectory, leading to costly rollout-based optimization. We instead ask whether stochastic exploration can be introduced inside the generator while preserving its original sampler. Routing Policy Optimization (RPO) answers this question by turning attention routing into a stochastic exploration action. At a few selected denoising levels, each query block keeps its own block and samples further key blocks in proportion to the generator's native attention mass, or keeps dense att...
| Authors | Qiuhong Shen, Yiding Yang, Yuxin Zhang et al. |
|---|---|
| Primary category | cs.CV |
| Published | 15 October 2026 |
| Last updated at source | 15 October 2026 |
Recent changes
- 16 Oct 2026 Description: Reinforcement learning can align visual generators with reward signals beyond their pretraining objectives, but existing score-function methods typically introduce stochasticity by perturbing the reverse denoising trajectory, leading to cos → Reinforcement learning can align visual generators with reward signals beyond their pretraining objectives, but existing score-function methods typically introduce stochasticity by perturbing the reverse denoising trajectory, leading to cos
- 16 Oct 2026 Topics added cs.CV removed hf-daily-papers, reinforcement learning, score-function methods, reverse denoising trajectory, routing policy optimization, attention routing, stochastic exploration, denoising levels, query block
Changes detected by Silytics between refreshes of the source.
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 16 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.