Towards Understanding On-Policy Distillation through the Lens of Test-Time Scaling
On-policy distillation (OPD) has emerged as a promising post-training technique for enhancing LLM reasoning. Under the reverse KL objective, the idealized optimum of OPD aligns the student distribution with that of the teacher. When the teacher consistently outperforms the student, this naturally suggests that OPD should yield broad improvements over the pre-OPD student. However, do such improvements extend across the entire range of test-time sampling budgets? In this work, we revisit this expectation through the lens of test-time scaling by varying the sampling budget $K$ and evaluating performance with pass@$K$. Across multiple settings, we observe two distinct patterns: OPD can improv...
| Authors | Xinmu Ge, Zizhuo Zhang, Yu Huang et al. |
|---|---|
| Primary category | cs.LG |
| Published | 12 August 2026 |
| Last updated at source | 28 September 2026 |
Connected in the knowledge graph
- Geraldxm/Code-Qwen3-1.7B-Qwen3-4B-Non-Thinking-RL-Code-Step300Model · cited by
- Geraldxm/Code-Qwen3-1.7B-Qwen3-8BModel · cited by
- Geraldxm/Fact-Qwen3-1.7B-Qwen3-8B-GRPO-Binary-RARModel · cited by
- Geraldxm/Math-Qwen3-1.7B-Qwen3-4B-Non-Thinking-RL-Math-Step500Model · cited by
- Geraldxm/Math-DeepSeek-R1-Distill-Qwen-1.5B-JustRL-DeepSeek-1.5BModel · cited by
- Geraldxm/Math-DeepSeek-R1-Distill-Qwen-1.5B-Skywork-OR1-Math-7BModel · cited by
- Geraldxm/Fact-Qwen3-1.7B-SESA-8B-searchModel · cited by
Models and datasets referencing this paper
- Geraldxm/Math-DeepSeek-R1-Distill-Qwen-1.5B-Skywork-OR1-Math-7BModel
- Geraldxm/Math-DeepSeek-R1-Distill-Qwen-1.5B-JustRL-DeepSeek-1.5BModel
- Geraldxm/Code-Qwen3-1.7B-Qwen3-8BModel
- Geraldxm/Code-Qwen3-1.7B-Qwen3-4B-Non-Thinking-RL-Code-Step300Model
- Geraldxm/Math-Qwen3-1.7B-Qwen3-4B-Non-Thinking-RL-Math-Step500Model
- Geraldxm/Fact-Qwen3-1.7B-Qwen3-8B-GRPO-Binary-RARModel
- Geraldxm/Fact-Qwen3-1.7B-SESA-8B-searchModel
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.