On-Policy Distillation Meets Off-Policy GRPO: Training Compact Instruction-Following Rerankers
Compact instruction-following rerankers are attractive for deployment, but conventional distillation pipelines typically train students by offline imitation of teacher outputs on a fixed set of examples, constraining supervision to the teacher's observed ranking space. We revisit reranker distillation through the lens of reinforcement learning. We propose a two-stage framework combining off-policy teacher optimization with on-policy student distillation. In Stage 1, a 4B teacher reranker is strengthened with off-policy GRPO using LLM-judge feedback on 88K instruction-following examples. In Stage 2, a compact 1B student samples rankings from its own policy and receives soft teacher-derived...
| Authors | Vignesh Prabhakar, Jialing Pan, Anil Babu Ankisettipalli |
|---|---|
| Primary category | cs.LG |
| Published | 1 September 2026 |
| Last updated at source | 1 September 2026 |
Connected in the knowledge graph
- everetteonhf/zerank-2-distilled-1B-GGUFModel · cited by
Models and datasets referencing this paper
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.