ErrOPD: When Does On-Policy Distillation Transfer Teacher Errors?

A distilled student can copy its teacher's mistakes along with its skills. On-policy distillation (OPD), used in recent post-training recipes, has the teacher grade every token of the student's own outputs. Which teacher errors does it pass on? We build ErrOPD, a testbed whose teachers are fine-tuned to close a JSON list one item early, an error their students almost never make. The answer depends on the KL direction: across four model families, forward-KL OPD transmits the error within the training budget (+13-23 points), while reverse-KL OPD at our reference learning rate adds under 0.05 points. A duplicated ID and, where the decision arises, two SQL errors show the same split. The reve...

<iframe src="https://silytics.si/embed/item/58f56e4e-b071-4e3a-8697-dcaa417b25f2" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.