SetGRPO: Calibrating Grader-Induced Policy Optimization
Rubric-based reinforcement learning uses fine-grained grader feedback to train large language models on open-ended tasks, but grader randomness can alter GRPO updates in ways that score discrepancies alone cannot characterize. Because GRPO jointly normalizes rewards within each response group and combines the resulting training coefficients with response-level gradients, a scoring discrepancy can shift the coefficients of the entire group and induce policy-dependent update changes. Selective execution further affects the direction-reversal risk of retained updates. To address this, we propose Set-Valued Group Relative Policy Optimization (SetGRPO). It comprises three modules: (a) Joint Re...
| Authors | Jiale Huang, Zixu Li, Zhiwei Chen et al. |
|---|---|
| Primary category | cs.LG |
| Published | 27 October 2026 |
| Last updated at source | 27 October 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 28 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.