GAC: Noise-Aware Adaptive Mixing for Hybrid SFT-RL Post-Training
Hybrid post-training combining supervised fine-tuning (SFT) and reinforcement learning (RL) is the standard paradigm for aligning large language models, yet fixed mixing schedules cannot adapt when the relative noise of the two signals evolves. We derive a noise-aware mixing weight mu* by minimizing an MSE upper bound on the mixed stochastic gradient, yielding a closed-form that balances gradient noise variance and SFT-RL disagreement. Building on the token-wise reweighting of CHORD, the practical Guided Adaptive Controller (GAC) adds EMA smoothing, a schedule prior, and capped updates around this estimator, with all statistics estimated online from existing training tensors. The noise-aw...
| Authors | Yuelin Hu, Wei Liu, Zhenbo Yu et al. |
|---|---|
| Primary category | cs.LG |
| Published | 25 May 2026 |
| Last updated at source | 10 October 2026 |
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.