SignalTech/AI
On-Policy Parameter Update Direction Underlies Generalization in LLM Post-Training
High-priority alert based on source significance and keyword relevance.
Context
What happened: A paper proposes OPSFT, a method that constrains supervised fine-tuning updates to the parameter update direction identified by on-policy post-training paradigms.
Why it matters: The authors report that OPSFT transfers the generalization advantage of on-policy training to SFT while keeping SFT's training efficiency and ability to use high-quality trajectories.
What changed
A paper proposes OPSFT, a method that constrains supervised fine-tuning updates to the parameter update direction identified by on-policy post-training paradigms.
Why it matters
The authors report that OPSFT transfers the generalization advantage of on-policy training to SFT while keeping SFT's training efficiency and ability to use high-quality trajectories.