SignalTech/AI
More Methods, Better Agents? A Reality Check on Self-Distillation in Agentic Reinforcement Learning
High-priority alert based on source significance and keyword relevance.
Context
What happened: Researchers benchmarked ten self-distillation methods against GRPO on ALFWorld, ScienceWorld, Search-QA, and WebShop across three model sizes, covering 132 training runs.
Key numbers: 12,124.4 recorded H800 GPU-hours; eight rollout seeds; ten methods plus GRPO.
Why it matters: No method consistently beat GRPO, strong final scores could mask unstable training, and extra cost did not reliably improve performance. The authors offer the benchmark to guide more stable, cost-effective agent training.
What changed
Researchers benchmarked ten self-distillation methods against GRPO on ALFWorld, ScienceWorld, Search-QA, and WebShop across three model sizes, covering 132 training runs.
Why it matters
No method consistently beat GRPO, strong final scores could mask unstable training, and extra cost did not reliably improve performance. The authors offer the benchmark to guide more stable, cost-effective agent training.