SignalTech/AI
Many Right Answers, One Training Example: Removing Responder Noise from GRPO for LLM Recommenders
High-priority alert based on source significance and keyword relevance.
Context
What happened: Researchers propose ReMaRk, a reinforcement learning method for LLM recommenders that scores generated lists against the union of all valid replies instead of one reply per example, unlike Rank-GRPO.
Key numbers: Rank-GRPO's advantage variance is 85% attributable to which reply was drawn.
Why it matters: The paper reports ReMaRk is never significantly behind Rank-GRPO when both train to convergence and reaches Rank-GRPO's best with fewer GPU-hours in four of seven pairs, and the reward also helps on ProtoQA.
What changed
Researchers propose ReMaRk, a reinforcement learning method for LLM recommenders that scores generated lists against the union of all valid replies instead of one reply per example, unlike Rank-GRPO.
Why it matters
The paper reports ReMaRk is never significantly behind Rank-GRPO when both train to convergence and reaches Rank-GRPO's best with fewer GPU-hours in four of seven pairs, and the reward also helps on ProtoQA.