TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language Models

What happened: Researchers propose TRACE, an FP4 quantization framework for reinforcement learning training of Mixture-of-Experts language models. It uses rollout-side quantization outcomes to guide training-side FP4 rounding decisions, reducing train-rollout discrepancy. Key numbers: Evaluated on four large-scale MoE language models. Up to 5.4x rollout speedup. RL performance comparable to BF16 rollout. Why it matters: The method enables joint FP4 weight/activation and FP4 KV-cache rollout, and reports stronger final FP4 performance than post-hoc FP4 quantization of BF16-trained policies.

Researchers propose TRACE, an FP4 quantization framework for reinforcement learning training of Mixture-of-Experts language models. It uses rollout-side quantization outcomes to guide training-side FP4 rounding decisions, reducing train-rollout discrepancy.

The method enables joint FP4 weight/activation and FP4 KV-cache rollout, and reports stronger final FP4 performance than post-hoc FP4 quantization of BF16-trained policies.