SignalTech/AI
A Princeton team says its Recurrent Looped Transformer grows the compute path with sequence length at a fixed per-token cost, attacking the fixed-depth limit that gates long-context work. Trained on 4
A Princeton team says its Recurrent Looped Transformer grows the compute path with sequence length at a fixed per-token cost, attacking the fixed-depth limit that gates long-context work. Trained on 4
Context
What happened: Researchers introduced the Recurrent Looped Transformer (RLT), which splits its layers between a parallel causal encoder and a recurrent decoder that merges encoder output with the previous token's final decoder state.
Key numbers: Six algorithmic tasks, five splits of eight layers, three seeds.
Why it matters: The results indicate that feedback and per-token recurrence drive the gains, and that chunking trades permutation tracking accuracy for parallelism.
What changed
Researchers introduced the Recurrent Looped Transformer (RLT), which splits its layers between a parallel causal encoder and a recurrent decoder that merges encoder output with the previous token's final decoder state.
Why it matters
The results indicate that feedback and per-token recurrence drive the gains, and that chunking trades permutation tracking accuracy for parallelism.