KV Cache compressed into a constant! New open-source OCR king: no page limit, cache cut by 96%

What happened: Baidu released and open-sourced Wenxin Unlimited OCR 2, which processes long documents continuously without a page limit. Key numbers: Peak KV cache fell from 47GB to 1.77GB, about 96 percent less. In a 16-way concurrency test, throughput rose from 360 to 1315 tokens per second, about 3.65x. An 884-page test took about 42 minutes 56 seconds, roughly 2.9 seconds per page. Why it matters: The content states SR-SWA single-step decoding reached 7.0x Full Attention and 5.6x the prior R-SWA, and that lower memory lets the same GPU handle more work for batch document processing.

Open-source OCR cuts KV cache 96% with no page limit, signaling major efficiency gains.

Open-source OCR cuts KV cache 96% with no page limit, signaling major efficiency gains. — impact context not separately stored; see What changed.