KV Cache cut by 99%, single-GPU throughput surges 6x, HKUST and CMU join hands with Yanshan Technology to open-source a new model

What happened: Yanshan Technology's Rock Matrix unit, with HKUST and CMU, released and open-sourced EdgeRock-4B, a model built from Qwen3.5-4B that keeps KV Cache fixed regardless of context length. Weights and inference code are open, with text and image support. Key numbers: KV Cache cut 99.1% versus the source model, fixed at 72 MiB at 256K context. Why it matters: The team reports throughput 4.94x Qwen3.5-4B on average across four input lengths, with the gap widening as context grows, and says capability is largely retained. More parameter sizes are said to be in progress.

Yanshan Technology's Rock Matrix unit, with HKUST and CMU, released and open-sourced EdgeRock-4B, a model built from Qwen3.5-4B that keeps KV Cache fixed regardless of context length. Weights and inference code are open, with text and image support.

The team reports throughput 4.94x Qwen3.5-4B on average across four input lengths, with the gap widening as context grows, and says capability is largely retained. More parameter sizes are said to be in progress.