SignalTech/AI
Evolving Infra: Agent-Adaptive LLM Serving at Scale
High-priority alert based on source significance and keyword relevance.
Context
What happened: Researchers present Evolving Infra, an agent-adaptive LLM serving framework that decouples compute allocation from KV cache management, letting nodes switch between prefill and decode at runtime.
Key numbers: Tested on up to 256 GB300 GPUs with GLM-5.2 and Kimi K2.6 across coding, math, and computer-use tasks.
Why it matters: The authors state that existing prefill-decode disaggregated systems rely on coarse signals, causing load imbalance between prefill and decode nodes and eviction of KV caches agents will reuse.
What changed
Researchers present Evolving Infra, an agent-adaptive LLM serving framework that decouples compute allocation from KV cache management, letting nodes switch between prefill and decode at runtime.
Why it matters
The authors state that existing prefill-decode disaggregated systems rely on coarse signals, causing load imbalance between prefill and decode nodes and eviction of KV caches agents will reuse.