AI inference is running into a memory bottleneck. Huawei wants to tackle it with OceanStor M900

What happened: Huawei introduced OceanStor M900 Context Memory Storage at HUAWEI CONNECT 2026 in Shanghai, extending KV cache beyond accelerator and system memory into a shared, multi-tier storage pool for AI inference. Key numbers: Claimed capacity up to 64PB; access latency reduced to 60 microseconds; potentially doubling token throughput Why it matters: Huawei positions memory, networking and compute as interconnected elements of inference performance rather than treating storage as a separate supporting component, arguing this could reduce replacement and maintenance costs over time.

Huawei introduced OceanStor M900 Context Memory Storage at HUAWEI CONNECT 2026 in Shanghai, extending KV cache beyond accelerator and system memory into a shared, multi-tier storage pool for AI inference.

Huawei positions memory, networking and compute as interconnected elements of inference performance rather than treating storage as a separate supporting component, arguing this could reduce replacement and maintenance costs over time.