DeepSeek just open-sourced its inference engine

What happened: DeepSeek open-sourced DeepInfer, the inference framework behind its large-scale production services, offering offline inference with the core features needed for serving at scale. Key numbers: Supports DeepSeek-V4.1-Flash; default max draft length Why it matters: The release exposes production features such as parallelism, asynchronous scheduling, batch-invariant inference, and deterministic sampling, and runs on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. A complete inference service still needs a KV cache system, disaggregation, and load balancing, which are not included.

DeepSeek open-sourced DeepInfer, the inference framework behind its large-scale production services, offering offline inference with the core features needed for serving at scale.

The release exposes production features such as parallelism, asynchronous scheduling, batch-invariant inference, and deterministic sampling, and runs on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. A complete inference service still needs a KV cache system, disaggregation, and load balancing, which are not included.