StockDuty ← Dashboard Oct 30, 2026 03:02 PM ET
SignalTech/AI

3 Ways to Boost MoE Inference on NVIDIA DGX Spark

High-priority alert based on source significance and keyword relevance.

What happened: A technical post examines three optimizations for MoE inference on NVIDIA DGX Spark, using Ling-3.0-flash from Inclusion AI as the implementation example: Humming MXFP4 MoE kernels, online FP8 quantization of the LM head, Why it matters: Profiling showed decoding was memory-bound, so reducing weight traffic and generating more tokens per verification round drove the gains. The post also notes that combining FP8 LM-head quantization with MTP requires the draft model to share the entire LM-head module, otherwise accept length drops to 1.00 and throughput falls below baseline.
A technical post examines three optimizations for MoE inference on NVIDIA DGX Spark, using Ling-3.0-flash from Inclusion AI as the implementation example: Humming MXFP4 MoE kernels, online FP8 quantization of the LM head,
Profiling showed decoding was memory-bound, so reducing weight traffic and generating more tokens per verification round drove the gains. The post also notes that combining FP8 LM-head quantization with MTP requires the draft model to share the entire LM-head module, otherwise accept length drops to 1.00 and throughput falls below baseline.
Sources