SignalTech/AI
When One GPU Isn’t Enough: Serving GLM-5.3 Across Two NVIDIA DGX Stations with vLLM
High-priority alert based on source significance and keyword relevance.
Context
What happened: A tutorial shows how to serve the NVIDIA GLM-5.3 NVFP4 model across two NVIDIA DGX Stations with vLLM, using tensor parallelism (TP=2) over ConnectX-8 RDMA and a single OpenAI-compatible endpoint on Station A.
Key numbers: 753 billion parameters, 40 billion activated per token; checkpoint about 464 GB versus 252 GB of HBM3e on a single GB300 GPU; at least 464 GB free disk
Why it matters: The checkpoint exceeds single-GPU memory, so the setup keeps all weights in GPU memory with CPU weight offload disabled, and the post provides a baseline for behavior under concurrent load.
What changed
A tutorial shows how to serve the NVIDIA GLM-5.3 NVFP4 model across two NVIDIA DGX Stations with vLLM, using tensor parallelism (TP=2) over ConnectX-8 RDMA and a single OpenAI-compatible endpoint on Station A.
Why it matters
The checkpoint exceeds single-GPU memory, so the setup keeps all weights in GPU memory with CPU weight offload disabled, and the post provides a baseline for behavior under concurrent load.