StockDuty ← Dashboard Nov 17, 2026 07:27 PM ET
SignalTech/AI

Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium

Open-source LLM inference engine targets AWS Trainium hardware.

What happened: RunAnywhere open-sourced TrnRT, an inference engine for AWS Trainium, and says it can serve GLM 5.3 flash (FP8) on AWS trn2.48xlarge. Key numbers: 3.57M TPM on SemiAnalysis AgentX benchmarks; 28.9 RPM with 32 agents at batch 12; P50 TTFT about 1.24 seconds at 16 agents with 35.9 tps decode; megakernel made Why it matters: The team says AWS's own stack did not support the model, citing vLLM Neuron not registering any GLM model and NxDI no longer under active development. They cite lower memory cost per TB on trn2.48xlarge versus 8x H200 on AWS as the reason to experiment.
RunAnywhere open-sourced TrnRT, an inference engine for AWS Trainium, and says it can serve GLM 5.3 flash (FP8) on AWS trn2.48xlarge.
The team says AWS's own stack did not support the model, citing vLLM Neuron not registering any GLM model and NxDI no longer under active development. They cite lower memory cost per TB on trn2.48xlarge versus 8x H200 on AWS as the reason to experiment.
Sources