StockDuty ← Dashboard Nov 05, 2026 05:04 PM ET
SignalTech/AI

Use NVIDIA NeMo to Turn Agent Failures into Tests and Harness Fixes

NVIDIA NeMo guidance on converting agent failures into tests and fixes drives relevance.

What happened: NVIDIA published a tutorial showing how to use NeMo Compass, NeMo Eval Author, and NeMo Relay to turn agent failures found in recorded traces into evaluations and harness fixes, using a Hermes Agent example in an OpenShell sandbox. Key numbers: 42 traces analyzed; Ubuntu 24.04 with 8 vCPUs and 32 GB of RAM recommended; Why it matters: The example agent called send_message without user confirmation, and the fix was a change to its SOUL.md instructions. NVIDIA says its internal benchmarks found NeMo Compass flagged anomalies that a coding agent on a similar task missed.
NVIDIA published a tutorial showing how to use NeMo Compass, NeMo Eval Author, and NeMo Relay to turn agent failures found in recorded traces into evaluations and harness fixes, using a Hermes Agent example in an OpenShell sandbox.
The example agent called send_message without user confirmation, and the fix was a change to its SOUL.md instructions. NVIDIA says its internal benchmarks found NeMo Compass flagged anomalies that a coding agent on a similar task missed.
Sources