LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Markets

What happened: Researchers introduced LiveMACE, a process-aware benchmark that evaluates persistent LLM agents in live cryptocurrency and US-equity markets using decision traces rather than outcomes alone. Key numbers: Five frontier LLMs were evaluated over 30 days of live trading across four capability configurations: tool use, persistent memory, rule following, Why it matters: The work reports an outcome-capability gap, where higher returns do not consistently imply stronger capabilities and similar returns can arise from different patterns of mechanism use. The framework is open source.

Researchers introduced LiveMACE, a process-aware benchmark that evaluates persistent LLM agents in live cryptocurrency and US-equity markets using decision traces rather than outcomes alone.

The work reports an outcome-capability gap, where higher returns do not consistently imply stronger capabilities and similar returns can arise from different patterns of mechanism use. The framework is open source.