SignalTech/AI
AMLworld-Compact: Importance-Weighted Downsampling for Cost-Effective LLM Evaluation and Error Diagnosis
High-priority alert based on source significance and keyword relevance.
Context
What happened: Researchers released AMLworld-Compact, an evaluation set using importance-weighted downsampling to cut LLM evaluation targets on AMLworld graph data, with code and audit tools published on Hugging Face and GitHub.
Key numbers: HT-Coreset reduces evaluation targets by 271x on HI-Small and 611x on LI-Small. F1 RMSE is 0.00-2.58 percentage points versus 4.44-7.41 for uniform sampling.
Why it matters: The method preserves unbiased full-test metric estimates while lowering evaluation cost, and the audit shows lexical markers do not establish correctness.
What changed
Researchers released AMLworld-Compact, an evaluation set using importance-weighted downsampling to cut LLM evaluation targets on AMLworld graph data, with code and audit tools published on Hugging Face and GitHub.
Why it matters
The method preserves unbiased full-test metric estimates while lowering evaluation cost, and the audit shows lexical markers do not establish correctness.