Ensemble Routing for Prompt-Adaptive Cost-Aware LLM Inference

What happened: The paper introduces PISA, a prompt-adaptive, cost-aware single-model router, and PISA-E, which extends routing to ensembles that select composition and cardinality under a configurable cost-quality preference. Key numbers: PISA-E surpassed the strongest ensemble competitors by 2.09 and 2.87 percentage points under ID and OOD conditions. Why it matters: PISA achieved the highest macro-average cost-performance hypervolume among evaluated single-model routers in both settings, and PISA-E reached the highest macro-average ensemble task performance when prioritizing quality.

The paper introduces PISA, a prompt-adaptive, cost-aware single-model router, and PISA-E, which extends routing to ensembles that select composition and cardinality under a configurable cost-quality preference.

PISA achieved the highest macro-average cost-performance hypervolume among evaluated single-model routers in both settings, and PISA-E reached the highest macro-average ensemble task performance when prioritizing quality.