Impactful scheduling for GPU clusters

What happened: Ai2's AI Infrastructure team replaced a priority-based GPU scheduler with a system using GPU time budgets, hierarchical fair-share allocation, and a time-slicing contract, shifting GPU time debates into an administrative budgeting process. Key numbers: Thousands of NVIDIA H100, B200, and B300 GPUs; clusters of 88 to 1024 GPUs; Why it matters: The old scheduler led to GPU squatting, priority inflation where 100% of workloads used HIGH priority, and engineer time spent negotiating shutdowns. The new model lets leadership allocate GPU time like investors before workloads exist.

Ai2's AI Infrastructure team replaced a priority-based GPU scheduler with a system using GPU time budgets, hierarchical fair-share allocation, and a time-slicing contract, shifting GPU time debates into an administrative budgeting process.

The old scheduler led to GPU squatting, priority inflation where 100% of workloads used HIGH priority, and engineer time spent negotiating shutdowns. The new model lets leadership allocate GPU time like investors before workloads exist.