The NVIDIA GB200 NVL72 functions as an exascale computer within a single rack. Maximizing the utilization of its NVLink fabrics through proper scheduling allows AI training jobs to significantly leverage the substantial networking bandwidth this system offers. Notably, recent MLPerf training results (>2.6x improvement with recent MLPerf training) confirm that GB200 NVL72 provides substantial performance gains across all AI workloads. This presentation will cover key topics related to operating and optimizing large GB200 clusters, including:
- Topology-aware scheduling within Slurm.
- Recommendations for optimal GPU occupancy and scheduling (slurm simulator).
- Our experience bringing up and running a large GB200 cluster. (Healtch checks, Supporting large scale training experience, scheduling, rack testing, slow nodes etc)
