Open Multi-rack Scale-up Domains

AI model architectures continue to evolve, and sizes continue to grow as frontier labs compete to train and serve more intelligent models. Increasing total parameters and more active experts create intensive all-to-all traffic, with less predictable traffic patterns, at a scale that continues to grow. Additionally, these frontier models are increasingly being trained and served across heterogeneous compute.

The objectives of this workstream are twofold:
1. To offer these future generation models scale-up domains that are larger than today’s single-rack scale-up domains while maintaining the high performance of a single tier of low-latency switching.
2. To break apart proprietary integrated rack architectures in pursuit of open system architectures with disaggregated components that more easily accommodate heterogeneous compute and the choice of infrastructure elements.

In pursuit of this, the workstream intends to develop a scale-up reference architecture that aligns to four principles:
• Multi-rack: grow the scale-up domain size to hundreds/ thousands of xPUs (accelerators), while maintaining high-bandwidth, single-hop connectivity.
• Disaggregated: separating compute racks and networking racks and providing the flexibility to connect other elements.
• Heterogeneous: GPUs, CPUs, xPUs and potentially other AI infrastructure elements for varying training, inference, and agentic workloads.
• Open: leverage open, standards-based building blocks including Ethernet scale-up networking (ESUN), open racks (e.g. ORv3), and optical MSAs (e.g. OCI).