Advanced

Rail-Optimized and Fat-Tree Fabrics

A GPU cluster's network is built from two ideas: a fat tree (Clos) that gives every node a path to every other node with a chosen amount of oversubscription, and rail optimization, which wires GPU i of every node to the same leaf switch so the collectives that dominate training stay one hop away. Sizing one is arithmetic on port counts, and the interview question is usually that arithmetic: how many switches, what oversubscription, and where the NVLink domain ends and the fabric begins.

Unlock the full curriculum — ₹2,000 / $25every concept + every answer · 6 months · no auto-renew

RELATED CONCEPTS

RDMA, InfiniBand and RoCEv2 NVLink, NVSwitch and PCIe Topology-Aware Communication NCCL and Collective Algorithms Training Cluster Design at 10k GPUs Congestion Control for AI Fabrics

LESSONS THAT TEACH THIS

Fleet and ReliabilityA GPU cluster has four networks and confusing them costs you a week14 min→

PRACTICE THIS IN REAL QUESTIONS

GPU & Accelerator ArchitectureDesign a non-blocking network fabric for 512 H100s. How many switches, how are they wired, and where does the NVLink domain end?→Networking, Interconnects & StorageDesign the fabric for 4,096 GPUs with 64-port switches. What does 1:1 cost, and what does 2:1 give up?→Networking, Interconnects & StorageExplain a rail-optimized fabric. What does it buy over a plain fat tree, and how many switches does one need?→Hardware, Cabling & Cluster Build-OutDesign the fabric for 10,000 GPUs. Two tiers or three, and what does the choice cost?→Hardware, Cabling & Cluster Build-OutDuring acceptance, seven rails hit expected bandwidth and one is 15 percent low. Find it.→Hardware, Cabling & Cluster Build-OutWalk me through cabling a rail-optimized cluster. What goes where, and what goes wrong?→