Design the fabric for 10,000 GPUs. Two tiers or three, and what does the choice cost?

Switch radix decides it, and at 10,000 endpoints the answer sits right at the boundary for current high-radix switches. The arithmetic, what a third tier costs in switches, optics and power, and the two designs that avoid it entirely.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Advanced

🔌 Networking & Storage🔒 Premium

Rail-Optimized and Fat-Tree FabricsA GPU cluster's network is built from two ideas: a fat tree (Clos) that gives every node a path to every other node with a chosen amount of oversubscription, and rail optimization, which wires GPU i of every node to the same leaf switch so the collectives that dominate training stay one hop away. Sizing one is arithmetic on port counts, and the interview question is usually that arithmetic: how many switches, what oversubscription, and where the NVLink domain ends and the fabric begins.

Foundational

🖧 Hardware & Cluster Build-Out

Scale-Out Fabric Choice: InfiniBand XDR vs Spectrum-XOutside the NVLink domain every GPU talks over a scale-out fabric, and as of September 2026 NVIDIA sells two at the same 800 Gb/s per port: Quantum-X800 InfiniBand and Spectrum-X Ethernet. They differ in congestion handling, operational familiarity and what happens when something misbehaves rather than in headline speed. The switch radix decides how many endpoints a two-tier fabric reaches, and that single number drives the switch count, the cable count and a large part of the budget.

Foundational

🧮 Open Weights & Serving Engines

Multi-Node Serving TopologiesOnce a model needs more GPUs than one NVLink domain holds, the deployment shape becomes a real design decision. Tensor parallelism stays inside the node because it communicates twice per layer per token. Across nodes the choices are data parallelism with replicas, pipeline parallelism with a bubble, expert parallelism with an all-to-all, or disaggregation that runs prefill and decode on separate pools and ships the KV cache between them. Each has a different failure mode and a different scaling story.

Foundational

🖧 Hardware & Cluster Build-Out

NVLink Domains and the NVL72 RackAn NVLink domain is the set of GPUs that can address each other's memory at full fabric speed, and its size is the single most consequential number in a cluster design. Eight on an HGX node, 72 on a GB300 NVL72 rack. Inside the domain a collective moves at terabytes per second over a copper backplane; outside it, the same collective drops to the scale-out fabric at 800 Gb/s per GPU, a gap of roughly twenty times that decides how models are sharded.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackFor 64 to 72 GPUs, is one GB300 NVL72 rack better than eight HGX B300 nodes?Next in this trackBuy hardware, colocate, or rent from a GPU cloud? Work the decision for a 512-GPU need.Next in this trackYou have exactly 1 MW of facility capacity. Maximize training throughput.

FEDITOR'S NOTE

Scored on the radix arithmetic deciding tier count, on quantifying the third tier's cost in switches and optics, and on the alternatives that avoid it.

DISCUSSION · 0

No comments yet — be the first to share your approach.