For 64 to 72 GPUs, is one GB300 NVL72 rack better than eight HGX B300 nodes?

Same GPU count, same silicon, and a completely different machine. The domain size is the whole argument, it is worth an order of magnitude on specific traffic, and it costs a failure domain nine times larger plus a facility that most halls do not have.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Foundational

🖧 Hardware & Cluster Build-Out

SXM, PCIe and Rack-Scale Form FactorsThe same silicon ships in three shapes and the shape decides the deployment. An SXM module is soldered to a baseboard with a full NVLink mesh and needs 700 to 1,400 W of direct power and usually liquid cooling. A PCIe card slots into a standard server, draws through the slot and a cable, and has no NVLink. A rack-scale system like GB300 NVL72 makes the whole rack one NVLink domain and stops being a server at all. Choosing between them fixes your power, cooling, cabling and scheduling story.

Foundational

🖧 Hardware & Cluster Build-Out

NVLink Domains and the NVL72 RackAn NVLink domain is the set of GPUs that can address each other's memory at full fabric speed, and its size is the single most consequential number in a cluster design. Eight on an HGX node, 72 on a GB300 NVL72 rack. Inside the domain a collective moves at terabytes per second over a copper backplane; outside it, the same collective drops to the scale-out fabric at 800 Gb/s per GPU, a gap of roughly twenty times that decides how models are sharded.

Advanced

🗂️ Scheduling & Orchestration🔒 Premium

Topology-Aware SchedulingTwo placements of the same 64-GPU job can differ by 2x in step time: one keeps every tensor-parallel group on a single NVSwitch node and every data-parallel ring on a single rail, the other scatters ranks across racks and pushes per-layer traffic through the spine. The scheduler is the only thing that can prevent the second placement, because the framework maps ranks to whatever GPUs it is handed. Topology-aware scheduling means the scheduler knows the hierarchy (NVLink domain, rail, rack, spine block) and places gangs to keep traffic low in it.

Advanced

🧩 GPU & Accelerator Architecture🔒 Premium

NVLink, NVSwitch and PCIeInside a node, GPUs talk over NVLink at 900 GB/s per H100 through an NVSwitch fabric that gives all eight cards full bandwidth to each other; to the host and to anything outside the node they talk over PCIe at 64 GB/s or a 400 Gb/s NIC at 50 GB/s. That fifteen-fold gap is why tensor parallelism stays inside the eight-GPU domain, why NVL72 changes the serving math for MoE, and why the question "how many GPUs share an NVLink domain?" is the first thing to ask about any cluster.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackBuy hardware, colocate, or rent from a GPU cloud? Work the decision for a 512-GPU need.Next in this trackYou have exactly 1 MW of facility capacity. Maximize training throughput.Next in this trackWhat does nvidia-fabricmanager do, and what exactly breaks when it is not running?

FEDITOR'S NOTE

Scored on the NVLink domain difference quantified, on which workloads actually use it, and on the failure domain and facility costs that come with it.

DISCUSSION · 0

No comments yet — be the first to share your approach.