Design the out-of-band management network for a 512-GPU cluster. What connects to it?

The endpoint count is larger than the node count and that surprises everyone building their first cluster. What has to be reachable, why this network must survive when every other one is down, and the security posture it needs because it can power-cycle the entire fleet.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

more free answers with an account · no card

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Foundational

🖧 Hardware & Cluster Build-Out

The Bill of Materials for a Training ClusterA GPU cluster is not a pile of GPUs. A 512-GPU scalable unit built to NVIDIA's DGX SuperPOD B300 reference architecture needs 64 nodes, four separate networks, thousands of transceivers, storage that can absorb a checkpoint burst, a management plane, racks, power distribution and cooling equipment. Writing the list out in order is how a design becomes a purchase order, and the items people forget are the ones that hold up a deployment for weeks.

Foundational

🖧 Hardware & Cluster Build-Out

Burn-In and Acceptance TestingNew hardware fails early or it fails late, and burn-in exists to move the early failures before the cluster is handed over rather than after. A proper acceptance test runs every layer under sustained load for days, compares every node against its siblings rather than against a specification, and produces a signed number the buyer and the vendor both agree on. The comparison is the important part: identical hardware running identical work should produce identical numbers, and the outliers are the finding.

Foundational

🖧 Hardware & Cluster Build-Out

Accelerator Selection: H100 to B300 and RTX PRO 6000Three published numbers decide which accelerator suits a workload, and they are independent: memory capacity gates what fits, memory bandwidth gates decode speed, and tensor FLOPS gate prefill and training. As of September 2026 the parts NVIDIA sells for datacenters span 80 GB to 288 GB and 1.6 TB/s to 8 TB/s, and the gap between the compute number and the bandwidth number has widened every generation, which is why a part that looks four times faster on a slide is often twice as fast on a decode workload.

Foundational

🖧 Hardware & Cluster Build-Out

SXM, PCIe and Rack-Scale Form FactorsThe same silicon ships in three shapes and the shape decides the deployment. An SXM module is soldered to a baseboard with a full NVLink mesh and needs 700 to 1,400 W of direct power and usually liquid cooling. A PCIe card slots into a standard server, draws through the slot and a cable, and has no NVLink. A rack-scale system like GB300 NVL72 makes the whole rack one NVLink domain and stops being a server at all. Choosing between them fixes your power, cooling, cabling and scheduling story.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackAir, rear-door heat exchanger or direct-to-chip? Decide for a 60 kW rack.Next in this trackBuild the schedule for a 2,048-GPU deployment. What is on the critical path?Next in this trackHow many spare GPUs, nodes, cables and transceivers do you hold for a 2,048-GPU fleet?

FEDITOR'S NOTE

Scored on counting all endpoint classes rather than just BMCs, on the independence requirements, and on the security posture given the network's power over the hardware.

DISCUSSION · 0

No comments yet — be the first to share your approach.