Buy hardware, colocate, or rent from a GPU cloud? Work the decision for a 512-GPU need.

The break-even is a utilization number, not a price comparison, and most teams overestimate the utilization they will actually reach. The full cost of owning, the number that decides it, and the two situations where renting wins even at high utilization.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Foundational

🖧 Hardware & Cluster Build-Out

Colocation, Power Contracts and Site SelectionFor most organizations the constraint on deploying GPUs is not the GPUs. It is finding a hall that can deliver 100 kilowatts or more per rack, reject that heat with liquid, and sign a contract for the power years before the hardware exists. Colocation contracts price reserved capacity rather than consumption, cooling capability is what eliminates most sites, and the lead time on new electrical supply is measured in years while GPUs arrive in months.

Foundational

🖧 Hardware & Cluster Build-Out

The Bill of Materials for a Training ClusterA GPU cluster is not a pile of GPUs. A 512-GPU scalable unit built to NVIDIA's DGX SuperPOD B300 reference architecture needs 64 nodes, four separate networks, thousands of transceivers, storage that can absorb a checkpoint burst, a management plane, racks, power distribution and cooling equipment. Writing the list out in order is how a design becomes a purchase order, and the items people forget are the ones that hold up a deployment for weeks.

Advanced

🧮 Napkin Math & Capacity🔒 Premium

TCO: Buy vs RentWhether to buy GPUs or rent them is a utilization question dressed as a finance question. An owned H100 costs a few tens of thousands of dollars up front and a known amount per hour in power, cooling, space and operations; a rented one costs a few dollars per hour and nothing when idle. The break-even is the utilization at which the owned hourly cost, amortized over the hardware's useful life, equals the rental rate. This page builds the owned cost from parts, works the break-even, and adds the terms the simple model leaves out: depreciation risk, reserved discounts, and the price of idle capacity.

Foundational

🖧 Hardware & Cluster Build-Out

Accelerator Selection: H100 to B300 and RTX PRO 6000Three published numbers decide which accelerator suits a workload, and they are independent: memory capacity gates what fits, memory bandwidth gates decode speed, and tensor FLOPS gate prefill and training. As of September 2026 the parts NVIDIA sells for datacenters span 80 GB to 288 GB and 1.6 TB/s to 8 TB/s, and the gap between the compute number and the bandwidth number has widened every generation, which is why a part that looks four times faster on a slide is often twice as fast on a decode workload.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackYou have exactly 1 MW of facility capacity. Maximize training throughput.Next in this trackWhat does nvidia-fabricmanager do, and what exactly breaks when it is not running?Next in this trackDesign the physical layer for a multi-tenant GPU cloud. What changes versus a single-tenant cluster?

FEDITOR'S NOTE

Scored on utilization as the deciding variable, on counting the full cost of ownership rather than hardware price, and on the cases where renting wins regardless.

DISCUSSION · 0

No comments yet — be the first to share your approach.