You have exactly 1 MW of facility capacity. Maximize training throughput.

A fixed power envelope turns hardware selection into an optimization with one objective and several overheads competing for the same watts. Where the 1 MW actually goes, why the answer is not simply the most efficient GPU, and the two overheads that can be cut to buy more compute.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Advanced

🧩 GPU & Accelerator Architecture🔒 Premium

Memory-Bound vs Compute-Bound KernelsEvery kernel is limited by one of two walls: how fast bytes arrive from HBM, or how fast the tensor cores can multiply. Which wall applies is decided by arithmetic intensity against the ridge point, and the two regimes need opposite fixes. Decode, LayerNorm and softmax are memory-bound; prefill GEMMs are compute-bound; the interview question is which one you are looking at and what you would do about it.

Foundational

🖧 Hardware & Cluster Build-Out

Direct-to-Chip Liquid Cooling and CDUsAbove roughly 40 kW a rack cannot be cooled by air in any practical hall, which is why every dense GPU deployment now runs liquid to the chip. A cold plate sits on each GPU, a coolant distribution unit isolates the clean rack loop from facility water, and the facility side runs warm, typically 30 to 40 degrees supply, because warm water is cheaper to make. The design numbers are flow rate and temperature rise, and both fall out of one equation that every operator should be able to do from memory.

Advanced

📐 AI Systems Design🔒 Premium

Training Cluster Design at 10k GPUsDesign a cluster for training frontier models is the prompt that tests whether a candidate can hold hardware, network, storage, scheduling and reliability in one head at once. The answer is a bill of materials with a reason for every line: how many GPUs and why, how they are grouped into pods, how the fabric connects the pods and what it costs a collective to cross one, how much storage bandwidth the checkpoints and the data loader need, how power and cooling bound the whole thing, and how the failure statistics set the spare pool and the checkpoint cadence. This page derives each line for a 10,240-GPU cluster.

Foundational

🖧 Hardware & Cluster Build-Out

Rack Power Delivery and BuswaysA GPU rack has gone from 10 kW to over 120 kW in a few generations, and the electrical design changed with it. At 132 kW on a 415 V three-phase feed a rack draws about 184 amps, which is past what a normal power strip carries, so distribution moves to overhead busway and the rack takes redundant high-current taps. On top of the steady draw sits a synchronized transient every training step, because thousands of GPUs finish a collective at the same instant, and that swing is what sizes the upstream equipment.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackWhat does nvidia-fabricmanager do, and what exactly breaks when it is not running?Next in this trackDesign the physical layer for a multi-tenant GPU cloud. What changes versus a single-tenant cluster?Next in this trackWalk me through cabling a rail-optimized cluster. What goes where, and what goes wrong?

FEDITOR'S NOTE

Scored on allocating the budget across every consumer rather than to GPUs alone, on optimizing throughput per watt rather than per GPU, and on identifying which overheads are compressible.

DISCUSSION · 0

No comments yet — be the first to share your approach.