Design the cooling for a 40-rack hall of liquid-cooled GPU racks.

TL;DR: Start from the load and the heat equation. Forty racks at 120 kW is 4.8 MW of heat, and at a 10 degree supply-to-return rise water carries it at about 115 kilograms per second, roughly 6,900 litres per minute, since one 120 kW rack needs 2.87 kg/s or 172 L/min. Distribute that across coolant distribution units sized 10 to 15 percent above the load they serve, with N+1 so that losing one does not take its racks, which for a hall this size means one CDU per two to four racks plus spares. Run facility water warm, 30 to 40 degrees supply, because dry coolers can often make it and chillers are a large fraction of facility power. Then specify the parts everyone skips: filtration to about 50 microns, conductivity monitoring, corrosion inhibitor, dripless quick disconnects and leak detection at rack and row level. Finally, automate the response, because at 1,400 watts per GPU a cooling failure reaches shutdown in tens of seconds and there is no time for a human.

How to approach it

Compute the heat load, then the flow, then the CDU count and the redundancy scheme, then the facility loop and its temperature, then water quality, then monitoring and the automatic response. Say the redundancy model explicitly rather than assuming it. Close with commissioning, because a cooling design is only real once it has been tested with the load it will carry.

A strong answer

A typical situation: a team is fitting out a hall for forty racks of Blackwell-class hardware. The building has a chilled water loop at 12 degrees from its previous life as a general-purpose facility, and the instinct is to reuse it.

The load and the flow:

heat load
  40 racks x 120 kW = 4,800 kW = 4.8 MW
  in imperial units the plant will quote: 4,800 x 3,412 = 16,377,600 BTU/hr

flow, from Q = m_dot x c_p x delta_T
  water c_p = 4,180 J per kg per K
  at a 10 K supply-to-return rise:
    m_dot = 4,800,000 / (4,180 x 10) = 114.8 kg/s
    at about 1 kg per litre: 114.8 L/s = 6,890 L/min for the hall
  per rack: 120,000 / (4,180 x 10) = 2.87 kg/s = 172 L/min

what a different delta_T costs
  delta_T = 5 K:  229.7 kg/s, 13,780 L/min, larger pipes and more pumping power
  delta_T = 20 K: 57.4 kg/s, 3,445 L/min, but the return is hot and the margin at the chip
    inlet shrinks
sanity: the chip's maximum inlet temperature bounds the choice, so delta_T is picked to keep
        the supply cool enough at the far end of the loop rather than to minimize flow

CDU sizing and redundancy:

capacity per CDU
  sized 10 to 15% above the rack load it serves
  a CDU rated for 500 kW serves 3 racks at 120 kW (360 kW) with 39% margin, or
    4 racks (480 kW) with 4% margin, which is too tight
  take 3 racks per CDU: 40 / 3 = 13.3 -> 14 CDUs to cover the hall

redundancy
  N+1 at the hall level: 15 CDUs, one spare that can pick up any group
  or N+1 within a group: pairs of CDUs each able to carry the pair's racks alone, which
    doubles unit count and gives the strongest guarantee
  the failure to design for is not a CDU dying but a CDU being serviced while the others
    run at full load, which is the moment a hall discovers it had no margin
sanity: 14 CDUs at 500 kW is 7,000 kW of cooling for 4,800 kW of load, 46% headroom before
        redundancy, which is the right shape: cooling is cheap relative to the hardware it
        protects and running it at its limit is a false economy

Direct-to-Chip Liquid Cooling and CDUs covers the two-loop separation and why it exists, and Thermal, Power and Cooling Events covers what the incident looks like when this design is wrong. The facility loop decision is the one worth arguing in this design:

reusing the existing 12 C chilled loop
  it works, and it costs chiller power continuously
  a chiller plant is commonly a large fraction of non-IT facility power

running warm at 30 to 40 C supply
  dry coolers reject to outside air whenever ambient is below the supply temperature
  chillers run only during the hottest hours, or not at all in some climates
  PUE falls, often materially, and the capital cost of the plant falls too
  the constraint is the chip's maximum inlet temperature, which is what makes 30 to 40 C
    workable rather than arbitrary
sanity: the hall's existing 12 C loop is a sunk asset and the wrong design point, so the
        useful answer is to run the new zone warm and use the chilled loop only as a
        secondary or for the parts of the hall that still need it

The requirements list that determines year three:

The automatic response, which is part of the cooling design rather than an addition:

the timeline after flow stops at 1,400 W per GPU
  seconds:        junction temperature rises; a cold plate has almost no thermal mass
  tens of seconds: GPUs throttle, which appears as a collective slowdown across the rack
  shortly after:  thermal shutdown to protect the hardware

what has to happen without a human
  detect: CDU flow and pressure, leak sensors, per-GPU temperature from DCGM
  act:    power-cap immediately with nvidia-smi or a DCGM policy, cutting heat at the source
  then:   signal the scheduler to checkpoint if the job can, then orderly shutdown
sanity: a page delivered to a person arrives after the hardware has already protected itself,
        so the automation is the design and the page is the notification

The reversal condition: for a hall in the 40 to 70 kW per rack range, rear-door heat exchangers are a legitimate alternative to direct-to-chip and are far less invasive. They put a water coil in the cabinet door and cool the air leaving the rack, which needs facility water to the row but no cold plates, no quick disconnects at the node and no coolant loop inside the server. That keeps standard air-cooled nodes serviceable by anyone and avoids the training and handling requirements of direct-to-chip. Above roughly 70 kW they run out, which is why they are a middle step rather than an answer for Blackwell-class density.

What interviewers probe next

• "Why 10 degrees rather than 5 or 20?" It balances flow against the inlet temperature margin at the far end of the loop. Say what bounds each side.
• "What happens when a CDU is serviced?" The redundancy question. N+1 at hall level is cheaper; pairs per group is stronger; say which you chose and why.
• "How do you commission it?" Run the hall at full synthetic load, measure supply and return temperatures and flow per rack, and confirm the automatic response by triggering it deliberately.
• "What is the water quality risk you worry about most?" Fouling, because it is gradual and shows as slowly rising temperatures rather than as a failure, which means it is found late.

Common mistakes

• Sizing CDUs at exactly the rack load with no margin for fouling or peak draw.
• Reusing a chilled water loop at 12 degrees when the design point is 30 to 40 and the chiller power is a permanent cost.
• Omitting filtration, conductivity monitoring and inhibitor, which is what decides whether the loop works in year three.
• Designing detection without an automatic action, when the thermal timeline is tens of seconds.
• Assuming a rear-door heat exchanger scales past roughly 70 kW per rack.

Key takeaways

• 40 racks at 120 kW is 4.8 MW, and at a 10 K rise that is 114.8 kg/s or about 6,890 litres per minute for the hall, 172 L/min per rack.
• Size CDUs 10 to 15 percent above the load they serve, then add N+1, and design for one being serviced while the rest run full.
• Run facility water at 30 to 40 degrees so dry coolers do most of the work, rather than reusing a 12 degree chilled loop.
• Filtration to 50 microns, conductivity monitoring and corrosion inhibitor decide whether the loop survives years.
• The response to a flow failure must be automatic, because throttling and shutdown happen in tens of seconds at 1,400 W per GPU.