Key takeaways
- NVIDIA's GB300 NVL72 reference architecture lists up to 142 kW per rack, up from roughly 120 kW for the earlier GB200 NVL72.
- Individual accelerator chips, including Blackwell GPUs, now carry thermal design power above 1,000 W each — direct-to-chip liquid cooling is part of the platform design, not an optional upgrade.
- A standard 60–80 kW side-car CDU can't keep up with a 130–142 kW rack; deployments move to larger in-row or facility CDUs (HPE, for one, documents an in-row 1.3 MW unit supporting up to eight racks).
- Power arrives over a 50–51 V DC busbar fed by eight 33 kW power shelves, not per-server supplies.
- Closed-loop direct-to-chip cooling largely eliminates the evaporative water loss of traditional cooling — a meaningful advantage where water availability is already constrained.
What "chip-to-chiller" cooling actually is
Direct-to-chip (D2C, or cold-plate) cooling clamps a copper cold plate onto each high-power processor. Coolant flows through the plate, picks up the heat at the source, and carries it out of the server to a coolant distribution unit (CDU). The CDU is the bridge between the rack's technology loop and the facility's primary water loop, where chillers or dry coolers finally reject the heat. Keeping the two loops separate lets the rack side run hotter and dirtier while the facility side stays compatible with existing plant equipment — a "chip-to-chiller" path in the literal sense.
Why GB300‑class racks need it
NVIDIA's GB200 and GB300 NVL72 systems pack 72 Blackwell GPUs and 36 Grace CPUs into a single rack. NVIDIA's own DGX GB200 rack guide lists approximately 120 kW for the reference configuration; the GB300 NVL72 reference architecture lists up to 142 kW. That density is driven by the chips themselves — individual accelerators, including Blackwell GPUs, now exceed 1,000 W of thermal design power each, more than triple what GPUs generated seven years ago. At that per-chip and per-rack density, the GPUs and CPUs are cooled through direct-to-chip liquid loops as part of the platform design, not as a rack upgrade bolted on afterward.
GB300 NVL72 at a glance
| Spec | Figure |
|---|---|
| Rack power | Up to 142 kW (GB200 NVL72: ~120 kW) |
| GPUs / CPUs per rack | 72 Blackwell GPUs, 36 Grace CPUs |
| Per-GPU memory | 288 GB HBM3e (B200-class: 180 GB) |
| Memory bandwidth | ~8 TB/s per GPU |
| NVLink bandwidth | 130 TB/s aggregate across the rack; 1.8 TB/s per GPU |
| Rack weight / footprint | ~1.36 t in a 600 mm-wide, 48U cabinet |
| Cooling | Direct-to-chip liquid, by design — no air-only option |
| List price | Roughly $3.0–3.4M per rack, ~$3.9M all-in with networking and storage |
Power delivery
Rather than per-server power supplies, the NVIDIA reference design feeds the rack through eight 33 kW power shelves into a common busbar running at roughly 50–51 V DC. That low-voltage DC architecture is built to move well over 100 kW into a single cabinet — the reference rack also weighs about 1.36 metric tons in a 48U footprint, a point load well above what most raised floors are rated for (typical raised-floor ratings run 1,200–1,500 kg/m²; a fully loaded NVL72 cabinet works out to over 2,000 kg/m²).
Sizing the cooling loop
Supplier guidance calls for around 2 liters of coolant per second per rack at a 25°C inlet temperature; at a 40°C inlet, the required flow rises to about 119 L/min, versus roughly 59 L/min at 25°C. A typical side-car CDU handles 60–80 kW of heat — well short of the 130–142 kW an NVL72 rack produces — so deployments lean on larger in-row or facility CDUs instead. HPE, for example, documents an in-row 1.3 MW CDU that can support up to eight racks, illustrating how operators scale CDU capacity rather than trying to force it through a single side-car unit. CDU placement itself is deployment-specific: it can be rack-integrated, installed in-row, or run as shared facility equipment.
Where this sits against air and rear-door cooling
| Architecture | Typical rack ceiling | Notes |
|---|---|---|
| Air cooling | ~10–15 kW | Works well at lower densities; relies on room-level airflow |
| Rear-door heat exchanger (RDHx) | ~70 kW | Liquid loop captures exhaust air; covers moderate-density racks |
| Direct-to-chip (D2C) liquid cooling | Keeps up above 100 kW | Cold-plate loops on chips, paired with CDUs and a facility water loop |
Above roughly 100 kW per rack — where GPU-dense AI racks like the NVL72 routinely operate — direct liquid cooling is the architecture that can keep up; it isn't necessarily the only technical path, but it's the one the current generation of NVIDIA reference designs is built around.
Facility-level considerations
Water. Traditional evaporative cooling can lose the majority of its water to the atmosphere; a single 100 MW hyperscale facility using conventional evaporative cooling can consume roughly two million liters of water per day. Closed-loop direct-to-chip cooling recirculates its coolant instead, which is why operators in water-stressed regions increasingly favor it — India's data centers, for context, consumed an estimated 150 billion liters of water in 2024–25, a figure regional planners expect to keep climbing with AI buildout.
Weight and floor loading. At around 1.36 metric tons in a 48U cabinet, a fully configured NVL72 rack lands well above the 1,200–1,500 kg/m² most raised floors are rated for — over 2,000 kg/m² of point load — which is one reason operators plan liquid-cooled halls as purpose-built spaces (slab-on-grade or reinforced pathways) rather than retrofits of older raised-floor rooms.
Loop design. Most production systems run single-phase, keeping the coolant liquid throughout the secondary loop because that behavior is predictable and easier to integrate with existing plant water. Leak management is treated as non-negotiable in these deployments: drip-less quick-disconnects, secondary containment, and continuous leak detection are standard practice, not optional hardening.
Conclusion
GB300-class racks push per-rack power into a range — 120 to 142 kW, on NVIDIA's own reference figures — that air cooling and rear-door heat exchangers were never designed to handle. Direct-to-chip cooling paired with a right-sized CDU is the path the current generation of reference architectures is built around, but it comes with facility-level costs beyond the chip: DC busbar power distribution, reinforced floor loading, water-loop infrastructure, and leak-detection systems that traditional air-cooled halls don't need. Planning a new AI-ready facility — or retrofitting an existing one — means budgeting for all of it, not just the rack itself.
Sources
This article was researched and fact-checked against the following sources:
- Nvidia Blackwell Explained: Data Center Impact | ModulEdge (moduledge.com)
- Planning liquid cooling for new AI data center builds in India - DCD (datacenterdynamics.com)
- Direct-to-Chip Liquid Cooling: Working Principle, Architecture and Engineering Guide (attom.tech)
- How to Plan AI-Ready Cooling for High-Density Racks: A Step-by-Step Guide for Data Center Engineers and Procurement Managers - Panel Builder US (panelbuilderus.com)
- GB200 & GB300 NVL72 Power and Cooling Requirements | Pantheon (pantheon.run)