Key takeaways

  • AI compute is moving beyond GPUs; CPUs and accelerator chips now demand dedicated thermal solutions.
  • Direct‑to‑chip liquid cooling (single‑phase) remains the workhorse today, while two‑phase and immersion approaches are gaining traction as power densities climb.
  • Air‑based methods top out around 10‑15 kW per rack; rear‑door heat exchangers extend to roughly 70 kW; direct liquid cooling is required beyond 100 kW.
  • Integrated stacks that combine facility‑level chillers, rack containment, and negative‑pressure CDUs can protect hardware across the whole thermal path.
  • Designing for modular growth (in‑rack and in‑row CDUs) lets operators scale from a few hundred kilowatts to multi‑megawatt AI clusters without disruptive rebuilds.

The density shift in AI compute

AI workloads are no longer limited to graphics processors. Modern training clusters lean heavily on high‑performance CPUs and purpose‑built accelerator chips, and they run at sustained loads that push rack power well above traditional server baselines. The result is a rapid rise in heat flux per foot of rack space, forcing data‑center engineers to treat cooling as a primary design driver rather than a secondary afterthought.

Cooling architectures landscape

Architecture Typical rack power limit Key characteristics
Air cooling 10–15 kW Relies on room‑level airflow; energy‑intensive at higher densities
Rear‑door heat exchanger (RDHx) ~70 kW Liquid loop captures exhaust air; fits existing racks
Direct‑to‑chip liquid cooling (single‑phase) >100 kW Coolant contacts chip surfaces via cold plates; highest thermal efficiency
Immersion cooling Variable, often >100 kW Servers submerged in dielectric fluid; eliminates internal heat‑sink constraints

The table reflects the density thresholds reported by leading thermal‑solution providers and matches the ranges that engineers see in practice.

Direct‑to‑chip cooling – the current workhorse

The most widely deployed AI cooling method today circulates coolant across CPUs and GPUs in a closed‑loop system. This approach, often called single‑phase direct liquid cooling, has become the default because it removes heat more efficiently than air, allowing GPUs to stay at full utilization without throttling. It also cuts fan power dramatically, translating into better overall PUE for the facility.

Two‑phase variants – where the coolant changes phase to absorb more heat – are still in early‑stage pilots. Industry observers note that adoption will pick up as chip‑level TDP and thermal flux exceed what single‑phase loops can handle without larger heat exchangers.

Integrated thermal stacks from the chip to the chiller

A growing number of vendors are bundling cooling across three layers:

  1. Facility‑level chillers that provide a stable, high‑capacity heat‑rejection backbone.
  2. Rack‑level containment that seals the hardware environment and adds fire‑suppression and DCIM monitoring.
  3. Negative‑pressure CDUs that draw coolant directly at the GPU die while protecting against leaks by pulling air inward if a seal fails.

This three‑tiered architecture is designed to protect assets at every level – from the chiller plant through the rack and down to the silicon.

Designing for modular growth

When planning an AI‑ready data center, the first step is to quantify the per‑rack heat load. Traditional designs centered on 5–10 kW averages, but a single AI rack can now generate 30 kW, 50 kW, 100 kW, or more. Selecting the right cooling architecture early avoids costly retrofits.

For densities under 15 kW, upgraded air handling may suffice, but anything above that quickly moves into the realm of rear‑door heat exchangers or direct liquid cooling. In‑rack CDUs that deliver up to 150 kW per unit are a common choice for space‑constrained deployments, while in‑row CDUs can be stacked in 250 kW increments to exceed 1 MW when clusters expand.

Redundancy is built into the pump and fan arrays, and leak‑aware manifolds with drip‑less connectors mitigate the primary failure mode of liquid loops. By treating the coolant distribution unit (CDU) as the circulatory system for the data center, engineers can size pressure, filtration, and heat‑exchanger capacity to handle worst‑case load spikes during intensive training bursts.

Emerging paths: two‑phase and immersion

Two‑phase direct‑to‑chip cooling promises a smoother path to densities that would otherwise force a move to immersion. It leverages the latent heat of vaporization to move more energy per unit mass of fluid, but field deployments remain limited to pilots.

Immersion cooling, on the other hand, submerges entire servers in a dielectric fluid, removing the need for internal heat sinks and enabling even higher rack densities. While the technology is maturing, many operators still favor direct‑to‑chip solutions because they retain a more familiar hardware stack and allow incremental upgrades.

Conclusion

High‑density AI compute is redefining thermal design. The industry has moved from generic air‑cooling to tightly integrated liquid architectures that address heat at the chip, rack, and facility levels. By matching cooling density to the expected rack power—air for low‑kW, rear‑door for mid‑kW, and direct liquid for high‑kW—operators can protect costly GPU assets while keeping PUE in check. As accelerator TDPs keep climbing, two‑phase and immersion options will become practical complements, but today’s deployments rely on proven single‑phase direct‑to‑chip loops backed by robust CDUs and chassis‑level containment.

Sources

This article was researched and fact-checked against the following sources: