Nvidia’s New Data Centers Run Way Hotter Than You’d Expect – And That’s The Point

As hyperscale data centers continue to grow both in size and power, with widespread backlash erupting over their presence and environmental footprint, greener cooling is no longer an option – it has to become a prerequisite. There have been a number of strides made in this area, such as using reclaimed wastewater. Amazon is using reclaimed water to cool some of its data centers and expects to use reclaimed wastewater at over 120 data centers by 2030. Other hyperscalers like Google, Meta, and Microsoft have also announced various water projects as part of a pledge to become water positive in the coming years. Nvidia’s new closed-loop liquid cooling design could address some of the environmental concerns that hang over the data center and AI industries.

Data centers have historically used some form of open-loop evaporative cooling, usually combining chillers and cooling towers to cool warm air by water evaporation. This method reduces electricity consumption, but the trade-off is extreme water consumption, as this method requires a constant supply of makeup water. According to a report published at arXiv, between 70% to 80% of the water used for evaporative cooling is lost in the process. Closed-loop liquid cooling is often more sustainable but has faced a variety of obstacles that have slowed its adoption in data centers, while newer technologies like direct-to-chip cooling and two-phase immersion cooling show even more promise. Microsoft has already been using direct-to-chip cooling and rack-level heat exchangers with its Maia 100 platform, and has also been working on its microfluidics approach to direct-to-chip cooling.

Nvidia’s take on direct-to-chip cooling uses 100 percent liquid cooling, and touts the potential for zero water consumption and no fans. This is because Nvidia’s new AI server platform is designed to run a lot hotter than you’d expect.

Nvidia’s new liquid-cooling design is part of the company’s DSX AI platform, a reference AI factory design that outlines infrastructure built on a suite of Nvidia technologies. Within the DSX platform, Nvidia’s methodology for direct-to-chip cooling is part of its DSX MaxLPS, which itself is a handful of technologies designed around performance per watt and maximizing compute across various power budgets.

Nvidia’s closed-loop, direct-to-chip cooling is designed around a coolant mixture of 75 percent water and 25 percent propylene glycol, with a coolant temperature of 45 °C, and the system has an inlet temperature of 45°C and an outlet temperature of 55°C. When the liquid exits the rack, it’s piped to an outdoor passive radiator that discharges the heat load. By using a loop that runs on warm water, a data center can theoretically avoid relying on mechanical cooling, such as chillers or cooling towers. This also means that the temperature delta between the warm water and the ambient air outside isn’t extreme enough for a passive radiator to discharge it, though Nvidia is mindful to point out that this can be geographically dependent. Arizona, for instance, where summer temperatures can regularly rise into the triple digits, chillers will still be required.

This design also incorporates monolithic cold plates, which allow the liquid to pass over not only the GPUs and CPUs, but also the NVLink switches, optic modules, and the power bus bars, which are rated for a continuous current of 5,000 amps. The closed loop is designed to be filled once, and Nvidia claims it will last for the lifetime of the facility –- a bold claim to be sure, and one that may or may not ease the concerns over data centers and water supply interference.

Nvidia’s move to 100 percent liquid cooling is not only a matter of resource management, but also one of necessity. Nvidia’s previous Grace Blackwell platform already used liquid cooling for the GPUs, but the CPUs and other components relied on air cooling. Nvidia’s Vera Rubin architecture pushes the power envelope to a different order of magnitude – one Rubin GPU uses as much as 2,300 watts, marking a 64 percent increase in TDP over the older GB300 GPU. A Vera Rubin NVL72 rack, which is composed of 72 Rubin GPUs and 36 Vera CPUs, in addition to other rack hardware (switches, optic modules, etc.), exceeds 200 kW –- well beyond what air can cool.

Data centers with cooling towers use roughly 2.6 million gallons of water per megawatt per year, so while Nvidia’s design that claims almost no water shouldn’t be taken at face value, it also shouldn’t be ignored. But it’s only part of the equation, as a data center’s on-site water usage doesn’t paint the whole picture –- AI’s water problem extends beyond the data center to the power plant. US-based fossil fuel power plants consume an estimated 2.7 billion gallons of water per day, according to the U.S. Geological Survey. The International Energy Agency (IEA) estimates that coal and natural gas power plants will supply 40% of the increased electricity demand from data centers through 2030.

That means there’s a lot of water being used down the street from data centers that those data centers don’t usually count in their water footprint. Many estimates that try to quantify how much water AI uses also don’t typically take power generation into account. Nvidia’s full liquid cooling may reduce water usage at the facility level, but there’s still a lot of work to do outside the data center.