Liquid cooling technology in AI servers

Liquid cooling is essential for modern AI servers, efficiently managing extreme heat, improving energy efficiency, and enabling higher server density compared to traditional air cooling.Why Liquid Cooling is NeededAI servers, especially those running large language models or high-performance GPUs like NVIDIA's Blackwell series, generate immense heat due to their high Thermal Design Power (TDP), often exceeding 1,000 watts per chip . Traditional air-cooling methods are insufficient for these high-density racks, as air is a poor conductor of heat and cannot remove heat quickly enough, leading to throttling or hardware shutdowns . Liquid cooling directly absorbs heat from components, maintaining optimal temperatures for performance and longevity .Types of Liquid CoolingDirect-to-Chip Cooling: Coolant circulates through cold plates attached directly to CPUs or GPUs. Water or a water-glycol mixture is commonly used to transfer heat efficiently from the chip to the cooling loop .Single-Phase Immersion Cooling: Entire servers are submerged in a dielectric liquid that remains in a single phase while absorbing heat. This method simplifies heat transfer and reduces noise .Two-Phase Immersion Cooling: Servers are submerged in a liquid that boils at the chip surface, carrying heat away as vapor. This approach is highly efficient but more complex .Cooling InfrastructureLiquid-cooled AI servers require a cooling ecosystem including manifolds, Cooling Distribution Units (CDUs), and outdoor heat rejection systems. CDUs manage temperature, flow, pressure, and fluid treatment, isolating the IT loop from the facility's cooling system . Heat can be rejected to existing water systems, air in the IT space, or dedicated facility loops depending on the architecture .BenefitsEnergy Efficiency: Liquid cooling can improve energy efficiency by up to 15% and reduce cooling costs compared to air cooling . NVIDIA's fully liquid-cooled AI servers operate with coolant up to 45°C, reducing energy consumption and eliminating most water usage .Higher Server Density: Enables racks exceeding 120 kW, which would be impossible with air cooling .Hardware Longevity: Maintains stable temperatures, reducing thermal stress on components .Reduced Noise and Space Requirements: Eliminates the need for thousands of fans and allows more compact data center layouts .Real-World ExampleNVIDIA's DSX AI factory design uses 100% liquid cooling in a closed-loop system with no fans, achieving high efficiency and sustainability. This approach drastically reduces energy and water consumption while supporting the latest high-power AI GPUs .ConclusionLiquid cooling is rapidly becoming the industry standard for AI data centers, addressing the thermal challenges of high-density, high-power AI servers. By improving energy efficiency, enabling higher server density, and extending hardware lifespan, it is a critical technology for scaling AI infrastructure sustainably .