Data centres: The new tech stopping chips from overheating

The showers and baths keeping data centre tech cool. Chris Baraniuk, Technology Reporter.

Data centres, the humming heart of our digital world, are experiencing a significant technological evolution, moving beyond traditional air-cooling methods to embrace innovative liquid-cooling solutions that are essentially treating sensitive computer chips like spa guests. These advanced systems, ranging from direct fluid showers and sprays to complete submersion in circulating baths, are crucial for managing the intense heat generated by modern, high-speed computing, particularly with the burgeoning demand for AI technologies.

"We’ll have fluid that comes up and [then] shower down, or trickle down, onto a component," explains Jonathan Ballon, chief executive at liquid cooling firm Iceotope. "Some things will get sprayed." This meticulous approach ensures that components are kept at optimal temperatures, allowing for aggressive "overclocking"—running chips at speeds beyond their standard specifications—without the risk of burnout. "We have customers that are overclocking at all times because there is zero risk of burning out the server," Mr. Ballon notes, highlighting the reliability gains. One particularly resourceful client, a US hotel chain, is exploring a novel application: repurposing the heat generated by its servers to warm guest rooms, hotel laundry facilities, and swimming pools, demonstrating a potential for energy synergy.

The critical importance of robust cooling systems was starkly illustrated in November when a cooling system failure at a US data centre disrupted financial trading operations at CME Group, a global leader in derivatives trading. In response, CME Group has since invested in "additional cooling capacity" to prevent similar incidents, underscoring the vulnerability of even the most advanced financial infrastructure to thermal management issues.

The demand for data centres is experiencing an unprecedented surge, largely propelled by the rapid advancements and widespread adoption of artificial intelligence. However, this growth is not without its controversies. The substantial energy and water consumption associated with many of these facilities has drawn increasing scrutiny from environmental advocates. In a significant show of concern, over 200 environmental groups in the US recently petitioned for a moratorium on the construction of new data centres within the country, signaling a growing unease about their ecological footprint. Despite these challenges, a segment of the data centre industry is actively pursuing solutions to mitigate their environmental impact.

This push for more efficient cooling is also driven by the escalating power of data centre computer chips. The relentless increase in processing capabilities means that conventional air-cooling methods, which rely on fans to circulate air over hot components, are becoming increasingly insufficient for the demands of some cutting-edge operations. Mr. Ballon acknowledges the growing public opposition to the construction of energy-intensive data centres, stating, "Communities are pushing back on these projects." He emphasizes that his company’s liquid cooling solutions offer a compelling alternative, requiring "significantly less power and water" and operating "silently" without the need for noisy fans.

Iceotope’s proprietary liquid cooling technology offers a comprehensive solution, capable of cooling multiple components within a data centre, not just the central processing units. The company claims its approach can reduce cooling-related energy demands by as much as 80%. Their system utilizes water to cool an oil-based fluid that directly interfaces with the computer hardware. Crucially, this water remains within a closed loop, eliminating the need for constant replenishment from local water sources, a significant advantage in water-scarce regions. Regarding the composition of their cooling fluids, Mr. Ballon clarified that while some are derived from fossil fuels, they are free from PFAS, commonly known as "forever chemicals," which pose risks to human health.

This stands in contrast to some other liquid-cooling technologies that employ refrigerants containing PFAS. Moreover, many of these refrigerants are potent greenhouse gases, contributing to climate change. Yulin Wang, a former senior technology analyst at market research firm IDTechEx, explains that "two-phase cooling systems" often utilize such refrigerants. In these systems, a refrigerant transitions from a liquid to a gas state when exposed to heat from server components, a phase change that effectively absorbs a significant amount of energy. While highly effective for cooling, this process can pose safety concerns, as "vapours can get out of the tank" in fully immersed systems, according to Mr. Wang. In alternative designs, the refrigerant is piped directly to the hottest components, primarily the computer chips. Notably, some companies offering two-phase cooling are actively transitioning to PFAS-free refrigerants.

The pursuit of optimal cooling solutions for data centres has a history of experimentation with diverse approaches. Microsoft, for instance, famously experimented with submerging a container of servers in the sea off Orkney, Scotland, in a project known as "Project Natick." The objective was to leverage the cold seawater to enhance the efficiency of the air-cooling systems within the submerged unit. While Microsoft ultimately discontinued the project last year, Alistair Speirs, general manager of global infrastructure in the Microsoft Azure business group, highlighted the valuable lessons learned. He noted that the absence of human operators in the subsea environment led to fewer operational failures, informing their land-based operational procedures and suggesting that more autonomous data centres exhibit greater reliability. Initial findings from Project Natick indicated an impressive Power Usage Effectiveness (PUE) rating of 1.07, significantly more efficient than most land-based data centres, and it consumed zero water. However, the economic viability of constructing and maintaining subsea data centres proved unfavorable. Microsoft continues to explore liquid-cooling technologies, including microfluidics, which involves circulating tiny channels of liquid directly through the intricate layers of silicon chips, a concept Mr. Speirs likens to a "liquid cooling maze through the silicon at nanometre scale."

Beyond these established tech giants, researchers are also developing novel cooling strategies. In July, Renkun Chen at the University of California San Diego, along with his colleagues, published research detailing a "pore-filled membrane-based cooling technology." This innovative approach aims to cool chips passively, eliminating the need for active fluid pumping or air circulation. Professor Chen explains that the system utilizes heat itself to generate the pumping power, drawing a parallel to the natural process of transpiration in plants, where evaporation from leaves draws water up from the roots. Professor Chen is optimistic about the commercial potential of this technology.

The increasing demand for advanced cooling solutions is further fueled by the rapid growth of AI, particularly generative AI and large language models (LLMs), which power sophisticated chatbots. Sasha Luccioni, AI and climate lead at Hugging Face, a machine learning company, points out that these technologies are exceptionally energy-intensive. "If you have models that are very energy-intensive, then the cooling has to be stepped up a notch," she states. Reasoning models, which require multiple steps to explain their outputs, are even more demanding, consuming "hundreds or thousands of times more energy" than standard chatbots. Dr. Luccioni advocates for greater transparency from AI companies regarding the energy consumption of their various products. Mr. Ballon, however, believes that LLMs, as one facet of AI, have already approached their peak productivity.