Data centres: The new tech stopping chips from overheating

The showers and baths keeping data centre tech cool

Chris Baraniuk, Technology Reporter

Data centres, the humming hearts of our digital world, are working around the clock at increasingly high speeds, generating immense heat. To prevent their sophisticated computer chips from succumbing to these searing temperatures, a revolution in cooling technology is underway, moving beyond traditional air conditioning to more intimate, liquid-based solutions. Some of these cutting-edge systems treat the powerful processors almost like they’re at a spa, employing methods akin to showers and baths to keep them operating at peak performance.

"We’ll have fluid that comes up and [then] shower down, or trickle down, onto a component," explains Jonathan Ballon, chief executive at the liquid cooling firm Iceotope. "Some things will get sprayed." This direct application of cooling fluid ensures that heat is wicked away efficiently from the most vulnerable components. In other sophisticated setups, the industrious gizmos, the computer chips, recline in circulating baths of fluid. This liquid acts as a highly effective heat conduit, ferrying away the thermal energy they generate. This meticulous cooling allows them to function at "overclocked" speeds, pushing their capabilities far beyond standard operating parameters.

"We have customers that are overclocking at all times because there is zero risk of burning out the server," Mr Ballon states, highlighting the reliability and performance gains offered by these advanced cooling methods. He further illustrates the innovative applications of this waste heat, noting that one client, a US-based hotel chain, is exploring the possibility of repurposing the heat generated by their hotel servers to warm guest rooms, the hotel laundry facilities, and even the swimming pool. This demonstrates a growing trend towards energy circularity within data centre operations.

The critical importance of effective cooling was starkly underscored in November when a cooling system failure at a data centre in the US disrupted financial trading technology at CME Group, the world’s largest exchange operator. In the aftermath, the company significantly bolstered its cooling capacity to prevent a recurrence of such a disruptive incident.

Currently, the demand for data centres is experiencing an unprecedented surge, largely fueled by the rapid advancement and widespread adoption of Artificial Intelligence (AI) technologies. However, this boom is not without its challenges. The substantial energy and water consumption of many of these facilities is increasingly drawing scrutiny and raising environmental concerns. In the United States, over 200 environmental groups have called for a moratorium on the construction of new data centres, citing their environmental impact. Despite these concerns, some data centre firms are actively seeking solutions to mitigate their ecological footprint.

Adding to the urgency is the ever-increasing power of data centre computer chips. Many industry experts contend that traditional cooling methods, such as air cooling that relies on fans to circulate air over hot components, are becoming insufficient for the demands of modern high-performance computing. Mr Ballon acknowledges the growing public opposition to energy-intensive data centre projects, stating, "Communities are pushing back on these projects." He emphasizes that his company’s liquid cooling solutions offer a compelling alternative, requiring "significantly less power and water" and operating silently, without the need for fans.

Iceotope’s innovative approach to liquid cooling, which can effectively cool multiple components within a data centre, not just the processing chips, has the potential to reduce cooling-related energy demands by as much as 80%. The company’s technology utilizes water to cool an oil-based fluid that directly interfaces with the computer hardware. Crucially, the water operates within a closed loop, eliminating the need for continuous water extraction from local supplies. When questioned about the origins of the oil-based fluids, Mr Ballon confirmed that some are derived from fossil fuel products, but stressed that none contain PFAS, commonly known as "forever chemicals," which pose risks to human health.

This distinction is important as some liquid-based data centre cooling technologies employ refrigerants that do contain PFAS. Furthermore, many of these refrigerants are potent greenhouse gases, contributing to the exacerbation of climate change. Yulin Wang, a former senior technology analyst at market research firm IDTechEx, explains that two-phase cooling systems often utilize such refrigerants. In these systems, a refrigerant transitions from a liquid to a gas state when heated by server components. This phase change absorbs a significant amount of energy, making it an effective cooling mechanism. However, in some designs, data centre technology is fully submerged in large volumes of PFAS-containing refrigerant, raising concerns about potential leaks. "Vapours can get out of the tank," Mr Wang warns, highlighting potential safety issues. In alternative configurations, the refrigerant is piped directly to the hottest components, primarily the computer chips. Encouragingly, some companies offering two-phase cooling are transitioning to PFAS-free refrigerants.

Over the years, the industry has explored a diverse array of cooling strategies in the pursuit of optimal solutions for data centre hardware. Microsoft, for instance, famously submerged a tube-like container filled with servers into the sea off the coast of Orkney, Scotland. The intention was to leverage the cold Scottish seawater to enhance the efficiency of the internal air-based cooling systems. Last year, Microsoft announced the decommissioning of the project, but the insights gained were significant. Alistair Speirs, general manager of global infrastructure in Microsoft’s Azure business group, noted that operating without human intervention led to fewer malfunctions, which informed their operational procedures and pointed towards increased reliability in more automated data centres. Initial findings from the subsea data centre project indicated a Power Usage Effectiveness (PUE) rating of 1.07, suggesting a significantly higher efficiency compared to most land-based data centres, and it required no water. However, the economic viability of constructing and maintaining subsea data centres ultimately proved unfavorable. Microsoft continues to explore liquid-based cooling concepts, including microfluidics, which involves circulating tiny channels of liquid through the intricate layers of silicon chips. "You can think of a liquid cooling maze through the silicon at nanometre scale," Mr Speirs elaborated.

Researchers are also contributing novel ideas to the field. In July, Renkun Chen at the University of California San Diego, along with colleagues, published research detailing a pore-filled membrane-based cooling technology. This innovative approach aims to cool chips passively, without the need for active fluid pumping or air circulation. "Essentially, you are using heat to provide the pumping power," explained Professor Chen, drawing an analogy to the natural process of water evaporation from plant leaves, which drives water uptake through the plant. Professor Chen expressed his ambition to commercialize this technology.

The demand for advanced data centre cooling solutions is escalating, according to Sasha Luccioni, AI and climate lead at Hugging Face, a machine learning company. This trend is driven, in part, by the burgeoning demand for AI, including generative AI and large language models (LLMs), which power sophisticated chatbots. Previous research by Dr. Luccioni has demonstrated the substantial energy consumption of these technologies. "If you have models that are very energy-intensive, then the cooling has to be stepped up a notch," she stated. She further highlighted that reasoning models, which explain their outputs through multiple steps, are even more power-hungry, consuming "hundreds or thousands of times more energy" than standard conversational AI. Dr. Luccioni advocates for greater transparency from AI companies regarding the energy consumption of their various products.

For Mr Ballon, LLMs represent just one facet of AI, and he posits that they may have already "reached their limit" in terms of productivity gains. Nevertheless, the ongoing development and deployment of increasingly powerful AI systems necessitate continuous innovation in data centre cooling to ensure operational stability and efficiency. The future of data centres hinges on their ability to manage heat effectively, paving the way for more sustainable and powerful digital infrastructure.