ALBUQUERQUE, N.M. — Keeping electrical circuits dry is generally considered a vital safety measure, but at Sandia National Laboratories’ high-performance computing center, technicians adjust live circuits submerged in liquid.
“It’s pretty cool — and a little unusual — to walk into a computing lab and see electrical hardware submerged in fluid,” said Sandia engineer David Damm.
Complete immersion of computer components appears to be the most effective way to cool them, explains David J. Martinez, engineering program project lead for Sandia’s Infrastructure Computing Services.
“We place whole computers — their power cables, everything — in a liquefied solution,” Martinez said. “We take an entire rack and drop it into fluid contained in big immersion tanks.”
Martinez believes that this direct contact between a liquid coolant and electrified equipment could reshape the future designs of data centers.
Reshaping data-center architectures
By submerging all parts of the computing servers in a liquid coolant that doesn’t conduct electricity, 100% of the generated heat can be captured, almost entirely eliminating the need for the power-hungry fans and chillers used in conventional cooling systems.
Martinez estimates that this type of cooling system, using liquid approximately the viscosity of cooking oil, would cut energy consumption by 70%. Unlike water-chill systems that require evaporation to lower operating temperatures, no water is lost; the coolant gives up its heat to the open air, given the right temperature differential.
Sandia’s forward-thinking approach arises from projections that the increasing water and electrical power demands for cooling in high-performance computing eventually will exceed the resources of small towns and become an unsupportable burden.
To address this, Sandia is formally testing a commercial non-conductive liquid system from the Barcelona-based company Submer throughout the summer. Early positive results led Martinez to envision a reduction in the two big problem areas of power and water use that are expected to hinder the development of faster, more powerful machines.
Promising innovations and partnerships
According to Submer promotional literature, their single-phase immersion cooling system uses a biodegradable, non-toxic, non-flammable and non-corrosive liquid. It boasts being 95% more efficient than traditional cooling technologies, while 1,400 times a better thermal conductor than air and eight times less electrically conductive.
Negotiating through ADACEN, a local data company, Martinez convinced Submer to partner with Sandia. Submer would provide fluid and equipment and Sandia would provide its expertise in system cooling and testing to perform a thorough technical evaluation of the system. Meanwhile, Martinez, who for 25 years has worked diligently to bring about the next stages of computer cooling, is happy to reduce Sandia’s costs for power and water and provide a stage for a potential revolution for the industry.
In partnership with ADACEN, code is being written specifically for the new task by Hoonify Technologies, composed of five former Sandia researchers who have taken entrepreneurial leave from the Labs.
In 2020, Submer, in collaboration with Intel Corporation, introduced the idea of a liquid coolant that would bathe all components of a computing system to provide even cooling and allow higher operating temperatures.
As a bonus, the warmer temperature of the returning liquid can offer some heat to adjacent buildings during winter months and can be used to heat labs and showers in the summer through heat-exchange processes.
Sandia will conduct comprehensive tests on this system, evaluating hardware reliability and infrastructure support cost and effort. A case study is expected to be released this fall to assess the pros and cons of the technology.