If a failure to plan is a matter of planning to fail, the participants in the Open Compute Project (OCP) are not going to let that happen when it comes to accommodating and protecting more intensive computing power. A key initiative the group is already mapping out is how to streamline the integration of more liquid-cooled ITE into data centers, including those applications where air-cooled ITE has historically been more prevalent.

The OCP mission started in 2009 as Facebook sought a way to develop energy-efficient data centers at an unprecedented scale and is now a coalition of more than 200 organizations, including Alibaba, Goldman Sachs, and Google.

In 2017, OCP began developing best practices for advanced cooling solutions (ACS). And, in 2020, the Advanced Cooling Facilities – Incubation (ACF) working group was formed and is co-led by John Menoche of Vertiv and myself, representing Victaulic.

The OCP collaboration for the ACF working group is a future-focused mind hive of 250 experts who are invited to weekly meetings from a wide array of companies, such as Asperitas, Schneider Electric, Microsoft, JM Gross Engineers, Green Revolution Cooling, CoolIT, and Alfa Laval. The group’s mission is to provide best practices and reference designs to easily accommodate for liquid-cooled ITE at scale, with speed, reliability, and minimum impact on operations.

Currently, more than 90% of data centers are air-cooled, but as demand rises for high-density chips from a range of user experiences and needs, including high-performance computing (HPC), AI, the IoT, virtual reality, and gaming — some of which involve sporadic use and rapid ramp-ups in data use — liquid-cooled ITE is expected to grow in popularity. The heat intensity around high density chips requires improved cooling technology, and those changes will impact how data centers are structured and maintained.

Liquid-cooled ITE is not entirely new. In 1985, IBM launched its 3090 processor complex, which employed liquid-cooled ITE. Of course, a lot has changed since then. Air-cooled ITE currently seems more convenient, but as densities increase, liquid-cooled ITE offers advantages in performance, efficiency, and sustainability. Going forward, basic computing demands will continue to be well-served by air-cooled ITE. But, if gamers using liquid-cooled ITE is any indication, demand for this technology will continue to expand.

To tackle this challenge successfully, data center owners need to know when, where, and how key changes will need to be made. ITE refreshes happen every three to five years, so laying the proper foundation for growth today will allow data centers to evolve in step with technology changes and avoid incurring the potentially high costs of becoming obsolete. With the right approach and proper planning, it is possible to simplify modifications like adding a liquid loop of rear door heat exchangers, cold plate servers, or immersion cooled servers. By taking advantage of expertise already available, companies can put scalable solutions in place without a huge capital outlay or significant impact on operations.

And this has been the impetus for the OCP ACF’s newest workstreams — one which provides guidance on standardized methods of connecting of liquid-cooled IT to cooling loops and, the second, which offers guidance on reference design, helping data centers efficiently add liquid cooling loops to data centers.

The intention of the OCP and ACF collaboration is to provide a framework for data center design that is open to all vendors, provides best practice guidance, and reference designs that can be adopted and widely deployed. This provides a turnkey approach for data centers to quickly upgrade. The practice of adding large amounts of liquid-cooled ITE to a data center designed around air cooling is becoming a new design requirement. Data centers that do not plan for the addition of liquid-cooled ITE may face levels of obsolescence.  

Victaulic's Reference Design Workstream captures the key concepts and requirements necessary to add liquid. The company is building a library of BIM layouts to illustrate the concepts and accelerate the speed and success of “Concept to Construction.” OCP vendors are encouraged to provide BIM content of their products. Having BIM content readily available for all pipe system components and cooling solution products simplifies prefab construction and delivery.  Minimizing construction and maximizing simple assembly of BIM modeled solutions are key when it comes to adding liquid-cooled ITE in an operating data center.

Using the BIM format provides confidence the design is accurate for construction and the components can be assembled quickly and correctly, regardless of the vendor or specific components, such as grooved couplings or flanges.

Another major workstream concerns standardizing connections and pipe system alignment. Over time, pipes used for liquid-cooled ITE will experience movement. After all, the pipes are installed at one temperature and then as the system heats up, pipes expand and may turn a little bit. Just as OCP created standardized guidelines for air-cooled ITE, we look to do the same for liquid-cooled ITE. Reducing the diversity of connection methods is key to optimizing global supply chain.

In addition, a standardized, more modular approach also means liquid-cooled ITE can be added to a data center without disrupting operations or risking damage to equipment. Hot work and job site machining processes pose unnecessary risk and time delays to operating data centers, but OCP ACF is fostering industry collaboration by creating a “concept drawings to construction” process using prefabricated solutions that increase on-site installation time, decrease risk, and, in turn, reduce cost. With just an impact wrench, installers can put together a data center expansion using grooved couplings on the pipes for liquid-cooled systems without impacting the operations of the rest of the data center.

A new area of focus for OCP-ACF is the advancement of heat reuse – capturing waste heat as an energy resource rather than releasing it. Heat reuse has already gained traction in Europe, where the energy coming off ITE is reused to power district heat in Sweden, Norway, and Denmark. Liquid-cooled ITE can utilize much higher temperature fluids, greatly expanding the opportunity for “free cooling” and heat reuse.

For OCP, the ACF subgroup is an open, global collaborative program enabling more standardized and simplified data center design, helping to guide the industry at large in the same direction. Rather than having custom solutions create custom problems, we’re solving problems by moving toward crowdsourcing best practices.