As previously observed by this publication, the modern data center is fairly new with most contemporary facilities emerging since 2015. Unlike partial-day operations, such as office buildings and education facilities, whose mechanical systems have multi-decade life expectancies, the complex 24/7/365 equipment within data centers is already reaching its useful and/or reliable life despite its relatively young age. Attention is therefore focused on upgrading this existing infrastructure to maintain optimal and uninterrupted data center operations

Data center infrastructure refers to all the physical components that play a vital role in day-to-day operations. They can be classified in two categories: IT components (computers, cables, etc.) and building systems (HVAC, electrical/power, plumbing/water systems, etc.) found within the walls of the facility. This article focuses on the latter.  

Staying up to speed

Data center owners and operators continue to construct new facilities at a rapid pace while simultaneously getting their arms around the inventory built over the past couple of decades. The speed of bringing these new facilities online, coupled with the limitation of personnel and the ongoing strain on supply chains, makes it burdensome to maintain older generation sites.

In addition, “gen1” sites require parity to match the current generation in terms of loads, technology, efficiency, redundancy, and maximum investment output. The explosive growth of the IoT, 5G, and AI creates increased information on a daily basis. Therefore, hyperscale facilities seem to be attracting the most investment. According to a 2023 report by Precedence Research, the hyperscale data center market size is projected to grow from $102 billion in 2023 to $935 billion in 2032, a compound annual growth rate (CAGR) of 27.9%. This tendency of data moving to these hyperscale facilities and powerhouse-of-infrastructure sites is known as data “gravity,” which establishes them as the backbone of multigenerational data centers.  

Owners and operators resultingly find themselves caught between the pros and cons of continuing to build new greenfield sites and/or repurposing and improving existing infrastructure to best serve end users and meet the dramatic pace of growth. 

The changing face of infrastructure replacements 

The data center infrastructure replacement industry has been known for its like-for-like replacement approach for a substantial part of history. Most data center owners and operators see this as complacency and unacceptable amid increasing data processing and storage demands that can strain mechanical, electrical, and plumbing (MEP) support systems. 

First, data center design and maintenance experts must improve energy efficiency. HVAC, for instance, is a key focus area given the significant chip heat that produces massive and ever-increasing amounts of rack heat in these facilities. On the other hand, tracking operating performance of the racks and the HVAC system, longevity of the HVAC equipment and racks, and maintenance is key to enterprise reliability. 

Stepping back and taking a long-term look at the entire ecosystem related to these supporting systems is also a necessary step in their imminent replacement. Environmental, social, and governance (ESG) needs to be the basis for decisions going forward. Addressing a single piece of the puzzle simply is not enough to create long-term solutions for completely tackling the ecosystem.

The possibilities are virtually endless within this climate, but below are 10 key data center infrastructure opportunities and challenges as well as applicable solutions that should be front and center.

1.    Ever-increasing chip technology

Ever-increasing chip technology has increased the rack intensity (kW/rack) from 3 kW/rack to 40-plus kW/rack over the last 15 years and even higher in cryptocurrency applications. Cooling is, therefore, critical for data center operations to keep IT systems online. 

• a. Status quo — Build new data centers. 

• b. Creative and innovative solutions: 

  • i. Controls improvements — Constant search for control optimization and energy reduction through higher cold aisle temperatures and use of economization.
  • ii. Direct expansion (DX) system improvements — Packaged DX air conditioners or computer room air conditioners (CRAC) are the most common type of cooling equipment for smaller data centers. Manufacturers have developed improvements in energy efficiency for cooling systems by employing DX units over the past decade. Indoor CRAC units are now available with different heat rejection options. Features, such as evaporative spray and pre-cooling water coils upstream of the evaporator coil, reduce the amount of mechanical cooling required by optimizing the use of mother nature.
  • iii. Air economization — Higher cold aisle temperatures and increased time-based use of outdoor air to minimize mechanical cooling.
  • iv. Cold-aisle temperature increases as high as 90°F.  
  • v. Chilled water to rack — Exceeding the thermodynamic property of air and, subsequently, the requirement to use water as the primary heat transfer in the rack versus air. 
  • vi. Immersion cooling — Around since the Manhattan Project, this chilling technology involves submerging servers in dielectric fluid.
  • vii.    Retro-commissioning — Constant and ongoing validation of MEP system operation.
  • viii. Recommissioning — Commissioning systems that were never validated for operational efficiency in the first place.

2.    Electrification/decarbonization/GHG emissions

Most data centers are 100% electrified already. The few remaining sources of carbon yet to be addressed are typically in the backup power (generator) systems, which primarily rely on diesel. 

• a. Status quo — Direct replacement of carbon-based systems, such as diesel generators.
• b. Creative and innovative solution — Use of natural gas for backup power generation, which provides reliability that can typically maintain Tier 3 status. Some of the most obvious pros of natural gas are it’s cleaner, less expensive than other nonrenewable fuels, doesn’t degrade, and is considerably efficient. Natural gas, or even propane, can be safely stored on-site.  In comparison to diesel, the emissions of sulfur, nitrogen, and carbon dioxide (a greenhouse gas) are also considerably lower. Supplement other technologies, such as solar, wind, and fuel cells, for demand reduction or complete replacement of carbon-based equipment.

3.    High-speed fiber improvements 

Transfer speeds and data usage needs increasingly exceed the capacity of original fiber and need to be upgraded accordingly. 

• a. Status quo — Constructing new facilities and mixing data halls.
• b. Creative and innovative solution — Undertaking outside plant (OPS) and inside plant (ISP) fiber upgrades utilizing existing and underutilized infrastructure to accommodate increased fiber speeds.

4.    Inventory assessment and documentation 

Many existing mission critical facilities were quickly built and have undergone fast-paced modifications without proper recordkeeping. Assessments, documentation, validation, and building information modeling (BIM) should be undertaken to create an accurate accounting of their infrastructure.

• a. Status quo — AutoCAD or contractor markups that fail to represent and maintain the speed of change within the data center infrastructure environment.
• b. Creative and innovative solution — BIM that is maintained in a “live” model of the space that represents 3D space allocation, including maintenance and operational clearances along with imbedded information attributes  maintenance instructions, parts locations, replacement parts quick ship data, labor hours to replace, etc. — to every component in the data center model.

5.    Noise pollution

While many data centers are constructed on greenfield sites away from major population centers, urban growth and the need for edge sites closer to the urban core create some issues due to noise transpiring from these facilities.

• a. Status quo/past — Construct new facilities away from sensitive areas.
• b. Creative and innovative solution — Perform acoustical baseline and post construction analysis of property line sound conditions. Utilize an acoustical consultant to address ways to reduce or eliminate sound concerns for neighbors.

6.    Power systems

Data center digital infrastructure requires significant and reliable electricity. A fractional second of interruption results in operational and economic impact. As a result, power infrastructure is one of the most critical components of a data center and one that requires close attention during improvements, replacements, and repairs. The power chain starts at the utility substation and ends at the 12-V supply to servers. Downtime along any part of that electrified chain is unacceptable.

• a. Status quo — Decentralized and segregated power chains.
• b. Creative and innovative solutions — Power efficiency. Utilizing newer cooling technologies to reduce energy consumption and transferring power load to revenue-generating IT equipment.

7.    Redundancy improvements

Single points of failure (SPOF) remain in many existing data center facilities. Some of these were overlooked during initial design and construction efforts and/or some facilities have decided to change/improve their Tier rating. 

• a. Status quo — Manual monitoring and visual observation of systems. Operations and maintenance (O&M) program is expensive and unreliable.
• b. Creative and innovative solution — Integrated control systems, elimination of SPOF through improved technologies, use of new innovations, and additional infrastructure to supplement the SPOF.

8.    Space allocation

Due to limited wall/floor space, many mission critical projects offer limited room for revisions. Infrastructure replacement experts often must anticipate space needs for future infrastructure upgrades during their initial site visit to “reserve” space for new equipment. Fortunately, larger support systems (lithium-ion batteries in UPS systems), innovative technologies (immersion cooling and water to the rack), and decentralized UPSs are all improving the allowable space for infrastructure improvements. 

• a. Status quo — Leaving open and wasted space both in and out of the data halls.
• b. Creative and innovative solution — Improved systems allow for additional revenue-generating IT equipment with higher densities and faster speeds.

9.    Structural limitation improvements 

MEP systems are significant, and data center structural loading is pushing 500 PSF in many cases, taxing the original structural design. As a comparison, a typical office building is designed for less than 150 PSF. The documented information on revised structural limits in many mission critical facilities is limited and/or inaccurate. This forces owners to take a conservative stance that “no additional systems can be supported," which leads to more expensive methods of designing and installing additional infrastructure. 

• a. Status quo — Loading servers into any available space in the rack, resulting in uneven weight distribution and safety hazards. Retro-structural reinforcement.
• b. Creative and innovative solution — Rack management through data center infrastructure management (DCIM). Documenting live loads on the floor and roof structure through BIM. 

10.    Attention to industrial water use efficiency (WUE)

To reduce waste, significant attention is focused on water use as public utilities begin to stringently regulate and reduce the allowable discharge of wastewater as well as push for treatment and reuse.

• a. Status quo — More chemicals, more water.
• b. Creative and innovative solution — Water reuse, closed-loop chilled water systems, and immersion.

Satisfying any of these demands individually is quite easy, but it takes creativity and innovation to simultaneously develop options that maintain a reasonable first cost while also addressing the complete ESG cycle.