Managing Thermal Loads In A Colocation Data Center
Data center have specific needs.
Data centers are more critical to the operation of our businesses than they have ever been in the past, and the criticality increases every day. In fact, they are not just critical to our businesses but to our daily way of life. It is also understood that the power densities in data centers continue to increase on a scale never seen before. Which is why, similar to the electrical distribution systems, properly designing, building, and controlling the HVAC cooling systems in the data center white space today is more critical than ever. The following is intended to be a high-level description of the HVAC control systems in data centers.
Before we discuss data center cooling control schemes, we must first make sure the cooling systems are properly sized from a cooling and efficiency standpoint. Even though data centers differ significantly from most building types, designing the HVAC and HVAC control systems are not unlike other building types. Where instead of designing a system for comfort cooling, we’re designing a system to maintain the proper cooling needed for the data center technology (servers).
You start with the basics: understanding the data center programming requirements, space criteria, data hall IT and electrical gear loading, indoor and outdoor climatic conditions, and redundancy requirements. From there we develop the cooling load profile and begin the selection of cooling systems. Basically, how much cooling is required and with what level of reliability to properly cool the white space to meet the client’s program requirements. Client requirements can consists of the server inlet air temperatures, hot aisle/cold aisle temperatures, differential pressure between hot and cold aisles, data hall pressurization, and dewpoint (dp) control.
Additionally, we also need to know the owner PUE requirements, Day 1 vs. Day N loading, cost budgets, owner design standards (if available), and flexibility needs. Once we start the design process we validate it by simulating the HVAC systems utilizing computational fluid dynamics (CFD) modeling software for interior data hall, electrical support rooms, and exterior. This allows us to further predict and confirm that the systems we designed will successfully support the cooling needs.
The following is an example of the design criteria for a colocation data center located in northern Virginia:
Outdoor Design Conditions
- Summer outside: 95.1°F drybulb (db)/ 75.8°F wetbulb (wb)/ (79.1 wb for cooling tower selection)
- Summer dehumidification: 75.4°F dp / 83.1°F
- Winter: 14.5°F db
- Winter humidification: -5.9°F
- Sensible heat rejection equipment (air cooled), 10 years extreme minimum: 1.6°F
- Sensible heat rejection equipment (air cooled), 10 years extreme maximum: 101.6°F
Source: 2013 ASHRAE based on 0.4% criteria
Indoor Design Conditions
- Data center: 64.4° to 80.6°F db / 42° to 60°F dp and minimum 20% relative humidity (RH) at the cold aisle
- Mechanical/electric rooms: 65°F min, 95°F max, 60° dp max
- Battery rooms: 77°F db max temperature
Please note that the inlet server temperatures, identified in ASHRAE TC 9.9 20111, are higher but our experience indicates that colocation owner requirements tend to be slightly lower ranging from 75° to 77°F db.
- Data center: 5 cfm/p + 0.06 cfm/sq ft
- Mechanical, electrical, utility: 0.06 cfm/sq ft
- Data center: 1,000 sq ft/person
- Data center: Lighting 1 Watt/sq ft /equipment 10 MW IT load
- Office, meeting: Lighting 1 Watt/sq ft; Equipment 2 Watt/sq ft
- Main electric: Lighting 1 Watt/sq ft; Equipment 180 kW heat rejection for 10 MW IT load
- Mechanical rooms
- Electric rooms
To describe the HVAC control systems, we’ll use the human body as an analogy. The equipment such as chillers, pumps, condenser units, computer room air conditioning (CRACs), computer room air handling (CRAHs), etc., are similar to the functionality of the heart. They distribute conditioned air and/or water through the system for means of cooling. The distribution systems such as ductwork and piping are the veins and arteries. The air and water that is distributed is the blood that flows through our bodies. The brain for the entire system is the HVAC controls. Similar to our bodies, the controls systems logically bind all the systems together. The HVAC controls can react to changes in system, overcome deficiencies, and indicate if there is an event.
HVAC control systems can be referred to as building management system (BMS), direct digital controls (DDC), programmable logic controller (PLC), energy management system (EMS), building automation system (BAS), building energy system (BES), supervisory control and data acquisition (SCADA), or simply “the controls system.” Each of these have different meanings and level of function but they serve similar intent, which is to properly maintain control of the building operations systems. Depending upon which type of control system is chosen (level of complexity and required reliability to meet the data center program requirements, the size of the systems, and how it’s configured) will affect capital expenditures (CAPEX) and operational expenditures (OPEX).
To further boil down the controls systems, we typically describe options to our clients of either a full DDC system, PLC system, or a “hybrid” of both:
DDC systems are frequently used in commercial applications and are made-up of microprocessors, control wiring, and control/measurement devices, e.g., thermostats, damper, and valve actuators. The DDC system gathers information by constantly monitoring analog and digital I/O signals such as temperature, humidity, pressure differential, and develops an output signal to control end devices, e.g., actuators and on/off switches. The output signal can be derived by instructions or control sequence and can be as simple as time of day or by utilizing proportional/integral/derivative (PID) control algorithms. DDC systems can be provided by the equipment manufacturer or can be provided by a separate controls manufacturer.
PLC systems are typically used in industrial applications. In the data center sector, they are often used for large central utility plants which may provide cooling and power for multiple buildings. PLC controllers are industrial grade, easy to program, flexible, and reliable. They can also be more costly in comparison to DDC systems. We recently completed a project where the PLC systems had a 20% to 50% premium from a CAPEX standpoint.
DDC/PLC hybrid systems are usually recommended for a campus environment where there’s a central utility plant (CUP) serving multiple buildings. The building HVAC systems are typically controlled via a DDC system. The CUP will be controlled via the PLC systems due to the industrial nature of large cooling equipment
There are advantages/disadvantages for each system such as costs, proprietary equipment, ease of programming, or robustness. In most cases we see owners leaning toward DDC systems for non-critical spaces and PLC for critical systems. This results in an overall hybrid BMS system.
Once you’ve properly sized and selected the HVAC equipment and control systems serving the white space, the control sequences of operation can be developed. Control scheme options are very much dependent upon the systems that are being used. HVAC systems can consist of CRAH or CRAC, centralized air handling units, close-coupled cooling such as in-row coolers, rear door heat exchangers, liquid-cooled servers, etc. Each of these systems include sub-systems such as air cooled DX, water cooled DX, air cooled pumped refrigerant, chilled water, indirect and direct evaporative cooling, and airside economizer.
The control sequences of operation can be described as a script which dictates how the systems will operate to maintain the design criteria. Meeting the program design criteria is directly correlated to meeting the owner requirements (indoor temperatures, humidity, pressure differentials, server inlet air temperature). The control schemes along with the equipment they are controlling also allow for energy conservation measures to be implemented during ideal outdoor air conditions. To further help define the control schemes, control process instrumentation diagrams “P&IDs” are also provided to reinforce design intent. Control diagrams illustrate the equipment, control devices, and associated I/O points diagrammatically.
In conclusion, there are many differences between HVAC systems serving data centers vs. other building types. The biggest differences revolve around reliability requirements, the quantity of I/O points, and ensuring proper server cooling requirements in lieu of maintaining human comfort. The systems and the way they are controlled can differ dramatically but the end result is to properly control the HVAC equipment to meet the client’s requirements.