In today’s 21st century business environment, the need for efficient data centers is increasing at unprecedented rates as the demand for computing, processing power, and data storage grows exponentially. The energy consumption in a data center can be significantly more than a typical office space, and a considerable portion of the energy cost (30% to 50%) is dedicated to the data center’s cooling system. More than ever, IT equipment is getting smaller in size yet more powerful, and the need for a proper and efficient cooling system design plays an important role in saving energy.
The new generation of computers operates under higher temperatures, which does reduce the cooling cost and makes it possible for a higher computer intake temperature (80° to 85°F). However, going beyond the intake temperature’s design criteria can cause overheating and IT equipment to be more susceptible to failure. As a result, the need for accuracy and a scientific-based design of the data centers’ thermal management requires the use of advanced engineering tools such as computational fluid dynamics (CFD) to parameterize and visualize variable designs. CFD enables design engineers to recognize issues at early stages of the design and tackle the engineering challenges that cannot be solved accurately using a conventional design approach.
CONTAINMENT DESIGN
As air passes through servers, its temperature rises. The recirculation of this hot air into the intake can eventually cause equipment failure. Installing a containment and chimney configuration can prevent the mixture of cold and hot air that forms hotspots while also improving the cooling system efficiency. In order to justify the installation costs and confirm potential energy savings, CFD should be applied during containment design. The current airflow situation in existing data centers can be investigated and the possible hotspots under the data hall's design can be predicted through room simulation and temperature impact evaluation. Figure 1 compares the temperature contours at 4 ft above the floor for a data hall with and without containment/chimney. The results show that the maximum temperature was reduced from 125°F (no containment) to 95°F (containment) due to preventing the recirculation of hot air.
GAPS AND CRACKS
While the use of a containment and chimney configuration is effective, it is not a standalone solution to separate cold and hot air in a data center. It is important to also investigate the impact of structural gaps in data center design. Air can penetrate the gaps and cracks that exist in the cabinet structure between the containment/chimney and racks. It can also enter a failed server when its fan cannot overcome the pressure gradient between the cold and hot aisles.
Depending on the location and size of such gaps, hotspots can form or the cooling load can become wasted, despite the investment in containment and chimney installation. CFD can model the impact of the gaps and provide valuable information to predict the issue in advance and enable the design to be improved. Further, hot air recirculation and cooling load leakage occur when enough pressure is present to force the hot or cold air through the gaps. Thus, the areas with a higher IT load are more susceptible to hot air recirculation and the areas with a lower IT load are prone to cooling load leakage. Figure 2 shows the recirculation of hot air through gaps between the ceiling and containments in a data hall at the area with a high density IT load.
MATERIALS AND INSULATIONS
The materials used in data center buildings such as racks, cabinets, and containments hold different thermal capacities, so heat resistance must be considered during the design. For example, heat transferred through the ceiling, cabinets, and containments has an impact on thermal management. Choosing the proper materials with reasonable R-values reduces the heat transfer between the hot and cold aisles. The heat transfer rate increases with higher temperature differences between the cold and hot sides. CFD helps model the outcome of using materials with different heat resistance at various temperatures in a data center. Figure 3 illustrates side wall diffusers located on the right side of a data hall. It is clear that the thermal boundary layers grow over the surface of the containment, and the ceiling influences the intake temperature of the servers located at a higher height.
RISK ASSESSMENT AND CONTROL STRATEGY
It is imperative to consider and plan for possible failure components in the cooling system to prevent any IT damage or interruption. These can occur during the failure of one or more computer room air handlers (CRAH), or during a power outage; or if an unexpected recirculation occurs. There is no conventional tool to simulate these failure scenarios, but they can be modeled using CFD. CFD can predict the length of time it takes each temperature to raise to the point so that an applicable solution to the data center is realized. Figure 4 shows a failure scenario in which three CRAHs failed at the same time. In this example, Southland Industries, a national MEP building systems firm, used CFD to calculate the right amount of cooling load through increasing the air flow of the adjacent CRAHs, as opposed to intensifying the airflow of all CRAHs. This compensated for the failed CRAHs in an efficient manner and ultimately conserved energy. This figure also shows that containments were removed at different locations in the data hall. Containment removal at some locations can cause hot air recirculation and is more crucial in locations where the containment removal causes cooling load losses.
CFD also can be used to locate the appropriate locations for control sensors, or to devise a smart control strategy that balances the supply airflow with IT density in a data hall. This alleviates high velocity called the wind tunnel effect that occurs as a result of rushing the air from a lower IT density to a higher density area. Figure 5(a) highlights the zonal control strategy that balances the airflow supply based on the non-uniform IT density in the hall. Figure 5(b) shows the temperature contours at 4 ft above the floor with adjusted air supply proportional to local IT loads. Figures 5(c) and 5(d) show the comparison between the velocity contours (ft/s) in the data hall with equal air supply at each CRAH, as well as the adjusted air supply based on local IT load. The illustration shows that the high velocity region in the middle of the corridor has been alleviated in the adjusted air supply case.
PARTICLES ENTRAINMENTS
High humidity in a data center can cause condensation, corrosion, and electric shortage, while low humidity can cause an electrostatic issue that harms the system. Moreover, the entrainment of generator engine emissions or other particles into outdoor air (OA) supply can damage the computers. For these reasons, it is important to design and control the data center for the right humidity ratio. CFD can aid design engineers in the investigation of potential humidity issues inside the data center. It can also expose any particle entrainments and high humid air in the data center, ensuring that the air quality meets the design criteria. Cooling towers, emergency generators, air exhaust, and suspended particles (e.g., sand grains) are various sources of high humidity air and particles. Figure 6 shows the outside view of a mission critical facility. In this example, Southland Industries employed CFD to calculate the cooling tower water particles, generator emissions, and humidity concentration at OA under different wind speeds and directions. This verified the appropriate location of the emergency generators, cooling towers, exhaust air, and OA in the design.
COMPONENT EVALUATION
Many challenges can be encountered during the design of mission critical facilities. Manufacturers typically test and validate most of the components under specific and controlled environment conditions. CFD can be used to model the performance of the equipment including air-handling units (AHU) and the humidifier, or of a new system under different design conditions. As a result, any possible problems can be predicted and planned for in advance, which brings more confidence to the design but more importantly, efficiency and effectiveness. Figure 7 illustrates a pressurized Thermal Storage Energy (TSE) system. Southland Industries’ implementation of CFD optimized the diffusers in the tank to increase the performance by 24%. As part of the commissioning effort, the installed system was tested to the same conditions originally simulated in the CFD model. The CFD results were within 2% margin of error and saved the customer time and money on projects during the building phase.
CONCLUSION
Many factors, ranging from IT load, diffuser size, humidity, and rack size to failure scenarios, ceiling height hot spots, and many more, have an impact on the performance of mission critical facilities. In order to save energy and cut down on costs, these must be considered during the design or renovation process. The cooling system design of these facilities continues to be even more challenging when the goal is an optimized design, yet engineers push the limit to save energy and costs. CFD is a reliable solution that can produce results with accuracy. Implementing the right model in collaboration with a partner experienced in both the HVAC industry and CFD software can shorten the design procedure and optimize the design effectively. The virtual design used during this process allows owners, engineers, and architects to visualize the outcome, predict critical scenarios, and propose practical solutions prior to installation in a manner that is more accurate than conventional approaches and less expensive.
Dr. Reza Ghias is the director of Advanced Simulation Center (ASC) at Southland Industries, a national MEP building systems firm. With more than 15 years of experience conducting research and executing computational fluid dynamics (CFD) projects in a wide range of industries, he works closely with Southland’s design engineers and clients to overcome design challenges and develop innovative building systems designs. Reza has received his Ph.D. in Mechanical and Aerospace Engineering and has authored and presented many papers, articles, and technical reports proving the results of his work. He can be reached at RGhias@southlandind.com.
