Employing sustainable strategies is second nature for today’s commercial and institutional buildings, but as tomorrow’s data centers search for reliable ways to color themselves green, finding the right balance between initial capital cost (CAPX) and operating cost (OPX) without sacrificing reliability, will be key. The answer is three-fold.

Efficient data centers must balance the following design principals like a three-legged stool: the use of modeling tools, specifying efficient IT equipment, and employing energy-saving techniques to find the right formula for each facility.

Allstate Insurance Co.’s new 52,000-square foot (sq.ft.) flagship greenfield data center, located just 80 miles from its Northbrook, IL, headquarters went “live” in May with approximately 14,000 sq. ft. of raised floor computer room space, including racks operating at 18 kilowatts (kW). The data center was designed with balance in mind, which helped the nation’s largest publicly-held personal lines insurer decrease its own energy usage by 70 percent over its previous data centers and reach a power utilization effectiveness (PUE) of 1.39. LEED Gold certification is pending.

Modeling Tools

While an experienced engineer is surely skilled to optimize individual MEP systems performance, complete infrastructure optimization is best done through the integration of both modeling tools and field experience. From building information modeling (BIM) to computational fluid dynamics (CFD) and energy modeling, each building system has a software tool that can assist engineers in realizing previously unreachable design goals.

Initially used by architects and structural engineers in building design, BIM software has recently made significant strides in MEP systems modeling that help conserve materials and avoid conflicts prior to construction, directly reducing the project’s overall carbon footprint. CFD modeling indicates the effectiveness of air flow and heat transfer in the data center space, therefore optimizing cooling performance by adjusting IT and mechanical equipment layout appropriately.

Leading energy modeling tools include Integrated Environmental Solutions Limited (IES, www.iesve.com) and Carrier’s Hourly Analysis Program (HAP, www.commercial.carrier.com). IES assists engineers in analyzing energy modeling through visualization, weighing the benefits and cost effectiveness of different sustainable strategies. For example, daylighting may be a positive way to bring natural light into a facility, but it may also add to heat gain in the space, which will require more from the mechanical systems. IES software will determine the viability of these types of strategies. A full energy model like HAP is required by LEED for certification and brings the big picture together so that the individually optimized systems can be further optimized as a whole, comparing all sustainable strategies within the data center and creating a benchmark.

Initially, it was determined that the mechanical systems in Allstate’s data center would reach optimal efficiency with two 900-ton chillers running at partial load. However, the energy model found that a smaller chiller scheme was ideal for the application after these data were entered along with data about electrical demand. As a result, two 450-ton chillers share the cooling load equally and a third sits idle until the others reach their capacity. This cut the building’s overall electrical consumption significantly, especially during partial loading conditions. The three smaller chillers totaling 1350 tons of capacity will cost more up front than two chillers with a total of 1800 tons of capacity, but the OPX and energy savings easily provided for a short ROI of two years (while gaining other reliability benefits like faster restart times as well. For Allstate, doing something that was not intuitive for the optimization of one system in turn raised the overall efficiency of the system as a whole, thanks to full energy modeling.

Modeling tools helped Allstate achieve a lower PUE than its competitors, too. While comparable data centers industry-wide operate at typical PUE values between 1.6 and 3.0, Allstate’s data center average PUE measures 1.39 with a full real-time load at 66 F outside air temperatures (see the table). PUE rose to 1.46 when outside temperatures reached 77 F and is expected to fall below 1.32 when outside temperatures drop during Chicago’s fall, winter, and spring months, in large part due to efficient operation of the facility’s waterside economizers.

ESD used each of these tools while designing Allstate’s data center, yet it was the combination of the energy modeling and the experienced engineering skills that allowed the facility’s electrical and mechanical systems and their predicted energy savings to became a reality in the field, as modeled. For many facilities, this doesn’t happen. While modeling tools can simulate field conditions, the gap between reality and simulation continues to live on.

Efficient IT Equipment

While PUE measures MEP equipment effectiveness, IT equipment efficiency also plays a role in the overall sustainability of a data center, as reducing total energy consumption is the ultimate goal. It’s known that trimming as little as 1 kW at the load level (IT cabinet) can translate to an energy savings of as high as 2.8 kW at the utility level. This is achieved by specifying efficient IT equipment that requires less energy to perform each task. By directly lowering the electrical demand at the equipment portal, the energy savings trickles down to lower the mechanical equipment demand as well. This cascading effect means less overall mechanical and electrical losses. These efficiency benefits are often in addition to that of the PUE calculations but still make a significant difference in total energy expenditure.

Specifying higher-efficiency power supplies with Energy Star ratings and higher-performance processors are just two of the ways Allstate achieved the cascading effect for its data center. Using an ESD-custom load-profile modeling program, ESD assisted Allstate in analyzing the effect of deploying virtualized servers, furthering efficiency gains across the data center.

Virtualization of Allstate’s data center eliminated hundreds of smaller less efficient servers in exchange for larger, more efficient servers running multiple applications simultaneously and with optimized power supplies. With aggressive application to server ratio, these servers eliminate idle time and support everything from Allstate’s email system to special software created for their insurance programs. The virtualized servers also provided the right environment for high-density mechanical and electrical systems.

Energy-Saving Techniques

While modeling and efficient equipment have natural limitations, the use of energy-saving techniques is as vast as current technology and innovation. Tweaking equipment setpoints and employing free cooling, heat recovery, and monitoring systems and equipment all provide a typical data center with the opportunity to be green.

Possibly the greatest way to promote data center efficiency over time is the use of proper measurement and verification tools. Achieving the LEED certification credit for Measurement and Verification can affect a data center’s bottom line by maintaining efficiencies through the monitoring of systems and their equipment. While individual systems are typically monitored independently, overall efficiency can be maximized with the use of a building automation system (BAS) and or an electrical power monitoring system (EPMS). With tools like these, when system components suddenly fall outside of their desired energy usage range.

In order to collect real-time data facility wide, Allstate’s data center employs a consolidated BAS and EPMS system that takes measurements from all points in the system all the way down to the cabinet level and brings them back to one interface. Monitoring this information in real-time allows facilities personnel to maintain efficiency as designed and pinpoint minor issues momentarily.

Because cooling alone can represent as much as 50 percent of a data center’s total energy expenditure, significant efficiency gains can be realized by optimizing a data center’s HVAC operation. Allstate’s critical facility did this in a number of ways.

First, running redundancy with the fluid coolers, or using all three instead of two with one sitting idle, also allowed Allstate to take advantage of an extra 50 percent heat rejection surface area. Because of this, the variable-frequency drives on the fluid cooler fans were able to slow down considerably to meet the load, resulting in a significant energy savings. Furthermore, implementing this running redundancy on the fluid cooler side drastically reduced, the pressure drop through the condenser water critical flow path saving pumping horsepower. When taking the design condenser water flow for two fluid coolers and spreading it out between all three, the pressure drop through each fluid cooler was cut by more than half. The energy savings realized here is actually doubled because the same running redundancy philosophy was used on the condenser water pumps for reliability benefits. Third, since the fluid coolers now reject more heat from the system, they are able to perform with a tighter condenser water approach to ambient conditions by a few degrees, resulting in additional hours of free cooling.

Techniques such as free cooling and heat recovery have been commonly used in commercial facilities for years and are also now applicable to the data center environment. For example, during Chicago’s winter months Allstate’s data center will use its chillers only as needed. With cold enough ambient temperatures, the fluid coolers are able to produce chilled water with very little horsepower, achieving free cooling and completely bypassing the chillers with a plate-and-frame heat exchanger. Heat recovery was also used by extracting heat from the computer room floor and battery rooms and reusing it to heat the facility’s office spaces and the “back of the house” infrastructure rooms where the UPS, switchgear, and HVAC equipment are housed.

While none of the above-mentioned design principals are especially unique when standing alone, they created a new benchmark for green data centers when they rose together. The balance and accumulation of these choices positions Allstate’s data center to be one of the first LEED Gold-certified mission critical facilities, balancing its unique, one-of-a-kind accomplishments on sustainable legs of its own.

Sidebar: PUE: A True Measure Of Data Center Efficiency?

By Steven A. Monforton

Power utilization effectiveness (PUE) is a good indicator of MEP systems efficiency, but doesn’t capture the data center’s true energy performance because it fails to incorporate IT equipment expenditure. Here is the traditional equation:

PUE = (total facility power/IT equipment power)

It is essential to consider the data center as a whole in order to optimize it, as IT equipment effectiveness up front can further optimize mechanical and electrical systems. Instead, an equation that, for example, considers total energy consumed, as well as other important IT-related factors including equipment placement strategies, air flow, virtualization, and the IT equipment itself could have a much greater impact on overall efficiency. Perhaps a more appropriate metric to consider would be corporate average data center efficiency (CADE), which is used to measure the overall energy efficiency of an organization’s data centers. Here’s the new equation:

CADE = facility efficiency (FE) x asset efficiency (AE)

FE = (facility energy efficiency) x (facility utilization)

AE = (IT energy efficiency) x (IT utilization)

The Uptime Institute (www.uptimeinstitute.org) and McKinsey & Co. (www.mckinsey.com) first introduced CADE in a joint report that proposed the metric as a single key performance indicator used to compare the energy consumption of one data center against another. CADE combines measurements of the energy efficiency of both the facility and the IT equipment into a single factor. A higher CADE measurement indicates a more energy efficient data center.


Steven A. Monforton, RCDD/NTS, vice president, Technology Consulting Engineering, Environmental Systems Design, has over 25 years of experience in all aspects of technology infrastructure applications from manufacturing and industrial to commercial and data centers.