School Uses Virtual Computing to Cool On-Campus Data Center
Fifteen years later, Bridges oversees a 1,435-sq-ft data center located in the district’s DeLand Administrative Complex that includes 38 cabinets holding more than 200 servers (and 300 virtual servers), three 15-ton CRAC units, and two 225 kVA three-phase 480-volt UPSs. At the heart of the system is a 10 gig fiber ring, with 1 gig to each site.
Getting to this point has not always been easy.
“The data center is constantly evolving, and I don’t always have control over what shows up on my doorstep,” said Bridges. “My job is to make it work, whatever it is, to support the center, which provides IT services and phone connectivity to more than 100 sites throughout the county of Volusia.”
The Volusia County School District is located in the heart of Florida’s east coast. Comprised of 16 municipalities, the district serves more than 61,000 students in 81 schools and employs 8,000 people, making it the largest employer in the county.
“Many people and systems depend on the service we provide,” said Bridges. As a result, providing cooling to the data center to counteract the heat generated by servers is critical. According to Bridges, in the event the CRAC units go down, the center has just 20 minutes before it goes thermal, disrupting service and putting data at risk. This threat explains the district’s decision to upgrade and provide additional backup to the aging air conditioning equipment in the expanding data center.
Bridges met with Larry Hood, the district’s senior construction manager, to identify the cooling needs of the center.
“At the time, we had two CRAC units, and I needed both of them running in order to provide the required cooling to the data center,” recalled Bridges. “So, there was an immediate need for a third unit to ensure the redundancy that is critical to any data center. I was also looking for a way to provide cool air more efficiently to the space.”
As is the case with most data centers, the Volusia County Schools facility relied on an underfloor air distribution (UFAD) system to deliver cool air to the space. However, the data center did not include a return air system, resulting in heat stratification at various locations within the room and forcing Bridges to keep the room as cool as possible in order to reach and eliminate.
“Not only did I not have enough cooling capacity,” she said, “but I had inappropriate cooling. I told Larry I needed a return.”
Hood addressed the challenge with the help of Wallace Stephens, principal at Parker Stephens, Inc., a mechanical consulting engineering firm in Tampa, FL. Together they devised a plan that featured a return air ductwork, one new air conditioning unit, two upgraded CRAC units, two in-floor cooling solutions — DirectAire™ and SmartAire™ — from Tate, and the improved efficiency both Bridges and Hood identified as a goal of the data center retrofit.
The solution they proposed and eventually delivered was not without challenges. Until now, the data room had not tied into the building automation system (BAS), a requirement for the new cooling system. In addition, the retrofit needed to be phased in such a way that the data center would remain cool and operational throughout the process. Finally, a tight budget meant equipment needed to be carefully selected to produce the desired results, including efficiency, energy savings, and lower operating costs.
Tying the dampers, along with other components of the data center cooling system, to the BAS was the job of Larry Odegaard, vice president and general manager of Automated Building Control Systems, Inc. (ABC) in Orlando.
“Using a BACnet communications protocol, we are able to talk with the dampers,” explained Odegaard. “As a result, Janna can look at a graphic that shows the temperature of each rack in the data center and the position of the damper associated with that rack. If a particular rack is getting too warm, she’ll get an alarm alerting her to the temperature so that she can take the appropriate steps to bring the temperature back to its setpoint.
“This was our first experience working with the dampers, and it was very positive. We were able to quickly and easily establish communications between the dampers and our building controller, which is not always the case when you work with different manufacturers’ equipment.”
According to Stephens, the bigger challenge was keeping the cooling system operational throughout the project, as the new CRAC unit was installed, the existing units refurbished, ductwork installed, and the ceiling replaced.
“The new in-floor damper includes three probes — one at floor level, one at mid-point and one at the high-point — on the outside of each rack to measure the temperature of the equipment inside the rack and cause the dampers to open and close to deliver enough air to maintain the setpoint for that particular rack,” he said. “So once the dampers were installed, and even before they were tied into the BAS, we were able to use the LED readout on each one to monitor rack temperature at any given time throughout the construction phase.
“Once the dampers were tied into the direct digital control system, Janna had remote access to temperatures from her office. And later, when we ran through testing and balancing of the CRAC units and could run only one unit, we were able to monitor the temperature of the data racks to know when we needed to stop the process, bring a second unit online and allow the space to return to setpoint. When all the probes indicated the equipment was satisfied, we took the second unit offline and proceeded with testing and balancing the single unit. That was huge. It reassured Janna, and we always knew when to stop an activity to allow the room to recover.”
Stephens elaborated on the impact the dampers have on energy savings. “As the full flow of air is delivered to the space, the static pressure in the floor plenum is monitored. As rack temperature is satisfied and the dampers close, plenum pressure increases and a signal from the static pressure sensors tells the direct digital control system to slow the CRAC unit EC fans by as much as 25%.”
And as energy is saved, so is wear and tear on the CRAC units.
But it’s really the combination of the panels and the dampers that contribute to overall energy savings.
“The panels direct the air to where it needs to go and help eliminate bypass air,” said Bridges. “As a side benefit, our technicians no longer have to wear heavy coats when they work because the air is not blowing straight up on them.”
Meanwhile, with the ability to monitor temperature rack-by-rack, thanks to the electronic dampering system, Bridges has been able to increase the temperature of the room.
“I used to keep it as cold as I possibly could — anywhere from 65°F to 68° — to address all the hot spots. Now we can raise the ambient room temperature to 75° to 77°, which represents significant energy savings and speaks to the efficiency of the system.”
Hood is just as pleased with the results. According to original estimates, the data center can expect to spend $9,918 annually on fan energy, as opposed to $124,856 for a system that employs typical floor grates and belt driven centrifugal fans. Annual energy savings should approach $115,000. If early indications mean anything, Hood predicts the system will pay for itself in energy savings faster than the 9.8 months originally estimated
Hood is also impressed with the flexibility the in-floor panels offer. “They make it easy for us to switch things out. We needed servers in a different location, so we just picked up the panel and damper and moved them to the new location. As cabinet loads change, the devices offer us the ability to automate and remotely control the cooling needs for each cabinet without having to physically re-engineer the space. I no longer have to worry about overheating. And with every new cabinet we add, we can add new floor panels and dampers, making it easy to expand the facility as needed.”
Bridges adds that the system also addresses the challenges of variable loads. “Loads vary according to time of day and rack-by-rack, so the damper gives me the ability to control airflow requirements from one rack to another or from one time of day to another.”
Meanwhile, a new return air plenum has alleviated the problem of air stratification. Air delivered at floor level to the data rack is picked up in registers above the rack and returned to the CRAC units, including a new, high-efficiency unit with variable-frequency drives (VFDs) and two refurbished units that also include VFDs, enabling them to ramp up or slow down fan energy. The installation of two new UPSs rounds out the retrofit.
“At some point, we’re going to outgrow this space and have to build a new data center from the ground up,” said Bridges “When that happens, the floor panels and dampers will be part of my plans.”