Cisco Systems achieved an estimated savings of $120,000 per year in energy costs by simulating a data center using Future Facilities’ Virtual Facility (VF) simulation methodology. The simulation results were used to guide the placement of floor grilles, blanking panels, and hot exhaust isolation that lowered information technology (IT) equipment inlet temperatures, making it possible to raise the chilled water setpoint by 8ºF.
Cisco has several teams focusing on external energy-efficient solutions development, including engineering, professional services, technical solutions marketing, and customer advocacy teams. Cisco is also taking steps to aggressively reduce its impact as a user of resources. Cisco announced in June 2008 that it would seek to reduce its carbon footprint by 25 percent from 2007 levels by 2012. This percentage is an absolute reduction across employee travel and owned and leased real estate. Cisco's strategy for reduction involves using its own technology to meet this public commitment. Given that Cisco's footprint is almost 80 percent lab and data center, the solutions developed to support an in-house reduction will have relevance to the company’s customer's IT management in support of a green agenda.
Cisco’s Data Center Advanced Services group is responsible for helping enterprise data center managers across both facilities and IT take control of their power growth and costs and the related green implications. This group utilizes a standard methodology focused on:
-
Identifying how much power and cooling is being used and where
-
Establishing an efficiency benchmark for the current data center power and cooling design
-
Establishing operative efficiency benchmarks across compute, network, and storage systems
- Designing and implementing more efficient data-center architectures that are measured using new energy management technologies
Cisco helps users achieve higher levels of operating efficiency, not just in the original data center design but also throughout the entire life cycle.
Cisco used its Efficiency Assurance Program (EAP) to develop a standardized process. The new process provides prescriptive guidance as opposed to professorial theory. Cisco used one of its own data centers in San Jose for a demonstration. The plan for this data center was to replace volume servers with blade servers. A single rack full of blade servers provides roughly 23 times the computing capacity of a rack filled with volume servers. But blade servers require much more power-typically 20 kilowatts per rack compared to 5 kilowatts per rack for volume servers. This substantial increase in density as volume servers are converted to blade servers represents one of the industry’s greatest challenges today.
The data center occupies approximately 7,000 square feet and is filled with 3,202 units of IT equipment, drawing 770 kW. There is 1 MW of total power available and 820 kW of cooling capacity. The facility has been in operation since 1999 with limited considerations for efficient operations. The total energy bill for the facility was originally $1.4 million per year, including $660,000 per year in cooling energy costs and $707,000 per year in IT equipment energy costs.
A team led by Chris Noland, DCSTG engineering manager for Cisco, employed two techniques in parallel to improve energy efficiency. The first included a familiar set of best practices that included blanking panels and plastic curtains to prevent mixing of supply and return air. Noland’s team saw the best practices approach had two major limitations. First, best practices offer no prediction of outcome, and Noland wanted a return on investment estimate in advance to justify the required expenditure. Second, best practices address room-level efficiency issues. In most data centers, efficiency problems are as likely to be caused by thermal incompatibilities between IT equipment and cabinets as they are by flawed room designs.
The second technique is VF simulation. The VF is a detailed, 3D model that can simulate the space, power, and cooling behavior of an actual facility, including thermal interactions between the room infrastructure, cooling system, cabinets, and individual units of IT equipment. Throughout the life cycle, from initial design, construction, commissioning to day-to-day operations, VF can replace inadequate rules-of-thumb with scientific precision to manage the resiliency and efficiency of a mission-critical facility.
Noland’s team gathered all the available information on the data center, including floor plans and asset inventory lists for the IT infrastructure. They entered this information into Future Facilities’ 6SigmaDC software package. This software package provides libraries that include the most popular data center infrastructure and computing equipment. Users can simply drag objects and drop them into a VF.
The VF greatly simplifies the modeling process, while at the same time increasing the level of detail at which the data center can be modeled. This makes it possible to provide a modeling capability that is tailored specifically to the needs of owner operators such as IT managers and facilities managers without a mechanical systems background. The computational fluid dynamics (CFD) solver is designed to handle thousands of individual objects such as server blades, each with their own individual thermal and airflow characteristics.
The first value of the VF comes from simple visualization. The VF brought everything needed by mechanical engineers, electrical engineers, and facility managers into a single view that they were able to access from anywhere in the world. This capability helped everyone get on the same page and understand each other’s constraints, and it eliminated the need to physically visit the site, saving money.
The VF enabled Cisco to evaluate a wide range of options to determine what the end-state design should look like. Three areas required consideration: The first being the thermal removal capabilities of the computer room air handler (CRAH) equipment, the second the supply and distribution airflow, and the third return airflow. The VF helped Cisco’s team to see all three elements while modeling changes to both the facilities and IT infrastructure.
Each segment of the cooling path has a specific design goal and an associated set of design options. The design objective for Segment no.1 was to meet the temperature and airflow requirement for each floor grille by finding the right combination of floor height and air handler selection as well as placement of air handlers, floor grilles, and under-floor objects. The design objectives for Segments no. 2 and no. 3 were to meet the intake temperature and airflow requirements for each unit of IT equipment. Additional requirements to minimize bypass and recirculating airflow were imposed to fine-tune to increase cooling system efficiency. These requirements were met by finding the combination of cabinet design, cabinet placement, cable routing, and u-slot location most compatible with the thermal characteristics of the IT equipment to be installed.
The VF simulation predicted the airflow and thermal environment, enabling optimization of the floor void depth and pressure distribution within the void including cabling and obstructions, flow rates through the grilles, and choice of grille damping, power load distribution throughout the facility, and other aspects of the data center.
The VF analysis showed clearly that exhaust recirculation within cabinets was the pressing problem, which was causing high IT equipment inlet temperatures and the need to overcool the chilled water system. Noland found that blanking and containment curtains actually increased inlet temperatures for some units of equipment in the data center. The VF was used to guide the tactical placement of floor grilles and blanking panels that eliminated the worst of the cabinet and room-level hot spots.
The resulting 8ºF increase in chilled water set point provided a 30 percent reduction in power required for cooling and $120,000 per year in energy-cost savings. There was no decrease in equipment resilience as determined by inlet air temperature. Based on the success of this application, Cisco Systems has adopted the VF approach to maximize resilience and efficiency over time.
Input 102 at www.missioncriticalmagazine/instantproductinfo
