An international global technology services company supporting two major automotive manufactures and their affiliate businesses in the North and South American markets has an existing data center in southern California that was reaching the maximum limits of physical space, electrical, and cooling capacities. To sustain operations, the company planned to expand the data center facility to ensure it would operate effectively and efficiently in the present location for 10 or more years.
The company sought to partner with an established, experienced, and innovative supplier to design and build the new section of their data center and update the current space using industry best practices, enhancing operational efficiency wherever possible.
The plan to build the new section included converting office space to expand the physical data center space within the confines of the overall structure. The existing site had significant limitations for placement of electrical and cooling infrastructure. The data center is located in a high-profile corporate campus office facility and the new cooling towers and standby generators could not be visible, meaning that the support infrastructure had to fit within the existing space.
In addition to these limitations, the existing data center, which every day supports millions of dollars of revenue for the company, had to remain active throughout the entire construction period. The design and construction team needed risk management processes in place to minimize impact to the data center operations during construction. Put another way: construction of the data center would be like a changing a flat tire while driving down the freeway.
This case study analyzes project to design and build scalable electrical and mechanical systems that could be expanded in the future with minimal impact to facility and operations as the hardware load grows. A robust design/build approach to the project ensured that the flexible scalable system was installed while the existing computer systems were left in place, the data center remained operational with no downtime, and the project stayed within budget.
The data center operations team had been managing the existing center for 10 years, doing an excellent job of getting every possible watt or Btu of cooling out of the facility. Because they were aware of the eventual need for a major remodel, they worked with their management team to develop a detailed list of requirements for the data center. This formed the foundation of all data center planning, defined the needs and requirements for the facility in the long term, and was imperative for gaining senior management and budget approval.
With large data center projects, there is never enough capital to get everything desired. Therefore, to ensure the data center would meet the client’s overall requirements, it was critical to create several different design topologies and explore the capital expenditures (CAPEX) and operational expenses (OPEX) of each. Senior management approved the design that used scalable, modular infrastructure. This scalable infrastructure allowed for the design of electrical and mechanical systems that could be expanded in the future with minimal impact to facility and operations as the hardware load grows. In other words, to reduce initial CAPEX, the company only needed to invest what was necessary to support the business forecast plus a little more.
Before construction began, the provider and company spent a tremendous amount of time planning, to ensure the design met the developed requirements and the existing data center remained operational. Using the design/build methodology, the construction team controls both the design process and the buildout. This eliminates redundancies and misunderstandings between the engineering and contracting teams and enables the solution provider to closely oversee all subcontractors and construction activities, monitor risk management, quality control and job site safety plans, and observe the data center operations and stay on top of room temperature and power related issues.
There were several significant design issues to address when creating the updated scalable electrical and mechanical systems for this client:
- The data center had to remain open during construction.
- The new exterior electrical and mechanical systems had to be built within the existing footprint.
- The existing raised floor could not be removed.
- The electrical infrastructure had to be updated.
The most significant of the design issues was the requirement that the facility remain operational during construction. The project included complete replacement of both the electrical and mechanical infrastructure, as they were reaching end of life and could not be properly integrated into the new design. In addition, the existing infrastructure did not meet new energy standards for data centers in California, requiring energy saving economizers for mechanical systems and highly efficient electrical system designs to minimize energy losses. To keep the facility operational, the existing electrical and mechanical systems were protected in place while new systems were constructed and commissioned.
In addition to ensuring that the facility remain open during construction, the client required that the new mechanical systems fit into the footprint of the existing systems. Because these systems are large and require a significant amount of space, most data center mechanical heat rejection systems are located outdoors to reject the heat from the computer hardware. In this project, the new design required tripling the capacity of the cooling systems — in the same footprint as the existing systems, while they remained operational. To accomplish this, the new systems were constructed on a steel platform above the existing equipment. Once the new mechanical systems were operational, the older systems were removed, and the vacated space was used for standby generators
The existing data center used legacy 24-in. high raised access flooring with downflow cooling units connected to cooling towers. This existing design limited both the supply and return airflow because there were no rack air containment systems and the underfloor area was congested with power whips. A chilled water based in-row cooling solution using a cold aisle containment system solved the cooling problems and enabled increased equipment rack power density, while leaving the existing raised floor system in place. With the new cold aisle containment system, the client can closely monitor and raise the inlet temperature to the server equipment, which improves the overall efficiency of the system.
The existing electrical infrastructure was not adequate for the projected load density of the IT equipment racks. The existing electrical feeders supporting the data center were undersized and, as a result, the data center was experiencing a significant voltage drop. To address this, the local power company installed a new utility service dedicated to the data center. The existing backup generators and uninterruptible power supply (UPS) systems were replaced using larger, more efficient and scalable technology. The design topology of the electrical system was upgraded to a 2N distributed redundant design to eliminate single points of failure and make the system more maintainable.
The electrical system was reconfigured so that overhead busway systems connected to step down transformers located in the power rooms distributed UPS power to the equipment racks. Though the busway systems cost more than traditional power distribution units (PDU) and remote power panels (RPP) with power whips, there were a couple of advantages. Overhead busways left the existing underfloor power whips in place until the new systems were put online and it was possible to completely commission the new busway prior to transferring the critical load to the new system. When it was time to switch to the new system, the existing dual powered equipment racks were transferred circuit by circuit to the new busway without any interruption of power.
Working in an operational data center is a major challenge and should only be performed by construction teams experienced working in live mission critical facilities. Effectively completing a project like this requires a lengthy planning, budget and design phase to allow the client company to identify risks to data center operations and select the best possible design to meet their corporate objectives.
This project was delivered on-time, on-budget, and without issuance of any change orders. The one stop shopping of hiring a design/build provider for data centers minimizes risk for the client company and provides a smoother process for the project. The key advantages are:
· Budget management: Clients avoid the burden and cost overruns of change orders caused by misunderstandings between the designers and the contractors.
· Single source of responsibility: Design/build contracts eliminate questions about accountability.
· Improved communications: Because the design professionals work under the direction of the construction team, there are fewer misunderstandings and change orders, and improved communications with the client company.
· Faster completion: Because the same provider designs the project, it is possible to pre-order equipment during that phase to eliminate delays during the buildout.
· Quality control: The design/build method protects the client company’s interests and helps to remove ambiguity that may arise in material and construction specifications because the designer, engineer and builder are from the same firm.