Much has been made about how demand for data center space will continue to outpace supply. So how do data center operators meet the needs of not only today’s clients but those of future clients in the coming years?
Phoenix NAP put a lot of careful thought and planning into all aspects of the project, from its design and build to its commissioning and operation. As a result, it took some different approaches than are typically found in multi-tenant data centers, all the while keeping focus on client wants, needs, and experiences.
Design and Build
Data centers need to be located in areas with available and accessible power and a solid network infrastructure complete with fiber from multiple carriers. It is also tremendously beneficial when these data centers are in locations well suited for disaster recovery or business continuity services. Phoenix is one such area, since it is not prone to natural disasters or catastrophes. In fact, in its most recent ranking SustainLane recognized Phoenix as one of the U.S. cities safest from natural disasters (http://www.sustainlane.com/us-city-rankings/categories/natural-disaster-risk).
However, a safe area means nothing without the power, cooling, and network connectivity required for a mission critical data center.
Power
When it comes to power and data centers, the common metric to reference is PUE (power usage effectiveness), and the engineered PUE for Phoenix NAP is sub-1.5. As a multi-tenant facility, Phoenix NAP takes a conservative approach to PUE, so as to not limit its clients’ choices of server equipment or desired hosting configurations. By leveraging its design principles and infrastructure, Phoenix NAP is able to provide flexible colocation solutions, all the while stating a sub-1.5 PUE in all of its contracts.
Phoenix NAP accomplished this goal by asking engineers and designers to develop a power distribution method that eliminates common single points of failure and includes equipment and design choices that enhance reliability and overall energy efficiency.
Starting with two utility transformers at a substation roughly one mile away from the facility, four 10-megawatt (MW) feeds deliver power to Phoenix NAP along two different paths. These paths never meet until the client racks, ensuring true A and B power all the way through. Phoenix NAP makes its power map readily available to illustrate this configuration. Additionally, parts of the power distribution - namely the UPS systems - are unique to multi-tenant data centers.
Phoenix NAP incorporates the S&C PureWave UPS system in its design.
Unlike more traditional systems, the PureWave units operate at medium voltage and are specifically built at larger-capacities (5 MW megawatts or greater) to support today’s power requirements. “As computer room densities continue to increase, the S&C PureWave UPS is positioned to handle the higher loads of mission critical data centers through its unique designs,” said Brad Roberts, S&C power quality systems director.
The PureWave UPS systems employed at Phoenix NAP are positioned in a secure equipment yard, freeing valuable data center space and strongly contributing to overall energy efficiency. Coupled with eight two-megawatt N+1 generators, the power distribution designs (and backup systems) are optimized for performance.
Cooling
Considering a data center is filled with servers, effective and efficient cooling is a huge priority. When Phoenix NAP examined what would be necessary to meet its current and future demands, the one setup that kept coming up involved McQuay Modular Central Plants (MCP). The modular setup means that the failure of one component doesn’t bring down the whole system. Going one step further, Phoenix NAP worked with McQuay International's Solutions Group to customize its MCPs to add more redundancies. The result was a product that McQuay states surpasses anything else they have available for that type of packaged chiller plant.
Designed for mission-critical facilities, McQuay’s MCPs feature removable walls and roofs and include multiple components in one pre-engineered and pre-assembled package. They have dual chilled water and condenser water pumps, independent power supplies, and controls to keep each module running separately in the event of mechanical or electrical disturbances. Their modular design allows for lower mean time to repair and rapid expansion.
“The main feature of our MCP is the modular concept which enables facilities to expand as necessary without the costs of typical brick-and-mortar construction. We shorten the on-site construction time and pre-commission certain components to ensure quick start-up on-site,” said Terry Young, general manager of the North American Solutions Group at McQuay International.
Of course, there is more to cooling a data center than modular central plants, and for Phoenix NAP, a bi-directional closed water loop and 80-ton CRAH units that are N+4 for each zone they are in are key contributors to facility cooling. The closed loop design includes valves, which means maintenance can be performed on any section(s) of the loop without taking the whole thing down. This concurrent maintainability of the system helps Phoenix NAP operate efficiently to meet mission critical needs.
Network
The “NAP” in Phoenix NAP stands for network access point, and a major goal of the data center is to be just that for the Phoenix and southwestern United States regions.
There are several network hubs around the country (Los Angeles, Dallas, Chicago, and New York City) that carriers backhaul traffic to when connecting site visitors in Phoenix (and other areas). This process, while effective, is both costly and operationally inefficient. To alleviate this situation, Phoenix NAP is administering a public peering exchange, on a not-for-profit basis.
Rather than charge network carriers a recurring monthly fee to have fiber in the facility, Phoenix NAP is instituting a one-time non-recurring cost (NRC) when carriers build in.
The NRC eliminates financial impacts of monthly fees for carriers, and the more carriers in the data center, the greater the opportunities are for peering agreements that will limit carriers’ needs to backhaul. With an increase in carrier peering, data center clients can enjoy additional connectivity options.
This isn’t the only thing Phoenix NAP is doing a bit differently regarding connectivity. At the edge of its property, two dedicated meet-me vaults allow fiber to run along completely diverse paths into the facility. Additionally, a highly secure 3,000 square foot meet-me room (MMR) is complemented with a highly secure cross-connect room (CCR) to give carriers their own dedicated space. This design encourages traffic peering and lets clients connect to a variety of carriers with little delay. If a client wants to add an additional carrier, a simple three-foot fiber from the CCR is run up to ladder racking and the connection is made. Phoenix NAP is also located along the major fiber trail in the city, and as connectivity demands grow, the facility can expand with a second CCR and/or second MMR.
Security
Equally important to clients, facility operators, and network carriers, the security systems in place at a data center must be visibly present and highly effective. Stressing a mixture of technology and on-site personnel, Phoenix NAP security operates on the premise of least-privilege logical access control. Access is not over-provisioned, and a multi-layer approach is used. A hosting services engineer might have access to the MMR, but a hosting services technician might not.
In addition to 24/7/365 on-site security guards, Phoenix NAP also leverages some different technologies to monitor and control accessibility.
Once visitors are buzzed through the locked front door, they sign in with a security guard, at which point they are issued a security badge with a temporary profile created using some general information from their government ID (name, address). Visitors also will receive a sticky label that has their photo on it.
After this initial security check in, visitors are accompanied through a secure door and proceed toward the mantrap. The mantrap includes door detectives that allow for only one person to pass through them at a time, causing an alarm if there is any sort of abnormal activity (multiple people entering, tailgating, walking through and back).
Additionally, the mantrap utilizes three-factor authentication, which includes a badge reader, PIN pad, and iris scanner. Phoenix NAP has 100 percent accountability for everyone on its data center floor at all times.
Some other significant features of Phoenix NAP security systems are the advanced video analytics, the two-man rule for the MMR (one person can never be alone inside it), and the general design of the meet-me and cross-connect rooms. Those areas are not accessible to anyone but Phoenix NAP staff and network carrier technicians. In fact, the cross-connect room is accessible to Phoenix NAP staff only.
Commissioning
A good design is not the only thing that is important in building a mission critical data center, the integration and commissioning process can be equally critical. The first contractor Phoenix NAP signed to the project was its commissioning agent, ZDT Group.
Starting with the development of its owners project requirement (OPR), Phoenix NAP staff created its basis of design (BoD), all the while working closely with its commissioning agent. “Phoenix NAP fully understands the importance of a properly integrated and commissioned facility and for that reason we were brought into the project at the beginning. This allowed us to address reliability, efficiency, and maintenance concerns while still in the design phase, and help create a data center that is mission critical,” said Joe Soroka, president of ZDT Group.
Operation
Phoenix NAP placed a lot of emphasis on its design and build to develop a facility to meet today’s and tomorrow’s demands. That also included client experience. When clients come to Phoenix NAP, they can see first-hand the inside of a modular central plant, witness the least privilege logical access control security measures, work out of dedicated office space, set up their own network operation center (NOC), and take advantage of amenities that include daily catered lunches and dinners. This may not be the typical approach to running a facility, but to be the best next generation data center, it takes the required power, cooling, network, and security configurations, along with an excellent client experience.
