Manhattan Mission Critical
Increased demand leads to a Tier 4 data center
PLANNING FOR FUTURE GROWTH
Time Warner Cable provides internet, phone, and television service to millions of homes and businesses across the United States. To support nearly 1.5 million customers in the New York metro area, the company operates mission critical equipment out of 40 hub sites across the five boroughs of New York City and the Hudson Valley region.
Since it was established in 1989, Time Warner Cable has seen requirements for power and cooling increase dramatically due to rapidly changing technology and an ever-expanding customer base. Most of the company’s Manhattan-based hub sites were built a number of years ago — prior to high speed data, digital TV, and voiceover IP.
“Hub Site C was designed at a time when there was no such thing as high-speed data. It started with Road Runner high speed data about 10 years ago, and then the speed of high-speed increased when voiceover IP was added,” said Azzaro.
In addition to changes in technology, Time Warner Cable has needed to increase mission critical load to support their steady increase of individual fiber-based customers, accelerated by the roll out of their business class products.
To meet this increased demand, Time Warner Cable is growing the capacity of one of their most active sites, Hub Site C, by building out 14,800 sq ft of data center space on the Upper East Side of Manhattan. The new facility, dubbed “Super C,” will be three times larger than the current Hub Site C and exceed the redundancy of all other hub sites in the area.
Time Warner Cable has begun populating the data center in August of 2012, with a day-one target sometime before the end of the year. Fiber will be cut incrementally from the existing space to minimize down time and redundancy equipment will be relocated from their Hub A facility. The space will be large enough to accommodate future growth, and on day one, the team estimates that the facility will run at approximately 50% capacity and full capacity by the summer of 2013.
FROM ICE HOUSE TO DATA CENTER
To minimize latency and support its hybrid copper fiber network, Time Warner Cable’s network of hub sites must be in close proximity to their customers — typically between 20 and 30 miles. However, dense, urban environments require hub sites to be closer together and higher in number. As a result, Super C had to be located in the Sutton Place neighborhood — one of the most expensive in Manhattan — to intersect with their existing network and feed their high-density mid-Manhattan customer load.
“Putting a mission critical facility in the most expensive neighborhoods in NYC was hard to sell to senior management, and locating in that neighborhood made us want to be as efficient as possible to allow for redundancy as well as cost control,” said Azzaro.
Restricted to only a handful of possible sites, the team leased out two floors in a four-story building on the Upper East Side of Manhattan. Super C’s raised-floor data center space will occupy a 10,000-sq-ft footprint on the second floor and electrical equipment will be installed in a 4,800-sq-ft space on the first floor. The remaining two floors will be occupied by the landlord and other commercial tenants.
As with most buildings in Manhattan, Super C’s new home has seen many tenants come and go, continuously repurposed to evolve with the changing times. The structure was originally built as an ice house in the 1930s and was designed with 20-ft ceilings and 12- to 18-in. cork-cored walls. The sound-proof walls and high floor load made the building ideal for its subsequent use as a printing press. Most recently, the space was used as a parking garage before it was purchased in the summer of 2011 and converted to a commercial space. Though the original architects could not have fathomed the building’s current use, many of their design elements including the super-insulated walls, high ceilings and floor load were ideal for Super C’s proposed mechanical design.
Energy efficiency played a significant role in the design of Super C. In addition to avoiding operating costs, Time Warner Cable champions energy efficiency as a part of a wider, across-the-board sustainability initiative that includes replacing all critical lighting with LEDs and replacing older computer room air conditioning (CRAC) units with more energy efficient units. Time Warner Cable is also engaging in an ongoing project to replace all CRAC units with new refrigerants and scrolled compressors across all of their hub sites.
Time Warner Cable worked very closely with their business partners during the design phase. To help maximize energy efficient design, they partnered with the New York State Energy Research and Development Authority (NYSERDA) and their data center consultants, Willdan Energy Solutions.
“Working closely with our business partners and NYSERDA has led us down a few design paths that we otherwise haven’t gone down,” said Ryan Capone, Time Warner Cable’s senior manager of mission critical infrastructure.
In addition to technical guidance, NYSERDA will provide financial incentives for kWh avoided by Super C’s power and cooling measures.
Azzaro added, “It’s been a real team effort. We sat around table with NYSERDA, Willdan, and our business partners and asked ‘What if we do this? How about this?’ Working with NYSERDA isn’t just about capturing the money for incentives — it’s not driving factor compared to what we spend — but what they help us recoup in efficiency and redundancy is really primary to the process.”
POWER DESIGN HIGHLIGHTS
As a service provider, uptime is absolutely central to Time Warner’s success as a business and redundancy plays a central role in all mission critical decisions. In addition to the energy required for full 2N redundancy, the power requirements of IT equipment housed in Time Warner Cable’s hub sites have increased dramatically over the years.
“Five years ago, our racks were 500 to 1,000 watts per rack. Our new, dense racks require 3 to 5 kW per rack, and some are at 10 kW per rack,” said Capone. Adding to the power required by the IT equipment is the necessity for redundancy as uptime plays a key role in Time Warner Cable’s business.
To meet power demands from IT equipment and redundancy requirements, Super C will be fitted with two Schneider Electric Symmetra Megawatt II 600 kW UPS units that will improve the power quality for the IT equipment and are estimated to run up to 95% to 96% efficient. On day one, Time Warner Cable will populate each 600 kW frame to 400 kW.
In addition, DC power is derived from an Emerson, NetSure™ 2,000 amp 48v DC source, which utilizes ten 200 amp rectifier bays. Both strategies, both AC and DC, allow for rightsizing the modules needed required based on the load which leads to efficiency gains and lower operating costs. In the event of a utility power outage, two 1.5 MW diesel emergency generators will kick in. The generators, which were disassembled and then reassembled inside the building due to the size requirements of the space, will be fed by two 3,000-gal diesel tanks in the basement.
"Especially since Sandy, redundant and large fuel storage systems are a must. At full design capacity, there is enough fuel on-site to run the facility for roughly two and a half days. It's a silver lining people don't think about: a more efficient data center can run longer on the same amount of fuel," said Capone.
COOLING DESIGN HIGHLIGHTS
Super C will be cooled by a high-density, in-row cooling system comprised of a Liebert XDC160 45-ton glycol-cooled chiller module, eight Liebert XDH20 rack-mounted cooling modules, and an Energy Labs 240-ton indoor dry cooler to reject the heat to the outside. The placement of the dry cooler was restricted by the landlord who secured air-rights to the building and plans to build six floors above.
Capone noted, “Due to structural building requirements and the requirements of the landlord, we had to figure out how to install the cooler in the building with very little disturbance to the building outside.”
The 240-ton dry cooler will be fitted with a waterside economizer, eight 15-hp fans driven by variable-frequency drives (VFDs) and free cooling coils that will allow Time Warner Cable to take advantage of cool outside conditions. When outside air temperature at the site is below 55°F, the condenser water can be cooled enough by ambient conditions to reduce the use of the compressors.
“When the ambient temperature allows us to reduce glycol temperatures to a point, we can lighten or bypass the compressor process, allowing for free cooling nine months out of the year,” said Capone.
Time Warner Cable will install an in-row cooling system comprised of one 40-ton high-density chiller that feeds eight in-row units on the data center floor. The units will be placed next to high density load (up to 10 kW cooling capacity) to address individual hot spots and prevent the need to draw cooling from across room. This will allow the team to strategically plan high density load next to the XDH units to increase efficiency even more.
Six, 20-ton Liebert DS070 CRAC units with digital scrolls will cool ambient air on the main data center floor. All of the cooling units are connected to a Liebert ICOM system, which will allow the units to cycle in team work mode and automatically turn off unneeded units. Additional energy efficient measures include LED lighting which will be installed across the entire facility.
NYSERDA and Willdan estimated that Time Warner Cable will save 970,552 kWh per year over a comparable baseline case. These energy savings will earn a projected $155,288 incentive and save approximately $184,405 on their electricity bill each year (assuming $0.19/kWh price of electricity). The final incentive amount will be determined through a measurement and verification period and will be paid at a rate of $0.16 per annualized kWh saved. As Azzaro noted, “We don’t mind being the environmental heroes, but we have to pay the bills at the end of the month.”