The use of colocation data centers by business and industry to store, manage, and exchange data has grown tremendously over the past decade and is likely to continue due to the convenience and economic advantages such facilities provide. The issues to consider when selecting a colocation data center depend upon many factors ranging from the type and criticality of data to geographical and reliability considerations.
All colocation data centers have three things in common. They all provide space, power, and environmental conditioning. What sets them apart, among other things, is reliability. According to Gross and Schuerger (2006), the term “reliability” is generally defined as the probability that a product or service will operate properly for a specified period of time under design operating conditions without failure. The reliability of a colocation data center power system can be affected by multiple factors. Curtis, 2011, notes that greater than 50% of data center downtime can be traced to human error related to the design, construction, commissioning, documentation, training, or maintenance of the facility.
Reliability requirements are not uniform. For example, a colocation data center serving an online meeting service would not necessitate the same level of reliability as that required for a 24/7 banking institution. A relatively low level of reliability or uptime might suffice for the former, perhaps 99.99%, whereas a much higher level would be required for the latter, perhaps 99.9999%.
The quality and reliability of utility power varies dramatically depending upon region and location. Aside from hurricanes, tornadoes, ice storms, lightning, and earthquakes, most major urban utilities are relatively reliable at around 99.9%, or equivalently, 8.76 hours of downtime per year per the IEEE Gold Book, 1997. Still, researching the reliability of the utility serving the data center location and the data center’s relationship with the utility is recommended as an initial step in evaluating a colocation data center.
TYPES OF COLOCATION DATA CENTERS AND ASSOCIATED POWER REQUIREMENTS
Colocation data center power systems will typically consist of a utility service, power distribution system, emergency generator, automatic transfer switch, and an uninterruptable power source. Redundancies or multiple redundancies in each of these systems results in a more reliable system overall. However, when selecting a colocation data center, usually the design, construction, commissioning, and documentation are completed leaving ongoing training and facility maintenance as the only avenue to affect reliability. A review of commissioning documents, training requirements, and maintenance schedules and documentation is essential in evaluating facility reliability.
According to Psychz.Net, 2015, colocation data centers typically provide several levels of service depending upon the facility.
Cabinets: Services range from a 1U server location in a shared rack with other customers to a private 42U lockable server rack cabinet sharing raised floor space with other tenants.
Cages: The next level of service is a secure cage with wire-mesh walls within the common raised floor area containing several server racks.
Suites: At the higher end of the cost scale is the secure suite. A suite is typically a fully enclosed space with a raised floor, solid walls, and a lockable door. A suite may share power and environmental conditioning with other tenants or have separate dedicated systems.
Modules: Colocation data centers with sufficient available space may offer data center modules at a premium cost. Modules are essentially compact, secure, self-contained data centers within the data center and contain their own cooling and power infrastructure.
COLOCATION DATA CENTER POWER CONSIDERATIONS
Some considerations for the colocation data center include an evaluation of the normal power source, emergency power source, and reliability. There is no one best solution for all applications, but an evaluation of each of these factors is critical to a tailored solution.
Normal power source: The normal power distribution system includes the electrical utility service equipment, circuit breakers, feeder conduit and conductors, distribution boards, transformers, panel boards, and surge protective equipment. Except for circuit breakers and surge protective equipment, most equipment associated with the normal power service requires minimal maintenance and have reliability ratings comparable to utilities. Circuit breakers and surge protective equipment must be serviced on a scheduled basis to insure reliability.
Location: The service equipment should be located within a secure room, separate from the emergency power equipment, and protected from outside influences such as weather or flooding. The utility power should be routed underground, and not on overhead power poles, to obtain maximum reliability. Colocation data centers with dual services fed from separate utility substations will provide the highest reliability.
Transformers and associated panelboards should be located as close to the loads as physically possible to limit voltage potential between neutral and grounding conductors.
EMERGENCY POWER SOURCES AND EQUIPMENT
Emergency power is necessary for those times when normal power is lost. We are considering generators, automatic transfer switches, and uninterruptable power sources.
Generators: Generators work in concert with UPSs to provide emergency power to a data center. Multiple configurations such as 2N (multiple generators rated for twice the load) and N+1 (multiple generators with an extra generator) exist. The more multiples (2N) or extra generators (N+1) that exist, the greater the reliability.
Fuel source(s): Generally, either diesel or natural gas is used as a fuel source. As noted in the computer-room-design article, either source can be impacted by a natural disaster which would impact the infrastructure as is the case with natural gas, or delivery as is the case with diesel.
Fuel storage capacity: The Uptime Institute, 2014, notes that the minimum fuel storage capacity for tier-defined data centers is 12 hours. Redundancy of fuel storage needs to be considered. For example, the capacity of two 24-hour tanks claiming redundancy is not 48 hours, but 24 hours.
Automatic transfer switches (ATSs): Sandberg, 1999, notes that the function of an ATS, independent of specific type and manufacturer, is to monitor utility power, and upon loss of power, automatically transfer to an alternate source of power, which could include generators or an uninterruptable power supply. Some types of ATSs are listed below:
- Open transition – Open transition switches are also known as break-before-make, and are among the most common transfer switches. An advantage to this type of transfer switch is that the sources will be fully isolated, however, transfer will require up to several seconds to complete. That is to say, for a few seconds the downstream system will be without power.
- Closed transition - Sandberg, 1999, summarized the closed transition switch, or make-before-break switch, as momentarily permitting multiple sources to supply power to the downstream system. The advantage is an increase in system reliability. Most utility companies require coordination and review of these systems since other momentary sources of power may pose potential problems to the utility grid.
- Solid state – The solid-state transfer switch is a combination of the open transition and closed transition. Utilizing solid-state technology, a solid state ATS maintains the concept of a “clean” power break but does so in terms of cycles rather than seconds.
Uninterrupted power supplies (UPSs): The website www.electricalengineeringtoolbox.com, 2017, describes UPSs as providing loads with tight voltage tolerances through use of a battery, rectifier, inverter, and static switch with no transfer time and can be equipped with a bypass switch.
- Battery – Jones, 2014, notes that although batteries provide an instant transfer of clean power, batteries account for a significant source of failure for UPS systems. There can be multiple reasons for battery failure, including limited service life, frequency of use, or a lack of maintenance.
- Rotating mass – As an alternate to batteries, Amiryar and Pullen, 2017, note that rotating mass or “flywheel” technology involves the inertia of a spinning mass providing energy to a load. While the reliability of a rotating mass may be more than a battery, the duration that a rotating mass can maintain any load may be quite limited.
Reliability: Colocation data centers rely upon multiple systems. A failure of any key non-redundant system could result in the data center power loss. Automatic, redundant, and serviceable systems; highly trained personnel; ongoing training; and scheduled proactive maintenance are the key factors to consider when selecting a reliable colocation data center.
Amiryar, M and Pullen, K. 2017. A Review of Flywheel Energy Storage System Technologies and Their Applications. School of Mathematics, Computer Science and Engineering, University of London. London.
Curtis, P. 2011. Maintaining Mission Critical Systems in a 24/7 Environment (2nd Edition). Wiley-IEEE Press. (online). https://online.vitalsource.com/#/books/9781118041635/cfi/6/18!/4/2/2/2@0:0. Retrieved August 17, 2018.
Electrical Engineering Toolbox. 2017. How UPS (Uninterruptible Power Supply) System Works. https://www.electricalengineeringtoolbox.com/2017/07/how-ups-uninterruptible-power-supply.html. Retrieved September 21, 2018.
Gross, P and Schuerger, R. 2006. Evaluating Risk. EC&M Magazine. https://www.ecmweb.com/archive/evaluating-risk . Retrieved July 23, 2018.
Heslin, K. 2014. Fuel System Design and Reliability. Uptime Institute. https://journal.uptimeinstitute.com/fuel-system-design-reliability/. Retrieved September 21, 2018.
IEEE. 1997. Gold Book: IEEE Practices for Design of Reliable Industrial and Commercial Power Systems. The Institute of Electrical and Electronics Engineers, Inc. New York.
Jones, P. 2014. Avoiding battery failure and outages. datacenterdynamics.com. https://www.datacenterdynamics.com/news/avoiding-battery-failure-and-outages/ . Retrieved September 21, 2018.
Psych.Net, 2015. Learn About Colocation Benefits And How To Get Started. Psychz Networks. https://www.psychz.net/client/kb/en/learn-about-colocation-benefits-and-how-to-get-started.html . Retrieved September 22, 2018
Sandberg, D. 1999. Automatic-transfer-switch guidelines. EC&M Magazine. https://www.ecmweb.com/cee-news-archive/automatic-transfer-switch-guidelines . Retrieved September 21, 2018.