Data centers consume tremendous amounts of power, on the order of 100 to 200 times the power that a typical office building consumes. In 2013, data centers consumed an estimated 91 billion kilowatt-hours (kwh) of electricity, a number that is expected to grow to 140 billion kWh by 2020. The impact of electricity costs on operations can be dramatic, especially in large data centers where a single facility may consume on the order of thousands of gigawatt (GW) hours per year. To be energy efficient and reduce wiring costs, power distribution units are located close to servers and other loads. The units receive power from the UPS at 480 V (400 V outside the United States) and distribute it at 208/230 V. While this structure results in less energy loss and lower utility bills, it distributes the risk of electrical arc flash.
Arc flash happens when electricity jumps across a gap from a conductor to ground or another phase. If sufficient current is available, the arc can quickly escalate into an explosion. In the United States, five to 10 arc flash incidents occur each day (across all industries), causing costly equipment damage and downtime. Fortunately, the majority of these events occur when workers are not present, but when they are, arc flash is known to severely injure and kill workers through burns, ruptured eardrums, collapsed lungs, and shrapnel emitted at ballistic speeds.
In commercial buildings, arc flash relays are typically mounted inside the 480 V, 3-phase switchgear in the maintenance room. They are generally not specified for other panels because the power at those panels is typically below 300 V, a value at which arc flash is unlikely to form. In contrast, data centers have 400 V electrical cabinets at many points in the data center. This puts significant voltage close to workers where human error, such as touching a test probe to the wrong surface or dropping a tool, has proven to be a dangerous source of arc flash incidents.
Best work practice requires workers to de-energize equipment before performing maintenance; in practice this is not always what happens. Data centers are under tremendous pressure to provide customers 100% uptime. Just one minute of downtime for a whole data center is estimated to cost $7,900. It is not surprising therefore that some managers may not always insist on de-energizing and lockout/tagout practices. In addition, human nature or ignorance may lead to work on energized equipment. The best way to protect workers and equipment, therefore, is to design arc flash mitigation into the electrical system.
ARC FLASH MITIGATION
There are multiple approaches to arc flash mitigation, ranging from arc-resistant switchgear to fast-acting circuit protection. Before exploring each type, it will be helpful to understand the physics of an arc flash event.
An arc flash is a type of electrical fault in which the current finds an unintentional path to ground or to another phase. Usually the arc starts out small, limited by the inherent resistance of an air gap. As the arc persists and grows, the temperature of the metal conductors rise. The metals vaporize and expand explosively, creating a cloud of ionized metal, which is a very efficient conductor. At this point the arc will pass all the current that is available to that circuit, which may be tens of thousands of amperes. The total amount of energy released during an arc flash event is directly proportional to the amount of current available to that circuit and the duration of the arc flash.
The best way to prevent arc flash danger is to minimize the available energy, which will protect both personnel and equipment. As shown in Figure 1, damage from an arc flash increases rapidly with time, and the faster the current can be shut off the less damage there will be.
The duration of the arc flash is mainly determined by the time it takes for overcurrent or ground fault protective devices to detect a fault, send a trip signal to the circuit breaker, and for the circuit breaker to subsequently disconnect the source of energy.
THE LIMITS OF STANDARD CIRCUIT PROTECTION
Fast acting fuses (known as current-limiting fuses) may disconnect the circuit from the source of energy in 8 ms or less when subjected to the high-value currents. Under short circuit conditions a current-limiting fuse can open in less than 1/2 an AC cycle (8.3 ms), which greatly limits the energy that can be delivered to the fault. In three-phase symmetrical bolted cases, other devices may take much longer to operate and remove the source of energy. But unbalanced, single-phase and high-impedance fault currents are lower than three-phase fault currents, so circuit protection devices may not necessarily detect and limit let-through current and will require more time to clear the fault.
In some cases the circuit protection devices at main distribution points are coordinated with downstream protection systems. This means that there may be considerable time delays before the standard overcurrent and ground-fault protection devices at the main distribution point operates to allow downstream protection devices to clear the faults first.
ALTERNATIVE APPROACHES TO PROTECTION
Given the limitations of a standard electrical system design, what are the alternatives? One choice is arc-resistant switchgear. Although they are more expensive, cabinets such as these reduce risk, but they won’t actually stop an arc flash from forming. The cabinet can’t protect a worker who drops a tool in the wrong place. Also it will not prevent internal damage which would lead to extended electrical outages. Another drawback is that this solution is not available for retrofit situations.
Another choice is a dedicated busbar protection scheme, such as high-impedance bus protection (based on the circulating current principle). This type of protection would typically detect a fault inside the protected zone in between 40 and 65ms (excluding the breaker operation time). The disadvantage of this solution is that it is regarded as a zone protection scheme, which means that the protected zone is limited to the actual position of the current transformers (mounted inside the switchgear). In most cases arc flash faults occur at the cable crutches, inside the cable compartment, which means that this area falls outside the busbar protection zone. The busbar protection will thus not operate for arc flash occurring inside the cable compartments.
This solution is also rather expensive, requiring complicated modifications and a number of additional equipment (e.g., current transformers, relays.) if intended to be installed on existing switchgear panels.
Some industrial plants use a resistance grounded electrical system that limits the fault current that can flow to ground fault. As most arc flash events begin as ground faults, this approach can greatly reduce the risk of arc flash. Nevertheless, high resistance grounding is not used in commercial buildings because it is incompatible with neutral wire electrical systems such as those used for lighting.
A fourth option is to install a dedicated arc flash relay. These relays use light sensors to detect the light of an emerging arc flash and send a trip signal in <1 millisecond (ms) to the main circuit breaker. The duration of the arc flash is thus reduced to the mechanical opening time of the circuit breaker. The operating time of modern circuit breakers is in the region of 30 to 50 ms.
ARC FLASH RELAYS IN DATA CENTER POWER DISTRIBUTION UNITS
By minimizing the severity of an arc flash in a power distribution unit, an arc flash relay protects workers from injury or death. By minimizing equipment damage, the relay can reduce costly data center downtime and lessen the cost of equipment repair. Also it may reduce the level of personal protective equipment (PPE) required by OSHA and NFPA 70E standards.
An arc flash relay may be installed as a separate protection scheme that operates independently of, or in conjunction with, normal overcurrent and ground-fault protection schemes.
The arc flash relay should not be connected to the breakers on the branch circuits coming from the bus bar, but rather to a breaker upstream. If an arc flash occurs on the bus bar, then tripping downstream breakers won’t help.
There are a range of arc flash relays available, from simple models to fully featured models. All are easy to retrofit into existing cabinets. Selection for a data center application depends on the features desired, cost, and cost of installation.
Selection considerations of an arc flash relay include:
Reaction time. Typically, arc flash relays see a fault and send a trip signal in the 1 to 9 ms range. The faster, the better, although some relays offer a programmable time delay to reduce nuisance tripping.
Redundant trip path. In case the primary switch fails to operate, a second switch takes over. This is useful upon restart, while the microprocessor takes time to boot.
Number of sensors. The light sensors can be either point-type (one sensor for each compartment) or a fiber-optic cable (which is sensitive over its entire length).
Event log. Software that reports the history of arc flash events.
Zone capability. Some arc flash relays allow for this natively, others require purchase of a second relay.
Current monitoring. The relay will trip only if the light sensor sees bright light in the presence of a current spike (to avoid nuisance tripping caused by a camera flash, arc welder, or direct sunlight).
Circuit breaker fail function. The arc flash relay will trip the upstream supply breaker after a time delay of 50 to 150 ms if the overcurrent and arc signal remains after the circuit breaker was supposed to open (meaning that the circuit breaker has failed to trip).
Plug-and-play installation. Some basic relays do not require the use of a laptop for configuration, and provide for simple wiring.
Option for battery backup.
Scalability and flexibility. Some arc flash relay designs allow the interconnection of multiple devices, such as multiple relays, each with several sensors.
Health indication. The point sensors blink to indicate that they are operating. This feature is valued by maintenance workers who can quickly close the cabinet doors if they see the relay or sensors are not operating.