Data centers are able to function thanks to the electrical power equipment humming and whirring throughout them. The equipment found in a data center is essential for functionality and requires regular maintenance for optimal performance. Habitual servicing and maintenance is crucial for long-term performance and smooth data center power system operation.
An electrical preventative maintenance (EPM) program is the first and strongest defense against premature power system repair and replacement. As a result, data center managers can improve reliability for large-scale data centers and also more efficiently perform maintenance, thus reducing energy and operational costs.
In an ideal world, the initial design of a power system would account for loads serviced in the facility and ease of maintenance. However, owners are often left to deal with systems that have been value-engineered and maintenance is extremely difficult to schedule and implement. There are several initial design opportunities to make scheduling and implementing maintenance in data centers easier. These steps include applying tie breakers and locating all personnel and test equipment in the same section of the facility.
EFFECTIVE DATA CENTER EPM PROGRAM
There are three parts to any EPM program: inspection, testing, and repair. Each part is vital to the overall success of service and maintenance procedures. It is also important to tailor each part of an EPM program around the operations and critical nature of the specific data center facility and the equipment therein.
Before an inspection can begin, the owner should gather up-to-date drawings and diagrams, along with applicable manufacturers’ operation and maintenance manuals for the equipment. Without these materials the inspection and resulting data could be deemed incomplete, inaccurate, or unsafe. This information helps the inspector to determine where maintenance priorities reside in the data center environment and how best to implement lock-out/tag-out before testing begins.
Inspectors should be sure that all equipment has been shut-down and verified as “dead,” all energy sources have been locked-out/tagged-out, and proper grounds have been applied before beginning. This type of inspection should be planned in detail prior to equipment shut-down to avoid wasted downtime, and only personnel who have been properly trained on the potential hazards should perform it. The areas of interest for inspection will be unique for each type of equipment in the facility, and inspectors should frequently consult equipment manufacturers’ operation and maintenance manuals for guidance as to key areas of concentration.
A thorough inspection should produce accurate and complete records. These results should lead to repairs, which should be immediately scheduled, prioritized for equipment that is either near failure or indispensable to a critical load. Without these processes, planning repairs can become extremely difficult. Also, data center managers should keep viable records of past inspections on hand to make future troubleshooting efforts more effective.
CIRCUIT BREAKERS TO PROTECTIVE RELAYS
Understanding the maintenance requirements of each type of electrical equipment is key for effective power equipment maintenance in a data center. Most equipment manufacturers recommend annual testing and maintenance in a clean environment and more frequent testing in a hot and dirty atmosphere.
Medium-voltage transformers are the most critical components in a data center’s power system. Failure can cause drastic service interruptions, and replacing this type of transformer may take weeks or even months. Visual inspection includes monitoring load current, voltage, liquid level, liquid temperature, winding temperature, and ambient temperature.
During a shut-down additional testing should take place, including insulation tests such as power-factor testing and insulation-resistance testing. Diagnostic tests to take at this time include turns-ratio testing and exciting-current. The inspection should also include examination of fans, relays, control power, and control power wiring, along with the general condition of the enclosure.
In some units, inspectors should also look for fluid leaks. Fluid should never be sampled while a transformer is energized, unless an external sampling valve is present. To produce the best test results, inspectors should track trends over a period of time.
Dirt and debris should be removed from the core, coil and insulators of ventilated and cast-coil dry-type ANSI/IEEE C57.12.01 medium-voltage transformers. Inspectors should ensure that airflow is not impeded and that connections are tightened. Insulation resistance testing, turns-ratio testing, and core Megger tests should also take place.
Many data centers have NEMA low-voltage dry-type transformers for lighting applications. The frequency of inspection is based on the operating conditions. Clean and dry conditions require less maintenance, while dusty, hot environments will necessitate more frequent inspections. During maintenance, inspectors should check connections.
ANSI/IEEE C37.20.2 metal-clad switch gear distributes power, contains circuit breakers, and provides overcurrent protection. The construction varies widely based on manufacturer and age, so accurate drawings and documentation are essential to maintenance. Moisture, combined with dirt, will degrade insulation in the equipment at a high rate. Therefore, manufacturers recommend major maintenance at least every three to six years.
During a shut-down, inspectors should inspect, clean, and tighten bus bars, examine supports and insulating barriers, thoroughly vacuum the interior, and verify that ventilation is adequate.
After withdrawing the circuit breaker(s) for inspection, facilities personnel should check moving mechanisms in the compartment. They should also inspect, tighten, and lubricate shutters, VT, and CPT compartments as needed.
Like medium-voltage circuit breakers, low-voltage power breakers should be completely withdrawn from their cubicles before being maintained. The primary drawout contact clusters may show signs of wear or damage. Dielectric tests (hi-pot or insulation resistance) can help check insulation.
Unlike medium-voltage breakers, low-voltage breakers usually have an integral trip unit. There are two main ways to test the trip unit: primary and secondary injection. Manufacturers usually recommend the primary injection method, since it tests the complete system, including the current sensors, while secondary injection only tests the trip unit itself. Primary injection requires a large test set that could need single-phase 480 V for operation. Planning for this power is necessary prior to any shut-down.
Insulated case circuit breakers can sometimes be maintained in the same way as power circuit breakers. However, in many cases, their maintenance may more closely resemble those for molded case breakers. Maintenance is limited to inspection for good operating conditions and, in some cases, primary injection testing with the circuit breaker removed from the enclosure. Insulated case circuit breakers should be manually operated annually, using the “push to trip” button to exercise the tripping mechanism.
PROTECTIVE RELAYS
Protective relays utilize inputs from CTs and PTs, along with their characteristic curves and settings, to provide protection to cables, transformers, motors, and other equipment within the power system. Relays will shunt trip their associated medium-voltage circuit breakers if they detect an over current or other anomaly.
The two types of relays are electronic and electro-mechanical. Electronic relays are the most common. However, harsh operating conditions could dramatically reduce their lifespan. Once an electronic relay has been commissioned, it should be maintenance-free unless future setting changes are needed.
In most cases, ANSI/IEEE C37.20.3 medium-voltage metal-enclosed switch gear is located at the primary side of unit substations in data centers. These units can also act as the main switch gear, but their primary functions are to protect downstream loads and to provide a disconnect for lock-out/tag-out. During a shut-down, inspectors should perform a hi-pot (dielectric) test to measure insulation integrity along with general cleaning and tightening of the insulators, bussing, and mechanical parts.
Switch boards and panel boards are built per UL 489 and are designed for low-voltage applications. Containing either molded-case circuit breakers or fusible switches, they are typically located downstream of the switch gear in a data center. Switch board and panel board maintenance is mainly limited to ensuring good electrical and mechanical connections plus good housekeeping. Manufacturers recommend regular infrared scanning and cleaning for continued dependable operation.
UL 845 low-voltage motor control centers (MCCs) are the most versatile components in a power system. These units are designed to contain starters for motors and are located within buckets or control units in the enclosure. The buckets can also contain drives, simply molded case circuit breakers, or fusible switches. Maintenance on the enclosure and bus bars of MCCs is very similar to switch gear, the major difference being the drawout buckets, which should be removed from the MCC before maintenance.
Once removed, the primary contact stabs should be examined for signs of arcing or overheating. A badly pitted stab may necessitate replacing the vertical bus within the MCC. Within the bucket, any circuit breakers, switches, starters, control devices and wiring should be examined.
Automatic transfer switches (ATS) are critical in data center power systems. They switch downstream loads between power sources in the event of source failure. Both types of ATSs, standard and bypass-isolation, can be open- or closed-transition. Bypass-isolation ATSs allow switches to be racked out for inspection, testing, and maintenance without any interruption of load power. Facilities personnel should check the controller/switch combination of a bypass-isolation ATS frequently. One manufacturer recommends a monthly test to verify the proper electrical operation of the controller and switch.
