Figure 1. Unused cabling must be removed or marked and identified for future use.


 

Afew years ago, Gary Jones, chair of the Texas Chapter and now an instructor with the International Association of Electrical Inspectors (IAEI) said, “My guys need to hear…again that just because a room is full of computer equipment doesn’t mean that the room is automatically covered by Article 645 (of the National Electric Code [NEC]). Many contractors, engineers, and users need to understand that, too.”

The confusion existed because chapters one to four of the NEC are mandatory, but chapters five to seven permit alternative wiring methods for special occupancies. Article 645 allows leniencies for power cables distributed under raised floors that would not normally be allowed in a plenum but only when certain conditions are met.

The revised scope of Article 645 of NEC-2011 makes clear that using Article 645 is “permitted” but not mandatory. Complying with the conditions of Article 645 is probably not necessary in data centers that that use overhead cabling or are not on raised floors. This small change is just one of many that were part of a complete revision of Article 645. Table 1 gives a brief summary of these changes.
 

Figure 2. Telecommunications cabling and information technology equipment can coexist in a data center but are covered by different articles of the NEC .

 

These changes to Article 645 include three significant revisions and several definitions and clarifications that have been added or revised. Article 645 now distinguishes between telecommunications and information technology equipment in the same room and alternative methods of disconnecting means are introduced.

In addition, NEC Article 645 references another document, National Fire Protection Agency (NFPA) 75 Standard for the Protection of Information Technology Equipment, four times in its “informational notes” (known in previous editions of the NEC as “fine print notes”). NFPA 75 addresses room construction for fire safety. Compliance with NFPA 75 is not mandatory in the NEC but is strongly implied. NFPA 75 likewise references NEC Article 645.
 

Table 1. Analysis of changes in Article 645.

DEFINITIONS

The new and revised definitions in Article 645 include:

  • Critical operations data systems. “Systems in which an interruption of operation could affect public safety, emergency management, national security, or business continuity.” This definition is aligned with the definitions of a “critical operations power system” in Article 708 and is significant in determining how a disconnecting means can be accomplished.
     
  • Information technology equipment (ITE). “Equipment and systems rated 600 volts or less, normally found in offices or other business establishments and similar environments classified as ordinary locations, that are used for creation and manipulation of data, voice, video, and similar signals that are not communications equipment as defined in Part 1 of Article 100 and do not process communications circuits as defined in 800.2.” Equipment that meets this definition is usually listed under UL Standard 60950-1
     
  • Remote disconnect control. “An electric device and circuit that controls a disconnecting means through a relay or equivalent device.” For example, the control is frequently a push button panel located in the ITE room or elsewhere in the building, but the actual disconnecting means might be a circuit breaker in an electrical room or on a power distribution unit (PDU). Whatever the method, firefighters use the disconnecting means in case of emergency.
     
  • Zones. “ A physically identifiable area (such as barriers or separation by distance) within an ITE room, with dedicated power and cooling systems for the information technology equipment or systems.” In the event of a fire, a zone must have the ability to remove electrical energy and prevent the spread of smoke, per other sections of the NEC. Some aisle-containment systems and “pods” might meet this definition.


Figure 3. These hot-aisle containment systems might be considered "zones" in the 2011 NEC.

ITE VS. TELECOM

Article 645 defines IT equipment as that which manipulates data, voice, video, and similar signals. Telecommunications equipment is cabling and equipment that transmits the output from the ITE. It is important to note that Chapter 8 applies to communications equipment installed in an ITE room (e.g., signal cabling), whereas Article 645 applies to the powering of such equipment (e.g., power cabling). A new section, “Other Articles,” links Article 645 to other key articles in the code and clarifies which requirements apply.
 

DISCONNECTING MEANS

Perhaps the most significant change made to Article 645 involves emergency power off (EPO) systems. In the past, ITE spaces seeking the leniencies of Article 645 have also had to install disconnecting means per 645.10. IT managers have historically viewed this requirement as a single point of failure that overrides all methods that ensure business continuity (UPS, back-up generators, redundant power paths, etc.). It required shutting down the entire room so firefighters could deal with an electrical fire of any size. Article 645 now allows some flexibility by creating two categories that can be applied to the whole room or to zones.

Ron Marts of Telcordia said, “By removing EPO as a single point of failure, these changes represent what is probably the most significant code change to affect IT reliability and availability in almost 50 years. Research has suggested that the majority of events leading to the complete shutdown of data centers was either inadvertent, accidental, or malicious shutdown via the EPO switch. Our change to Article 645 and subsequent expected changes to NFPA 75 greatly lower the risk of unwanted data center shutdowns.”
 

Figure 4. The disconnecting means might disappear in the data center .

 

The new provisions are:

  • Remote disconnect controls. Whereas a disconnecting means has previously been required at every principal exit door in the ITE room, Article 645 now permits the controls to be located at any location in the building suitable to both the authority having jurisdiction (AHJ) and the IT manager. Multiple controls are permitted for power and HVAC in one or more zones, but a single control is also permitted. For example, such controls might be located at a continuously staffed security center.
     
  • Critical operations data system. The remote disconnecting controls can be eliminated altogether if the following are met:
     
  • A procedure is established, maintained, and approved by the AHJ for removing power and air movement within the room or zone
     
  • In the event of fire, qualified personnel are continuously available to meet emergency responders and to advise them of disconnecting methods
     
  • A smoke-sensing fire detection system is in place
     
  • There is an approved fire suppression system suitable for IT equipment
     
  • Cables installed under a raised floor, other than branch circuit wiring and power cords, comply with identified sections in Article 645 and elsewhere in the code.

Tom Roberts, director of Data Center Services for Trinity Health Care, concurs with Marts, “We put millions of dollars into fault tolerant design and still suffered two crashes attributed to EPO. I have often said I will not retire until this EPO thing is resolved. Now I can finally retire.”
 

Sidebar: Reducing Risk of EPO Downtime: How It Happened

A group consisting of members of the Uptime Network and AFCOM met to make an organized effort to either eliminate EPO or to educate users on how to make EPO more reliable. They contacted the chair of NEC Code Making Panel 12 (CMP-12), Tim Croushore, and asked him to create a task group to study the problem and to make recommendations to the panel. Croushore enthusiastically agreed and expanded the scope to review the entire Article 645.

In its first face-to-face meeting, the group had to understand why the disconnecting means has been required for over 40 years. They agreed that its purpose is to allow a firefighter to contain and de-energize a fire, thereby making it a Class-A fire instead of a Class-C electrical fire. They looked at data provided by AFCOM, Uptime Institute, and other sources substantiating that more than 90 percent of EPO initiated shutdowns are unintentional. They also considered that data center operations affect communications, building, and manufacturing safety, security, and other applications that could potentially affect life safety.

APC sponsored many Task Group web meetings over several months. The TG submitted ten recommendations for alternative methods to achieve the same goals of firefighter safety. The CMP-12 accepted the proposals with minor changes, which then went out for public review in the NFPA proposal and comment cycles. There were no objections in public hearings, and the changes were implemented in the 2011 NEC.