Afunny thing happened on the way to the forum ... for better power reliability. Monitoring and controlling that power started taking center stage.
That's because reliability-based design, reliability centered maintenance, and failure prevention depend on gathering, analyzing, and acting on data from critical power generation and distribution components and systems.
Everyone involved in designing, constructing, commissioning, and maintaining a building — consulting engineers, contractors, and building owners and managers — has a stake in optimizing power system monitoring and control. They all 'win' when it works and they all lose when it doesn't.
Data center and health care facility executives, especially, crave more power control information. One reason is that power systems are more complex and sophisticated than ever, and could mean the difference between “life or death” to an organization's operations.
Data center downtime, for example, costs business more than $5,000 per minute, according to a 2011 Ponemon Institute study of U.S.-based data centers. With an average reported incident length of 90 minutes, that's nearly a half-million dollars on the line ... or off the bottom line.
Complex power systems are vulnerable to problems that can undermine the very power reliability they're designed to provide. Sophisticated power monitoring and control help ferret out potential problems and provide a raft of benefits that can extend throughout an organization.
But unfortunately, many facility executives are unfamiliar with current monitoring and control technologies and best practices. According to a nationwide survey of facilities managers, only 24% have a single best-practices system that monitors, controls, and provides reporting and power quality analytics for the emergency/backup power system. In fact, the most common control capability in systems today is stop/start, but 40% of respondents indicated their systems lack that capability.
The starting point for evaluating and selecting sophisticated power monitoring and control systems is for the facility executive to pinpoint information needs and thoroughly understand the business’ operational processes.
“The needs assessment helps determine the level of system required and involves also understanding who’s watching the system, what happens when an event happens, and how well equipped the facility is to respond to that event,” said Caroline Fenlon-Harding, senior vice president of WSP.
Other aspects that ought to be evaluated include system flexibility and working relationships with vendors. Facility executives should probe to find out how a vendor will handle any issues that may come up.
“The company relationship is almost as important as the products themselves,” says Gary Coley, senior real estate manager for Digital Realty in Arizona. Digital Realty monitors and controls emergency power at 116 mission critical buildings worldwide.
Four monitoring and control technologies are usually considered in the assessment: legacy supervisory control and data acquisition (SCADA) systems and building management systems (BMS), and two new technologies — data center infrastructure management (DCIM) and the critical power management system, or CPMS.
Another way to look at them is that the first three are overarching technologies. They aim to monitor and control an entire facility or campus, including critical power. The fourth dedicates itself to controlling only critical power generation and distribution systems.
The four technologies have similarities and differences that are important to consulting engineers, contractors, and building owners and facilities executives. This article describes each technology, highlights their capabilities and limitations, and suggests which system may be best suited for a given application.
Technically, all systems designed to monitor and control business operations and processes are SCADA systems. This discussion addresses legacy SCADA systems meant for a variety of industrial, commercial, and institutional applications. Telecommunications, power utilities, water and waste control, energy, oil and gas refining, and transportation have historically applied SCADA.
They help improve efficiency and operational reliability, and lower costs, thus increasing profitability of operations and processes, and enhance worker safety.
Best-in-class SCADA provide alarm handling, trending, diagnostics, maintenance scheduling, logistics management, detailed schematics for a particular sensor or machine, and expert-system troubleshooting guides. For alarm handling, though, a cascade of quick alarm events could 'hide' the underlying cause.
Third-generation SCADA systems include a computer and open, or “off-the-shelf,” system architecture that acquire data from, and send commands to, monitored equipment, a human-machine interface, usually a computer monitor screen, a networked communication infrastructure, sensors and control relays, remote terminals units (RTU), and programmable logic controllers (PLC).
As a quick aside, the range of available RTUs and PLCs require careful consideration to ensure the classes of equipment selected will provide needed scalability, optimize functionality, and prove most cost effective for a given SCADA application.
Standard protocols and Internet accessibility of networked SCADA systems make the systems susceptible to remote attack. In April 2008, the Commission to Assess the Threat to the United States from Electromagnetic Pulse (EMP) Attack issued a “Critical Infrastructures Report” that concluded, "SCADA systems are vulnerable to EMP insult. The large numbers and widespread reliance on such systems by all of the nation’s critical infrastructures represent a systemic threat to their continued operation following an EMP event. Additionally, the necessity to reboot, repair, or replace large numbers of geographically widely dispersed systems will considerably impede the nation’s recovery from such an assault."
Additionally, in June 2010, an anti-virus security company reported the first detection of malware — Stuxnet — that attacks SCADA systems running on Windows operating systems. It steals design and control files, changes the control system, and hides those changes.
SCADA and control product vendors have developed specialized industrial firewall and virtual private network products for TCP/IP-based SCADA networks.
A BMS controls, monitors, optimizes, and reports on such mechanical and electrical equipment as air handling and cooling systems, lighting, power systems, fire systems, and security systems.
BMS comprises software and hardware similar to that of SCADA. Software can be either proprietary, using such protocols as C-bus, Profibus, etc., or open architecture that integrates internet protocols and open standards like XML, BacNet, Lon, and Modbus.
Basic controls include manual switching, time clocks, or temperature switches that provide the on and off signals for enabling pumps, fans, or valves.
Unlike other monitoring and controls systems, BMS enables two-way communication between building and property managers and their employees, tenants, or residents. Two-way communication is a valued capability for hospitals and office buildings since both those types of facilities need to maintain a comfortable environment and in the process, save energy. Systems linked to a BMS typically represent 70% of a building's energy usage, including lighting. It also tracks and schedules building maintenance.
The Bryan Medical Center East Campus in Lincoln, NE, for example, employs a BMS to maintain temperature and other environmental conditions for patients, visitors, and staff.
BMS also can play a role in protecting the critical power system. Geisinger Medical Center monitors crucial power generators through both its BMS and its security system.
“We are monitoring emergency power at both locations 24 hours daily,” explains Al Neuner, Geisinger's vice president of facilities operations. “So, if one misses the alarm, the other location will catch it before we experience power problems.”
But, some say the functionality offered by a legacy BMS does not include the software tools needed to manage mission critical operations and processes. More than basic alarm and control notification are required.
In addition, BMS may not distinguish between critical and non-critical monitoring. The same technology manages temperature of both offices and data center hot aisles.
Also, data transfer between critical power equipment occurs at speeds and bandwidths that could incapacitate most BMS. Power quality data such as wave form capture or transient harmonic displays are a case in point.
BMS need to be sophisticated enough to import crucial operational data from power controls. “The building automation system should allow a one-line diagram of the emergency backup power system,” said Robert McCarthy, senior associate with Environmental Systems Design.
Gartner defines DCIM as "tools that monitor, measure, manage, and/or control data center use and energy consumption of all IT-related equipment (such as servers, storage, and network switches), and facilities infrastructure components (such as power distribution units, and computer room air conditioners)."
Said another way, it manages energy, assets, availability, risk, services, the supply chain, and IT automation by acquiring data using SNMP (simple network management protocol), BACnet, or Modbus.
As the relatively new monitoring and control technology continues evolving, it seems the larger the data center, the greater the need for DCIM. Well-known internet service providers, search engine entities, and upcoming exascale computing centers have particular need for DCIM's specialized capabilities.
As a system, DCIM can encompass specialized 3-D visualization software, hardware, and sensors to monitor and control all IT and facilities infrastructure equipment in real time. It automates three primary functions: data collection, infrastructure modeling, and analytical reporting.
What's driving DCIM? Greater power and heat densities, growing virtualization and cloud computing, increased dependence on critical IT systems, demand for energy efficiency, and achieving green IT initiatives.
At its best, DCIM produces improved uptime, efficient capacity planning and management, valuable business analytics, and deeper process and change management. However, the relationship between IT and facilities infrastructure managements and the equipment they manage need to continue evolving to realize all its promise.
Like BMS, data center infrastructure management systems need to be sophisticated enough to import crucial operational data from power controls to effectively monitor and control critical power systems.
As it stands, the majority of this data transfer occurs at speeds and bandwidths that would incapacitate most DCIM systems. Power quality data such as wave form capture or transient harmonic displays are a case in point.
“At that level of technology, there are no standardized platforms or protocols like Modbus or BACnet," said Morris Toporek, senior vice president for Environmental Systems Design. "The facility executive is tied to vendor proprietary software to accomplish sophisticated analysis.”
According to 451 Research, “DCIM systems today mostly look at the present status of the data center, for the purpose of improving operational efficiency and availability. But data center managers must also look forward — some of their biggest challenges are in avoiding huge cost overruns by over-provisioning, and avoiding becoming constrained operationally by a shortage of power, cooling, or space.”
According to Forrester, it isn't extreme scenarios like terrorism or weather that tend to disrupt business; in fact, power failure is the most common cause of downtime.
Compared to legacy SCADA and BMS, and emerging DCIM, CPMS monitoring and control capabilities are an apple to their oranges. Rather than being all things to all infrastructure systems, CPMS monitoring and controls are dedicated to managing critical power generation and distribution. These high-end power controls are proprietary or semi-proprietary solutions, running on their own dedicated, independent backbone.
They typically work in tandem with a SCADA, BMS, or DCIM, providing the needed sophistication, speed, and analytics that are specific to power generation and distribution. Bryan Medical Center, mentioned earlier, depends on the seamless exchange of information between its CPMS and BMS.
CPMS typically oversee gen-sets, circuit breakers, transfer switches, bus bar, paralleling control switchgear, uninterruptible power systems, and other critical power distribution equipment.
They watch normal and emergency voltages and frequency, indicate transfer switch position, source availability, normal and emergency voltage and frequency, current, power, and power factor; and display transfer switch event logs, time-delay settings, rating, and identification. They facilitate critical power system load management, bus bar optimization, testing, maintenance, reporting, trending, and analytics. The bottom line: ensure power reliability during surges, sags, and outages.
Their reporting capabilities help health care facilities comply with NFPA 70, NFPA 99, and NFPA 110 requirements for hospitals, as well as Joint Commission reporting requirements for maintaining accreditation. They also satisfy specific requirements for NEC Article 708-mandated Critical Operations Power Systems (COPS).
“The control and monitoring systems supporting COPS must be designed to provide a high degree of reliability and resiliency. For COPS, the design of the control and monitoring systems would need to include physical security and protection, fully selective coordination, surge protection, and other specialized requirements,” said Fenlon-Harding.
A dedicated and fully integrated power monitoring system helps data centers and telecommunications sites satisfy National Electrical Code requirements and EN50160 Power Quality Compliance. Some server farms, notably those using Sun Microsystems, have power factor requirements for their servers. IEEE standards also may come into play.
Sophisticated power controls operate at extremely high speed —think milliseconds — and cache or share voluminous data from one device to the next, without disrupting other building functions.
“When you are doing forensics, you need fast and accurate time marks to track down where things went wrong,” said Toporek.
Northwestern Memorial Hospital’s Prentice Women’s Hospital in Chicago accomplishes data transfer with a self-sustaining, isolated network that includes an Ethernet dual fiber optic ring that is self-healing.
“Self healing means that communication happens both ways on both rings,” explained Junnaid Malik, electrical engineer with Cosentini Associates’ mission critical group. “One ring could be physically cut and the system could still communicate.”
Power quality analytics are the leading edge of power control technology. Power quality analysis is very different than traditional monitoring. Analytics look at power harmonics and high-speed transients, and can be used for trending and predicting growth. According to Malik. “You may be a colocation facility and want to plan for growth by adding servers."
Power quality applications of power controls are as unique as the businesses that are using them, say experts. For example, power quality analytics can be used to identify where a failure could have occurred. They also facilitate pre-function testing for scrutinizing systems and their responses, and to simulate transients.
Post-event troubleshooting is another key function of power quality analytics. Why did the facility lose a particular breaker that tripped the PDU and caused a chain of events that caused a switchover to the UPS?
“Was it an electrical spike, a floating ground, or a short?” asks McCarthy.
In terms of security, some CPMS are protected with the same 128-bit encryption technology used by the National Space and Aeronautics Administration.
Power controls can provide benefits beyond supporting the reliability of the emergency/backup power system. For example, in terms of operations, maintenance is more easily scheduled by bypassing units and swinging their loads to others in the emergency power system. What’s more, reports help with budgetary planning as well as documenting power reliability and other business benefits for co-lo tenants.
Power controls include both functional testing and ongoing com-missioning, even when facilities are not required to perform monthly or periodic emergency power testing. Although power reliability is their primary function, power controls also can help save energy. For example, Fenlon-Harding sees the knowledge gained through such systems being used to transfer loads, optimizing utility peak demand loads.
Which of the four monitoring and control technologies is optimal? That's for you to decide, based on the importance of reliable power to a given set of operations or processes.
If reliable power isn't absolutely critical, a SCADA system or BMS might be appropriate. If a holistic view of a data center's IT and facilities infrastructure reigns supreme, DCIM might be the logical choice.
When stakes are high for maintaining critical power, consider the specialized capabilities of CPMS monitoring and control that can work alongside one of the other monitoring and control technologies.