In recent years the industry has become passionate about monitoring anything and everything inside the data center and in its supporting infrastructure. It’s reached the point that the reliability of the data collected has become as important as the data processed by the data center.
This passion for information has created a whole new distributed system of electronics in electrical and mechanical gear. We now place PLCs, BMS cabinets, BAS cabinets, and a multitude of other distributed processors in unprotected environments throughout our facilities. Further, the demand for instant and accurate information about the data center operation has driven our conversion from dumb contact closures and 4-20 mA proportional signals to intelligent circuits that talk to the system that monitors the status of the data center along with diagnostics of the monitoring circuits as well.
Although monitoring of mechanical systems continues to increase at a good clip, electrical monitoring seems to be advancing at lightning speed. At every level of the one-line, the industry is monitoring volts, amps, kW, kVA, kVAR, circuit breaker positions, heat, power quality, and more. We measure and monitor at the utility entrance, the generators, the transformers, the PDUs, the remote power panels, and the cabinet power strips right down to the outlet.
As our zeal for this information causes it to be more readily available, we continue to increase our dependence on it. New applications today provide real-time analysis of the daily operations and instant notification in changes of state. Today, almost every system does change of status notifications via text messaging, e-mail, pagers, radio, and automatic call notices. Some systems are now sufficiently sophisticated that they try multiple approaches until the individual being notified acknowledges the notice, which is then time stamped and logged.
The story here is that we are creating distributed data centers that monitor, report, and control our primary data center, but we don not pay the same attention to the data centers as our primary data centers.
I recently came across a data center where the vendor supplying the EPMS (electric power monitoring system) furnished a mini-UPS for the head end but nothing for the remote input electronics. The irony is that this EPMS was doing power quality monitoring while it was running blind on every power excursion. The power quality excursions were never recorded.
I thought the vendor’s logic was inconsistent but being curious about what others thought, I posed the question to different industry groups. The results were nearly unanimous. Here are a few of the excerpts from the replies:
The Question
When EPMS is installed in incoming service switch gear that has electrically operated CBs (circuit breakers) and station batteries, the EPMS is powered off the station batteries and therefore is unaffected by power interruptions.What is your opinion when the incoming service switch gear only has manually operated CBs and therefore no station batteries?
Would you specify the EPMS with a UPS or other form of power backup to operate continuously during an interruption of the primary service power?
The Design Engineer
“Absolutely yes. If one is going to the added expense of specifying the installation of a qualified EPMS in the service entrance switch gear, and presumably the standby/emergency power switch gear equipment, then, whether the breakers are electrically or manually operated, the EPMS must stay operational under every reasonable ‘failure mode’ one might envision.”IT at a Server Manufacturer
“In today’s world, having a facility that is NOT instrumented in most every way possible is careless. Data is crucial to forming information, and information is crucial to making decisions. The cost of instrumentation is inconsequential to the value of the data you protect, so it should be hard to justify a denial of action.“That being said, keep in mind that IT systems are accelerating their ability to capture, store, and analyze environmental and electrical data that has been traditionally managed by other types of systems.
“The fact is that IT systems are embracing environmental and electrical measurement telemetry features, largely because we can use the data to make better power management decisions on the fly, and provide better information to the facility.”
From a Switch Gear Manufacturer
“Great topic. I would always recommend some sort of continuous power to the metering and monitoring devices. The benefit is for the advance metering with power quality detection to show the downward spiral of the electrical system data and most importantly the startup or return of the electrical system. When you lose power, advanced monitoring can tell you the magnitude, duration, and most importantly, the direction the event is coming from. When power is re-applied, the monitoring devices are in a “boot” cycle that may take a few seconds to regain its full monitoring capabilities. This is an eternity in electrical terms and may lead to missing critical data for this duration. In the restart mode, the advanced metering data will show system loading, motor starting curves, chiller time delay verification, and help understand system restart stability. This information is priceless and the cost of continuous control power is minimal compared to the benefit of the data produced. Power outages sometimes cannot be prevented. At least with continuous control power, you will have an explanation what happened, when, where, and how can you improve the restart.”The Guarded Dissenter, A Contractor:
“At the end of the day it is all bent metal managed by man. I do not suggest the caveman approach to measuring and monitoring, but I have had it with the special sauce of the day to achieve a PUE or other new math metric to keep an unsuspecting client or boss happy. Drive slowly here. We have all ripped out someone else’s best practice or ‘have to have’ solution. Clients are no longer going to get vendored into new software, hardware, and retrofit cost with unclear long-term benefits or TCO enhancements. Today, you can measure or monitor yourself into a coma...careful....”Consensus?
So everyone (except the operator of the site that got me started on this research) agrees. Because of the criticality of the data, these distributed electronics need power back-up. So then, what about the environmental needs of these electronics? We’ve established that they are critical, so why do we subject them to different environmental standards?The most obvious difference in treatment has to do with cleanliness. When new data centers are built most users will not install their computers until the data center environment is at near clean-room condition. Yet the electrical and mechanical spaces housing the infrastructure that supports our critical data centers are often loaded with dust, dirt, and debris. When these areas are cleaned, it is usually done with a broom and then a Shop Vac. Seldom are the inner components of the gear cleaned of dust, dirt, or debris. As more electronics are pushed into this gear we need to change how it is treated.
If you had the foresight to write the cleanliness of these systems into the project scope, the electrical gear would stay shrink wrapped until the rooms were clean. But unless it is explicitly written into the contract, the cleaning won’t be done because electronics are relatively new so it is not yet customary to the trade work. Requiring this level of caution goes well beyond customary protection from physical damage and the elements.
These systems are not just the contractor’s responsibility, however. I remember an instance when the client refused to allow the contractor to pave the roadway around the data center until the end of the project. As a result, despite the contractor’s extraordinary best efforts, the dust kicked up by all the construction vehicles permeated the entire facility including the switch gear. In the end it cost the entire team (owner, designer, contractors) time and money to take everything apart, clean it, and put it all back together.
So as owners, designers, contractors, vendors, and consultants, what are you doing to address these ever-changing needs for continuous and reliable monitoring? It appears that most everyone is getting it when it comes to continuous power for the electronics but what about all the other environmental issues such as heat, condensation, and cleanliness?
In the end the operational reliability is only as good as the weakest link in the chain.
