Editor’s note: The growing importance of data centers to health-care facilities took center stage during a round table of national experts brought together to share insights on maintaining power for life safety and mission critical operations.

Sponsored by ASCO Power Technologies, a business of Emerson Network Power, nine executives from health-care facilities, consulting engineering firms, and an electrical inspection organization discussed unique power strategies for hospitals and their data center operations.

They also shared insights on maintaining electronic health records, ensuring power for on-campus and colo data centers, fully redundant hospitals, and ‘selling’ emergency power to hospital management and directors.

Patel: Let’s begin our discussion by talking about what’s involved with emergency power systems. For example, are designing, installing, commissioning, constructing, and maintaining power systems changing to reflect the trend toward digital health care?

Hungerford: The data centers that support what we do have gotten larger and larger. The expectation that it’s going to be always ready, never corrupted, and never go down is a cornerstone of how we do it. Our clinic is a very busy outpatient clinic, and there, too, I see the same expectation of reliability. Everything is built towards never, ever going down. That’s the data center side, and then the digitization of how they intake patients, how they move them from one specialist to another and keep those records moving with the patient through the day, that has become more and more critical.

Patel: Dan Michalak, would you share your perspectives in terms of commissioning emergency power systems? What’s the focus at the University of Nebraska?

Michalak: We really focus our commissioning efforts on long-term outage testing. We try to get a good eight-hour test during which normal utility power is gone and see how the whole building reacts. Especially if it’s a hot summer day, you may have emergency power to your so-called critical loads, optional standby loads, equipment loads, etc. But what about equipment that might be taken for granted, such as cooling in the conference room, or basic equipment like condensate pumps? It’s an entirely different animal with a long-term outage. All sorts of problems start popping up that you didn’t expect.

Willey: We have all of that in place at Benefis. We reassured our IS (information systems) people that if we lost power, you guys are going to run. We proved it to them several different times. What ended up happening, though, is that they lost the main server for our patient documentation system; hence they had power but they didn’t have a server. So, for 24 hours, we reverted to a paper system. The interesting thing is that many of our staff had never documented on paper, so we ended up doing a quick training for staff, nurses in particular, on how to document.

Michalak: Let me provide a scenario that we encountered last year. During the record heat wave that went through the midwest last summer, average temperatures were 95°F to 100°, with a heat index of 110°, 115°. A ‘what-if’ scenario for a research medical facility under those circumstances might be what happens if our local utility substation fries? And that’s very possible. We could have a terminal or a PLC failure at the utility level, and the whole campus goes black. We don’t have any normal power whatsoever, and we would have to rely on our emergency power.

From a code perspective, we’re in compliance. But the reality is that it’s going to take one day for the local utility to repair their substation before they can restore normal power to our campus. If we’re in an eight-to-12-hour outage when it’s 115° heat index, the generators will run fine but the way we’re set up is the chillers in the central plant go off line because they’re not considered an emergency or critical load. So, we’ve lost all chiller capacity or chilled water to the hospital, aside maybe from ORs, which have their own local DX units. But you also lost chilled water to probably half of your data center. So the wheels start turning and you’re thinking, what’s going to happen if the power outage occurs at 10 in the morning and you just started hitting the peak of the day? How is this research facility or the hospital facility going to be running or be capable of operating and maintaining patient care at six o’clock in the afternoon? Yes, they comply with Article 517 and Article 700. But other loads like patient-care rooms, which are just on normal power for HVAC, are going to get real hot in a hurry. You’ve got to wonder how long the data center will stay cool before things really start happening. So that’s a what-if scenario for being code compliant, but would it work in a true emergency crisis situation? That’s another story.

Patel: Daniel Caron, can you give us the perspective of consulting engineers on this topic?

Caron: What’s the price of finding out the problems in a functioning hospital? If you find a problem, you need to fix it right away. Whereas if it’s found during the commissioning process, you have the time to straighten it out in a controlled environment, while the contractors are still on-site to fix it (and under warranty). After the fact, it‘s going to cost much more.

Patel: You make an excellent point, Dan. Let’s switch gears for a moment. What role would virtualization play in a health-care data center? Jim, can you share your thoughts?

Brownrigg: If you move towards virtualization, the data center will help that dramatically. But, most of the key EHR [electronic health record] vendors today are still pushing pretty hard for dedicated servers, but that’s starting to change. After all, virtualization requires fewer servers, which leads to lower power consumption. Overall, virtualization would increase the hospital’s power reliability.

Degnan: There’s a huge need to work out several aspects of the National Electrical Code [NEC] with regard to life safety and critical operations. Traditionally, I would view Article 700 as the article that addresses life safety and building egress systems. Article 708 is fairly new to the National Electric Code, and that is sustaining critical operations, not getting people out of a building.

Patel: Jonathan, can you tell us how electrical inspectors approach this?

Cadd: We have our critical operations power systems [COPS] and our emergency systems, both designated as the number one power sources in order of importance. Now that we’ve got our NEC Article 645 requirements that tie our data back into NEC Article 708 requirements for COPS, NEC Article 708 says I’ve got to keep my critical operations power systems running, and it also says in 708.22, for an unlimited number of hours. Only during times of maintenance can we take that system down and guarantee 72 hours from an alternate power source.

Michalak: It really has a snowball effect.

Cadd: In a healing environment and for health care, in general, that data is critical, and could very well produce a chain reaction of adverse circumstances. All of a sudden we’ve made code changes. We’re looking at critical operations power systems, but yet we’re looking at data. We’ve tied those two in. However, we’ve changed the hierarchy of how the actual required power and how that emergency power actually function now to two completely different code sections.

Wilkening: Does 708 understand the difference of critical data systems as it relates to sustaining life or is critical data to manage patient care? Because I see a difference in the ‘sustain life’ vs. ‘management patient care’.

Cadd: That’s a great question, because if you look at it from the inspector’s standpoint, it’s life safety, pure and simple. Typically, you’re not going to have JCAHO [Joint Commission on the Accreditation of Health-care Organizations], and HIPAA [Health Insurance Portability and Accountability Act], as those type of acronyms are generally foreign to day-to-day inspectors. At the end of the day, from an inspector’s standpoint, if I get everybody out of this building, if I’ve got the required emergency power systems, along with required battery back-up, on life-sustaining systems, the building will take care of itself. But it’s pure life saving, it’s getting bodies out of that place and making sure they’re safe from an inspection standpoint.

Willey: Except, I don’t want to get the bodies out of the building. I want to defend in place, because I’ve got several floors of people that I can’t move. I’ve got NICU (neonatal intensive care unit) babies. I’ve got intensive-care patients that are hooked up to every life support monitor you can get. I can’t move them, so it’s not about getting them out of the building. It’s about keeping them.

Wilkening: As a facilities director, the hierarchy is the critical branch is first because that’s serving airway, breathing, and circulation; it’s life or death. Critical power branch are systems that can create huge risk issues to either the patient and then life safety.

Degnan: Now you’re jumping over to Article 517 and that’s where hospitals should perform. In our discussions of a critical facility, 708 is probably not the best article to look at for the performance of most hospitals. Hospital disaster centers have recovery areas that are focused on certain types of disasters. 708 might appear to be a good choice for that hospital, but because of all the interlocking departments, there isn’t a very good way to define a critical operations area within a hospital for 708. You have to go back to 517.

Michalak: One of the underlying questions, though, is when you go to electronic paperless records, does a data center in a business occupancy that is relatively off campus, or on the edge of campus, that is served by an optional standby system, does that now become an Article 708 critical operations facility beyond NFPA [National Fire Protection Association] 75 requirements of a so-called information technology system? There are a number of codes and standards that could apply to that. So when somebody wants to connect emergency power, what does that really mean? The snowball effect, cost implications, and logistics of doing that could be huge.

Fernandez: One of the things that has to be looked at is events like Katrina that lasted an extended period of time. I don’t think any code was developed for applicability over 72 hours, like the field capability of diesel generators. Katrina lasted much longer, And horror stories that came out of some of the hospitals, wow, how are we addressing that as a community?

Wilkening: One of the challenges is that hospitals seldom forget that they can also become victims in times of disaster and what do you do to provide patient care when you’re the victim as well. That needs to be understood extremely well.

Patel: There’s a trend in regards to supporting an entire facility. What are the drivers for what gets placed on emergency power or not? What are the drivers for facilities that can’t be fully backed up?

Caron: The main drivers are what’s allowed by code.

Willey: One driver to add equipment is if you are the sole provider in a 200 square mile area, you do it. We also support a military base from a hospital standpoint. They have a clinic. So that’s one of our drivers.

Wilkening: Historically, if we were on limited power due to a main power system failure feeding a hospital, our surgery department would shut down all surgicals that weren’t considered urgent, and that would cause a huge backlog of rescheduling surgicals. That would mean staff was standing around doing nothing.

In understanding the cost of not doing business, and why we needed a fully redundant hospital is why I interviewed a surgical director. I had no idea that from a lost revenue perspective for our hospital it was $17,000 per OR per hour. And when you add that into the equation of risk aversion to the cost of not doing business, and that’s not counting soft costs inside the hospital, such as staff standing around doing nothing, it really adds to the business case very rapidly. We also interviewed our emergency department, our NICUs. When these people feel a power loss, their lives are in chaos. We’ve got to think how do we use that paper and pen again. That’s the real world and people don’t want to live that. They want to do the right thing.

Patel: Todd, you’re hospital is fully redundant, isn’t it? Would you like to talk about that?

Wilkening: We’re the 911 backup for Carver County, and we sustain emergency response for the community. There was a little more than a million dollar gap between getting a 1.25-MW generator and a 2-MW generator. The dollars just weren’t in the budget, so I got creative. We looked at rebates, but I also interviewed service line leaders of the hospital who were overseeing the babies, the emergency departments, and surgical center and started talking about what they went through in terms of not only business loss but the operations, what they experienced. It was scary for them. They don’t want to experience that. I bundled these stories in a package and presented it to our board of directors. I helped them understand at this point in time we’re going to have to cease providing NICU services. At this point in time, we’re going to have to close our ER and go elsewhere. They just couldn’t fathom being in that position…of being a community leader and saying we cannot provide patient care. Even though every hospital has limitations, we wanted to make sure we were doing everything we can to prevent any limitation at all. The program sold pretty quickly. After 15 minutes of review, the board of directors said, ‘We’ve got to do this.’

Hungerford: I think I run into that too when we do design reviews. They know what we want in the beginning. They try to VE [value engineer] out the bigger generator, the better UPS, the isolation bypass features, and all that. But when we make a case for it, then it usually will come back. Nobody wants to go on record as saying we can’t afford that, and then a couple of power outages later, are faced with three or four times the cost to add it in.

Patel: What are some of the unique approaches you use to solve emergency power requirements innovatively?

Degnan: There’s a lot of discussion on the critical nature of providing emergency power in hospitals. I’m finding that we serve a variety of clients with a variety of utility reliability experiences. If the hospital board of directors sees their local utility has twenty ‘nines’ of reliability and it never goes down, they’re far less likely to be interested in extensive E-power provisions and investing in more generators beyond what the code is going to require. If it’s a hospital that is in a more rural area and has seen some challenges to delivery of power from the utility, they’re being served by overhead lines that are subject to outages and so forth, they’re far more likely to be interested in investing in emergency power. One of the things we’re seeing in the marketplace now is a more extensive use of uninterruptible power supplies to serve critical loads. The data-center pieces of health-care facilities are always on uninterruptible power supplies [UPS], for example, because of the critical nature of medical records.

Michalak: Our hospital also added rotary UPSs to the system. One was a 900-kW unit and the other was a 600-kW unit. But the one interesting thing is the load flow study aspects of it. Prior to the UPS, during a normal power interruption, as the transfer switches transferred power and loads started up, especially HVAC, that load ramped up.

With a UPS on the system now, you have a block load all of the time. It’s an interesting commissioning question. What happens with the UPS when that transfer switch slams and your generators are going to see the full HVAC running mode, vs. actually stopping and starting and slowly ramping up? From a load-flow perspective, that needs to be looked at, especially at an existing facility that has a rotary UPS. How does that affect the overall dynamics of the electrical system, specifically stopping and starting mechanical equipment and emergency generators, especially with the 10-second rule?

Caron: Generally speaking, it is difficult to get multiple generators in parallel on line in 10 seconds. In our designs, we make sure the priority one loads, life safety and critical, are less than the rating of a single generator. You may have to break up the paralleling bus into multiple components if you need to have that much priority one load.

Patel: Some of you have data centers in your facilities already. I’d like to know if any of you are using a remote facility that’s not part of the system or owned by the hospital.

Brownrigg: Many of the EHR vendors, and one in particular, markets very successfully a hosted solution—an ASP-based solution for their EHR. For many hospitals, EHRs are not actually hosted in their facility or in their data center. As a matter of fact, they’re hosted by Cerner in Kansas City. So that’s pretty predominant. The particular applications will be hosted by the EHR or CPOE [computerized physician order entry] vendor, but they’re also hosted within the facility.

Patel: Are hospitals becoming data centers with patients?

Cadd: From an electrical inspection standpoint, we are seeing more and more occupancies that are part of the modern hospital complex. Indeed, we are seeing large areas that are earmarked for data storage in conjunction with the day-to-day hospital operations. Where large amounts of power were being earmarked for hospital diagnostics and procedures such as MRI, proton beam therapy, and others, we are now seeing the same amount of power or more now being provided on the data side.

Caron: I don’t think so. When I think of a data center, I think of UPS for everything, for power that absolutely can’t go down, and if it does go down, money and information are lost. Generally, if power does go down in a hospital, it’s inconvenient to the patients and staff, but they don’t, in theory, lose money. Most of the critical equipment that is directly taking care of the patients has its own battery back-up power. Power reliability in hospitals is (obviously) important, but hospitals normally have contingency plans to deal with it. If they lose power to an MRI or a CT, again it’s inconvenient for the patient and staff, and they may lose that one session if they have a problem with the power. They do lose some money, but not like a data center for a bank where millions of dollars could be lost.

Degnan: My perspective is actually no, that’s not the case. Data centers are evolving reliability standards that are in excess of any code. The reliability of a data center is being set by what the owners consider is the financial aspect of the information that’s being processed and what the financial implications are regarding that loss. In hospitals, the most critical elements—the ORs, the ICUs, the neonatal care centers, and so forth—are being driven by conventional codes requiring, at a minimum, that those systems have a single transfer switch and give the option of multiple transfer switches. But there’s no mention in those codes of anything like uninterruptible power supplies or redundant configurations of uninterruptible power supplies being required for electrical systems that support patients. It’s somewhat surprising, but if a hospital chooses to have a data center, it’s not uncommon to have a data center within that hospital with a higher level of power reliability than the operating room

Hungerford: I thought about that and asked myself,  “Is planet Earth becoming a data center with humans on it?” The answer for me is, generally, yes. There’s room for improvement on how patient records, consultation notes, and test results are handled. More efficiency is needed. Getting our health-care costs under control will no doubt include looking at managing data digitally wherever we can. The very first data center at Fred Hutchinson existed specifically to make cancer patients’ records and treatment protocols digitized, searchable, and retrievable.

Michalak: Saying it’s a ‘data center with patients’ may be a little strong, but the emphasis on electronic records and not having paper records, especially onsite, from an electrical perspective is a little disturbing to me.

Wilkening: I’ve heard that term several times, and we are extremely sophisticated with information systems and also communication systems. We are essentially data centers with patients. I would agree with that. I just hadn’t thought of it in terms of the context of the entire building or business, but it is very true.

Willey: I believe we are health-care with a great need for information access and data. We are becoming more and more high tech, and hence have an increasing need for space, FTEs, and data software/storage; but we still exist to care for our communities in whatever capacity needed. We are storing several hundred times the data that we stored a few years ago.

Brownrigg: That relates to the first topic and the answer is, yes and no. You have to step back, because you say data centers. What is the strategy from a hospital’s perspective, and where are they willing to put that critical information? Nothing is changing from a physical environment, from a critical facility, and critical patient care standpoint. There still are all the requirements for emergency power associated with life safety and with critical patient operations, or clinical path. It all depends on a hospital’s IT strategy for deployment. Are they hosting all of those applications at a data center within their hospital? Are they doing that in a system-wide data center? Are they doing a hosted or cloud application? Or, are they going to a version that has servers and software hosted in a collocation facility? There are a multitude of options for the facility to deploy this.

The importance of data centers in helping health-care facilities defend in place will continue to become more critical. Ensuring power for data center operations as well as traditional hospital operations is focusing attention on the scale, redundancy and reliability of emergency power systems. As this roundtable discussion shows, there are a number of philosophies and procedures for making that happen. Facility and IT managers working together can decide which is most appropriate for their given facilities. 

Patel:Thank you all for sharing your perspectives on emergency power for health-care facilities and especially for your insights on the importance of data centers to those facilities.