Every data center has backup generating capacity. Most hope that they will never need it but know that should the situation arise, they must be able to power the data center.
Despite this, how many data center managers know how much power they really use? Are they measuring power accurately enough? Do they have the right level of maintenance and spares to maintain their generating capacity?
LEARNING OBJECTIVES
- Power mix changes require data center owners to talk to suppliers in order to meet environmental goals.
- Onsite generation equipment must meet installation standards, be accurately sized, and subject to proper maintenance.
- Power usage must be carefully monitored.
INTRODUCTION
The Datacenter Dynamics, Data Center Market Trend Report 2012 shows that 1.8% of all the electricity generated globally is consumed by data centers. To put a specific figure on that number 1.8% is equal to 322 Twh of electricity, enough to power the majority of the countries on the planet.
U.S. energy usage by data centers rose just 8.7% in 2012 putting it 18th in the global growth league. In 2013, usage is predicted to be 9.8% moving the U.S. up to 15th. Despite its modest growth rate, the U.S. data center market is the biggest global consumer of data center power. In the period 2012-13, it is projected to consume over 92 Twh of power or just over 25% of the global data center power usage.
The Datacenter Dynamics, Data Center Market Trend Report 2012 also provides projected figures for the growth of U.S. data center space in the period 2012-2016. These figures show that while space will increase by 4.5%, energy requirements will increase between 6.5% and 11%. This is due to greater consolidation of hardware which will drive the energy requirements per rack upwards.
There is good news for the U.S. in these numbers. It has one of the most efficient data center markets in the world. Carbon emissions are falling and green technologies are on the rise. However, given the amount of power consumed by U.S. data centers, there are questions as to whether the efficiencies can match the growth.
THE U.S. POWER MIX
There is a perception that the U.S. is a major carbon generator and that data centers are part of the problem. While that may have been the case a decade ago, there has been a remarkable turnaround in the U.S. energy mix towards a low carbon footprint. Part of this has been driven by renewables, part by the soaring cost of oil and coal, and part by the impact of fracking, which is delivering much lower costs for gas and oil.
The George Marshall Institute has recently published a paper by Michael E. Canes, a Distinguished Fellow at not-for-profit LMI, titled, “Changes in the Fuel Mix Used to Produce Power in the U.S.” This document makes for extremely interesting reading as to how the U.S. power mix is changing and the rise of both gas and renewables.
Some data center managers are now looking to control their environmental impact by choosing how their power is generated. Reading Canes paper suggests that there is a good case for buyers to talk to their suppliers as to how the energy they use is generated. With a need to provide environment statements on corporate websites, this will allow data center managers to choose the most sustainable and affordable power mix.
RENEWABLES
Unlike Europe, the U.S. has been successful in selling the idea of renewable energy to data center managers. Getting figures on how successful isn’t easy. The Environmental Protection Agency (EPA) has a Green Power Partnership list that shows how well companies are doing. In its latest list of top ten green power users, the EPA lists IT vendors Intel, Microsoft, and Apple.
" Emissions are a major issue for any form of EPSS. One of the most common fuel types is diesel which is a known emitter of contamination. If a data center chooses diesel they will have to deal with a higher maintenance schedule, especially in terms of filters. In California, the Bay Area Air Quality Management District lists a large number of data centers as major polluters due to diesel fumes.
To try and reduce pollution, the EPA encourages companies to only use low sulphur diesel products. "
The problem is that this is not a completely accurate list because not all U.S. companies have signed up to the EPA program. Conspicuous by their absence are Facebook and Google, two companies that have been winning awards for their commitment to renewable energy. What is needed is a concerted effort by the U.S. to create a single list that could be used to drive further adoption of renewable energy sources.
While subsidies play a significant part in the development of renewables in the U.S., especially for wind and solar, they are beginning to decrease slowly. The cost of geothermal, hydro, and wind has already fallen below the cost of generating power from conventional coal.
Solar is still some way behind other renewables because of its current inefficiencies. That hasn’t stopped Apple, IBM, Google, Facebook, and others, building solar into their power mix. This suggests that we are on the cusp of solar taking a big dive in price, which will make it much more accessible for corporate use. Since 2011 the price of solar panels in the U.S. has dropped by around 60% which is driving adoption of the technology.
DODGING THE NUCLEAR AND COAL PROBLEMS
One of the big advantages of the U.S. over Europe is the lack of reliance on nuclear power. While Europe is beginning to worry about how to replace nuclear as part of its power mix, the U.S. has been slowly reducing its nuclear reliance over the last six years.
The big worry for the U.S. has always been the carbon emissions from its coal based power generation. With the explosion in natural gas from fracking, there has been a significant shift away from coal to gas and that is set to continue for the foreseeable future.
EMERGENCY GENERATION EQUIPMENT MUST BE PROPERLY SIZED AND MAINTAINED
One of the risks of any power intensive industry is the risk of a brown-out. This is why it is good data center practice to have access to both emergency and standby generating capability on site. The National Fire Protection Association Standard for Emergency and Standby Power Systems (NFPA 110) is a document that all data center managers should read and adhere to.
Chapter five deals with emergency power supply systems (EPSS), the energy sources, converters, and accessories that can be used. It provides a classification of EPSS based on the load and the length of time that they are expected to provide cover, the types of fuel and how the energy will be converted. As data centers expand and the load required to keep equipment working changes, managers must check that their EPSS are capable of meeting current requirements.
Chapter seven deals with installation and environmental considerations around the siting of equipment. It deals with exhaust, fuel tanks and their location, how systems should be cooled and, essentially, how systems are earthed. What it does not do is set noise or emission limits. These are set by local governing bodies and they are consistently being tightened.
As demand for data center space has risen, there has been an increase of data centers being built in refurbished industrial units. Some of these are close to houses and this means that there are likely to be serious restrictions on noise levels and times of day that the generating capacity can be used. Noise limits are likely to determine the type of EPSS in use.
Emissions are a major issue for any form of EPSS. One of the most common fuel types is diesel which is a known emitter of contamination. If a data center chooses diesel they will have to deal with a higher maintenance schedule, especially in terms of filters. In California, the Bay Area Air Quality Management District lists a large number of data centers as major polluters due to diesel fumes. To try and reduce pollution, the EPA encourages companies to only use low sulphur diesel products.
GET GENERATOR SPARES AND SERVICE RIGHT
Chapter eight of NFPA 110 deals with routine maintenance and operating testing of equipment. For some data center managers the schedules defined here may come as a shock. It is not uncommon for some facilities to only test their generators once a year when they run an emergency test. This is not only insufficient but should be considered a complete failure of governance.
Using diesel generators as an example, NFPA 110, 8.4.2 states “Diesel generator sets in service shall be exercised at least once monthly, for a minimum of 30 minutes.” It then goes on to determine how that testing will take place. Many facilities do not do this level of testing and data center managers must review their policies urgently.
In the aftermath of Hurricane Sandy, some data center owners discovered that their fuel tanks had become contaminated. This led to stories of teams of workers using jerry cans to carry diesel to the top of buildings to refuel generators. Fuel contamination is not unusual but it is not the only fuel related issue around data centers.
Fuel for diesel generators needs to be properly conditioned. This helps to remove impurities which reduce damage to the engine. As well as the conditioning, filters and strainers need to be regularly checked and changed. Where a generator is not used regularly, these tend to get forgotten. A clogged filter can cause a generator to fail unexpectedly.
One of the main causes for clogged filters is contamination of the fuel supply. Another is a build-up of muck and algae at the bottom of fuel tanks. When the tank is run very low, muck is passed through the fuel lines and clogs the filters. Cleaning can be a long and costly process.
A major concern for some data center owners has been theft of fuel. Data centers that do not have high levels of physical security are becoming attractive targets for thieves who can sell on stolen diesel for a lot of money.
REDUCING ENERGY COSTS
Reducing energy costs has been high on the data center agenda for years. While better management of the IT side of the business and alternative cooling still have their part to play in reducing costs further, understanding power requires a wider view.
There are three key power related steps that data center managers need to take:
• Power reduction programs need to be accelerated and cut deeper than before. Central to this is replacing equipment more frequently to ensure that only the most power friendly systems are installed.
• A detailed understanding of where and when power is used is essential to ensure key systems are kept powered.
• Emergency power systems in facilities must be properly maintained or updated, reviewed, regularly tested, and given a higher priority than many other systems.
ACCURATE MEASUREMENT A PRIORITY
A priority for data centers is the ability to accurately measure the amount of power that they use. With so many different management systems monitoring the hardware in the data center, there is a real risk of ending up with too much data and too little information.
Where to monitor power usage is a subject of much discussion. Power is lost through transmission and through conversion. A lot of data centers do multiple AC/DC conversions inside their infrastructure. What they do not do is calculate how much power is lost during the conversion process. By measuring that loss and the number of power conversions, it is possible to see where gains can be made from better power management.
The most effective way to know how much power is being consumed by devices is through intelligent PDU strips. These strips allow you to monitor either individual devices or groups of devices. The ability to monitor a group of devices is especially useful when dealing with a cluster of systems that need to be protected as a single unit. The operator can see how much data the group uses and ensure that there is sufficient power to keep all devices running.
Once you are getting usage data from devices, the next stage is to understand usage patterns. Not all devices pull data at the same rate 24x7. While storage devices and switches are generally a constant load, computational device power varies constantly. With virtualization, predicting power requirements has become more art than science for many data centers.
One reason for this is that power monitoring is not linked to the automated systems that manage the virtualization. This means that data centers need to develop better systems to capture historical data and to be able to restrict the automated movement of virtual machines when there is a power situation.
When the data center is in a power restricted mode, it should be possible to start shutting down lightly used hardware and consolidating loads elsewhere. While this already happens to a limited degree, it is not driven by power availability. This will require a rethinking of how power and computational load work together.
There are solutions on the market that allow operators to see where power is being used, capture usage patterns, and power down systems remotely.
DON’T FORGET TO MEASURE POWER USED BY COOLING
A major source of power consumption is the cooling systems in the data center. When calculating the amount of power required to keep the data center running, it is critical to understand the requirements of cooling systems.
If cooling systems are allowed to lose power, there is a significant risk of thermal runaway, especially in high density racks containing switches and blade servers. Once thermal runaway begins, it is extremely hard to stop.
CONCLUSION
Energy is the lifeblood for any data center, it is also the biggest cost. New generating capacity in the U.S. continues to lag behind peak usage. This means that power outages will continue to happen and data center owners need to plan carefully.
Detailed understanding of where the power is consumed by both IT equipment and cooling is essential. It is also important to understand where power is lost through conversion to ensure that data center owners have the most complete picture of power requirements in their facility.
Once this is done, time spent getting the right generating capacity and making sure that spares and fuel supplies are secure and fit for purpose are a critical investment.
Data centers need to act now and act fast.
