Fast forward to today and distributed systems, cloud computing, and virtualization are ubiquitous. It’s not unusual to find data centers now that can support an average density of 300 to 500 W/sq ft. Operating expenses, not capital costs, are more important as efficiency becomes more of a priority.
This data center evolution has led to unrelenting improvements in the way these facilities process, store, and network data. Since 1980, the central processing unit (CPU) has improved in performance approximately 60% annually.1 Storage continues to double in capacity every 24 months and has improved in performance tenfold over the last 15 years.2 Network speed continues to increase by an order of magnitude every five years.3
This cycle of innovation, adoption, and scale has driven substantial growth in the number of users and sensors (or devices) connected to the web. The Internet user population doubled from 2007 to 2012, from approximately 1.15 billion users to 2.27 billion. Surprisingly though, this accounts for less than 50% of the world’s population. The number of devices connected to the Internet in 2010 was 12.5 billion and this number is anticipated to reach 25 billion by 2015 and 50 billion by 2020.4
TOO MANY SNOWFLAKES
However, even with this explosive growth and continuous improvements in the way data centers organize and manage data, data centers in many ways are still being designed in inefficient and wasteful ways. One of the drivers behind this is the fact data center design and construction is still considered a craft passed down through generations of engineers and governed by individual design preferences so that no data center looks, operates, or is maintained the same. Each data center is unique just like a snowflake, which makes it challenging to deliver the performance needs that a data center’s customers and users demand.
BREAKING THE PARADIGM WITH MODULAR DESIGN
To combat this, some of the industry’s most innovative data center suppliers are breaking away from this legacy approach with the introduction of standardized, prefabricated modular designs supported by contemporary, lean manufacturing systems. Clear advantages exist with a modular design in terms of availability, efficiency, and budget vs. a conventional brick and mortar build. These benefits address many of the challenges data center owners and operators are faced with today including speed to deployment, high capital costs associated with traditional builds, and rising data center capacity.
What does modular mean in the context of data centers? To avoid confusion and for the purposes of this article, modular design/build is one that includes a pre-engineered, factory built, and pre-tested IT, power, and/or cooling module(s). The module uses standardized, repeatable designs and can provide for site and/or user specific customization options (Figure 1).
The design enables easy on-site assembly that consumes less time and effort as opposed to a conventional build. A modular deployment can be containerized or skid built. Put simply, modular is the exact opposite of a custom design and/or conventional brick and mortar build (Figure 2).
A modular deployment, for example, can be an existing or new building with factory built power, cooling, and/or IT modules. The deployment can also be a purpose built, prefabricated building with modules, or it could be a fully containerized modular data center with modular power and cooling infrastructure. It could also be a variation of any of the examples described above.
Today’s modular data centers are designed as complete packaged solutions that are assembled and tested before reaching the customer’s site. A wide variety of vendors and products are available in the market, including modules that house servers, storage, and networking or the power and cooling infrastructure required to operate the IT in an environment optimized for maximum energy efficiency. Modular data centers can simplify the on-site installation and startup, which can be placed adjacent to existing buildings or in parking lots and can offer a viable solution for clients that are space and schedule constrained.
Is a modular design right for you though? What questions should you ask before evaluating such an approach? The following five questions are ones any data center owner or operator should ask before considering a modular design build. This isn’t an exhaustive list, but addresses the critical topics to consider before engaging with a modular supplier, helping avoid costly mistakes, and save time in the long run.
Why should you consider a modular data center? Of all the reasons why customers look to modular data centers, the most common considerations are based on a customer’s space constraints, schedule, and energy efficiency goals. For example, modular power infrastructure solutions are typically compact in size and provide greater deployment flexibility that allows operators to reduce the size and cost of their land and building. In fact, these solutions can reduce the amount of traditional building space needed for a data center by 20% compared to a traditional electrical room.
Modular data centers are also very fast to build and deploy. These solutions can be up and operating 60% faster than a conventional brick-and-mortar deployment, reducing total time from more than a year to 30 weeks or fewer.6 Further, these systems are factory constructed, integrated, and tested prior to deployment, which drives down overall electrical infrastructure capital expenses by as much as 25% vs. a conventional build in some cases.7 Reliability is also inherent in these solutions derived from trained professionals performing repeated tasks they can only learn from and improve on.
Is my application a good fit for a modular design/build? The benefits of a modular data center approach are so great, and the variety of products so wide, that we believe every data center build or expansion project should evaluate a modular approach. The level of benefits will of course vary depending on the particulars of the project, but in our experience, modular data centers have thrived in a number of scenarios:
- Greenfield deployments where the operator was looking to maximize energy efficiency and density
- Expansion projects where the data center could not be expanded in traditional ways
- Niche solutions where a modular approach was needed to meet a specific schedule or space constraint
We have also seen several proposed deployments that turned out to be poor fits for modular solutions after thorough evaluations, especially with outdoor containerized solutions that present unique challenges. While these issues can be overcome, they need to be considered in the total cost of the deployment project and may swing the cost analysis.
The common reasons to reject a modular deployment include:
- Incompatibility with local building codes
- Lack of available outdoor space
- Difficult and/or expensive access to usable space during installation (staging, craning, rigging, etc.)
- Difficult and/or expensive connections to power, water, cooling, etc.
What should I think about in terms of capital and operating costs? In the past, customers typically built out all of the physical infrastructure they believe they needed and then held off on populating the data center with racks and servers as they waited for their customer base to grow. While prudent, operators still need to fund the cost of the land, design and architecture, and construction of their entire physical shell before a dollar of revenue comes in.
A modular approach enables the customer to expand the physical infrastructure, power and cooling infrastructure, and IT load at blocks of capacity based on their actual growth needs. Customers are then able to defer large capital outflows normally incurred at the start of the project into smaller, more palatable stages (Figure 3).
Beyond timing, modular approaches are in most cases less expensive to deploy than traditional data center builds. 451 Research estimates operators can save up to 30% in capital costs relative to a traditional build.8 This calculation is based on the collection of estimated build costs for traditional and modular designs and percentage of capital expense savings and calculating bottom up estimates of potential savings. Moreover, Rosendin Electric claims a modular data center can reduce construction costs by up to 20% vs. a conventional build and can be 40% less costly than the recommended guidelines provide by Uptime Institute.9
For operating costs, let’s consider power usage effectiveness (PUE) and how modular can impact this number. Typical PUEs average about 2.0, which means data center providers are paying for just as much in losses as they are for total IT server loads. This is a result of electrical line losses, transformers, UPS losses, and cooling required to keep the data center at a sufficient operating temperature. Modular data centers can significantly reduce PUE values because they are designed as unoccupied spaces whose environments can be optimized for energy efficiency.
In fact, modular data center users have reported PUEs under 1.10 while utilizing “free cooling” economizers. These PUE levels are not just for fair weather job site locations either. One of the lowest PUE values reported to date has come from the Phoenix area where the customer recorded an average annual PUE of less than 1.10.10 The savings in utility costs add up quickly and along with the initial capital expense reductions can significantly decrease time required for the data center to receive a return on initial investment. As a result, this reduces overall total cost of ownership.
What work needs to be done on-site? One of the advantages of modular implementation is that site preparation and offsite modular construction can occur simultaneously. This can almost make the lead time of the modular equipment irrelevant in that the modules can typically be built in less time than the customer can obtain a construction permit, perform preliminary site work, and pour concrete pads.
One main idea behind modular design/build is to eliminate the unknowns in installation and integration time and to ensure minimal discrepancies in startup and testing on-site. These are tangible benefits of building the modular enclosure in a controlled environment by experienced personnel that are continually working to improve the constructability and reliability of the product. On-site startup and commissioning benefit from the factory integration work and factory startup and testing performed before the product ships. Startup procedures for both factory and field startup are aligned to ensure no issues exist when the product is started up on-site. Once started up, on-site personnel will assist the customer’s third-party commissioning agent with any tests or operations in the commissioning script.
Keep in mind skilled engineers and contractors are still required to incorporate the equipment and complete the installation. Although factory testing will alleviate much of the problems usually incurred, the fact remains this is still an on-site construction project complete with site inspections, third-party commissioning agents, and authority having jurisdiction (AHJ) approvals. The modular approach helps to improve, but does not eliminate the required interactions and involvement of all of these entities (Figure 4).
What should I think about when it comes to a modular vendor? It seems like more players are entering the modular market these days, which can make it challenging to differentiate the good ones from the not so good ones. Costly, unplanned expenses in deliveries, site installation, and local AHJ approvals can be avoided by choosing an experienced modular supplier. While the initial idea of stuffing electrical equipment into a one-time use shipping container seems like a way to save money, we have found that experienced modular packagers prefer to build purpose-built enclosures so they can easily integrate them and size them for National Electrical Code (NEC) required clearances, which comply with UL standards and certifications. Once the units are ready to ship, they then must be delivered.
Many inexperienced modular packagers find out the hard way about the structural integrity of the enclosure when the unit is populated and is craned onto a truck for delivery. Weight distribution and center of gravity play a large role when the item is in the air. Experienced riggers and the appropriate hardware are needed to ensure the enclosure is not damaged when being loaded onto a truck. The trucking company must also follow designated routes to ensure the enclosure is not damaged by low underpasses on the way to the project site.
With the unit on-site, does your enclosure manufacturer have construction experienced personnel to aid in the setting and connection of the modular infrastructure? Experienced vendors usually deploy a project manager that attends project meetings and aids in the planning and installation of the site infrastructure required for connectivity of the modular components. The AHJ can be another obstacle when deploying a modular enclosure. Based on experience, AHJs have viewed these enclosures as buildings which can derail the approval of the installation. Seasoned enclosure manufacturers know the procedures and can provide the documentation to satisfy the local AHJ.
Before you take the first steps down the modular road, make sure your vendor has the tools for producing a successful manufactured product. Experienced modular vendors will follow ISO procedures to ensure their products can be manufactured in a repeatable manner. The supplier will incorporate ongoing improvement processes such as lean sigma which will help improve your product and your price.
Possessing the right tools for the job is also essential. Make sure your vendor has the correct equipment to load and offload the enclosures and that they have the facilities to work on these large enclosures in a climate controlled environment. The standard manufacturing process can be applied to large modular enclosures and your supplier should have sub-assembly stations equipped with the required tools and parts to complete and test each subassembly.
Safety practices are another important factor to consider. Your product can be affected if the supplier’s factory is shut down because of an accident. Avoid delivery delinquencies by selecting a supplier that has a strong emphasis on manufacturing safety.
Ongoing maintenance support will be critical in ensuring your modular data center and/or infrastructure has been serviced and maintained per the manufacturer’s standards. Make sure your modular vendor has adequate representation in your area to enable a quick response time and preferably without third-party support personnel.
The design and construction of the data center must evolve and keep pace with trends in population, big data and applications, data storage and management, and energy consumption. With all of this progressing at breakneck speeds, the traditional approach to data center construction can take too long, cost too much, and is inflexible.
Fresh, forward thinking approaches like modular design can help operators better manage the infrastructure supporting their data center requirements and achieve their availability, efficiency, and budget goals.
Consider the above questions before choosing a modular design/build to help evaluate if this approach is right for you. These five questions could potentially save you time, money, and maybe even a few sleepless nights as you plan for future data center capacity.
Patterson, David, Thomas Anderson et. al., “A Case for Intelligent RAM: IRAM,” IEEE Micro. March/April 1997: Vol. 17, No. 2.
Malik, Kevin. “The Tipping Point: Data Center 2.0.” Gartner Data Center Conference. December 2012: slide 15. http://www.slideshare.net/IOSlideshare/gartner-data-center-las-vegas-2012-km.
Fusion-io.“Taming the Power Hungry Data Center.” Whitepaper. p.10.
Internet World Stats. http://internetworldstats.com/stats.htm.
Evans, Dave. “The Internet of Things: How the Next Evolution of the Internet is Changing Everything.” Cisco Internet Business Solutions Group. April 2011:3.
This figure is based on approximate duration of time to deploy modular infrastructure and modified to focus on build out of a conventional brick and mortar data center electrical room comparison. Data provided by a San Jose, CA- based electrical contractor.
Up to 25% savings in capital expenditures based on total installed cost of Uptime Institute certified Tier III 500 kW system including equipment, land, building, installation, wiring and cabling, etc. Data provided by Dallas-based electrical contractor using market rates.
451 Research. “The Economics of Prefabricated Modular Data Centers.” May 2012:23.
Mazzetti, Bill. “The Pros and Cons of Modular Data Centers.” Rosendin Electric blog. Oct. 27, 2011: slide 12. http://datacentergurublog.rosendin.com/2011/10/the-pros-and-cons-of-modular-data-centers/.
Miller, Rich. “eBay Containers Push Frontiers of Efficiency.” Data Center Knowledge. Feb. 27, 2012. http://www.datacenterknowledge.com/archives/2012/02/27/ebay-containers-push-frontiers-of-efficiency/.