For the better part of the last two decades, data center observers have been predicting increases in average rack density regardless of the fact that they have continuously been proven wrong. Yet, there are reasons to believe that those long-anticipated increases are headed for the racks now.
Many are probably thinking: “I’ve heard this before.” And that’s true. It wouldn’t take much time on Google to find examples of failed forecasts predicting higher rack densities. It hasn’t happened for a number of reasons, and some of those barriers still exist. But change follows demand, and demand for compute is, well … changing.
The advent of 5G; the relentless push toward the edge; and advanced applications and workloads related to AI, such as machine learning and deep learning, will make pockets of high-performance computing necessary and more common. It will start with single racks, but the early deployments are just a catalyst for a more widespread movement beyond the 4- to 6-kW averages that have been the industry norm since the turn of the century. More importantly, server vendors are pushing past density norms with aggressive technology innovation that will force data center managers to adjust.
To truly understand the case for high-density racks, it’s important to evaluate the reasons rack density has remained more or less static for the past 20 years.
The Stubborn Status Quo
According to Cisco, 51% of the global population had internet access in 2018, meaning there were 3.9 billion total internet users. They expect those numbers to reach 66%, or 5.3 billion users, by 2023. In fact, Cisco projects the generation of more IP traffic in 2022 than happened in the first 32 years of the internet era combined. This has been a pattern, more or less, since the introduction of the internet. But, despite the relentless, wildfire-like growth in computing demand, rack densities have remained remarkably consistent.
The 2019 Uptime Institute data center survey results indicated an average rack density of 6 kW, which is largely in line with longstanding industry averages. The stubborn stability of this number is unusual in an industry famous for fast, often incremental change across technologies, systems, and demand. It’s not an exaggeration to say that everything changes in the data center — everything, that is, except rack density. But why is that?
Let’s start with the evolution of server technologies, which for many years seemed to stay a step ahead of demand. As servers advanced and became capable of performing more and more computing, data center managers simply swapped out old servers for new ones or added additional racks when they wanted to add more computing capacity. Those servers produced more heat, but data centers managed that with traditional cooling systems.
This was the heyday of the enterprise data center, when real estate was readily available and standard practice was to build big facilities with plenty of room for growth — often far more room than needed. There was no reason to pack racks with servers, push density boundaries and tax traditional cooling systems. It was easier, safer, and cheaper to add low-density racks to those big, empty spaces. Eventually, many of those data centers started moving applications to the cloud, further eliminating any need for high-density racks.
Through it all was the understanding that increasing rack density at some point would require new approaches to power distribution and thermal management. When there were other ways to manage increasing compute demand, why bother? It’s a question that persists today.
I’m Your Density. I Mean … Your Destiny
A tongue-tied George McFly may have been, well, ahead of his time when he uttered those words in Back to the Future, but they apply to today’s data center. When it comes to increased rack density, despite all the false alarms over the years, it may just be meant to be.
It won’t happen overnight, and it won’t happen everywhere, but the data ecosystem is evolving in ways that seem to be favoring higher-density racks. Already, server vendors are developing higher-powered servers with unprecedented compute capabilities to support all kinds of advanced applications, and those technologies are going to force data centers to adapt. These aren’t incremental improvements. There are server vendors today who are looking for ways to cool racks that will reach 30 to 50 kW routinely and peak closer to 100 kW.
If server manufacturers are providing the pull toward higher-density racks, there is no shortage of factors providing the push — starting with an increasing premium on data center floorspace. Those big, roomy enterprise data centers are an endangered species, replaced instead by smaller, more efficient facilities at the hub of more complex and distributed hybrid networks. In these environments, there isn’t always room to add a rack when additional computing capacity is needed, so, at some point, the only choice is to add servers to existing racks.
Government and military applications and academic research efforts require high-performance computing and have been early adopters of higher-density architectures. Advanced analytics, AI, and consumer demand for things like high-definition video, and virtual reality will drive early investment. Gaming and e-sports are highly reliant on high-performance, low-latency computing, and those communities are active early movers. Eventually, even more critical applications, such as automated vehicles and intelligent transportation systems will tax these networks in ways the industry has only begun to understand.
And then, there’s edge- and 5G-driven telecom network densification, perhaps the two most significant computing trends of the day, converging to force more compute capacity into smaller spaces. Again, it will happen gradually, but as 5G networks expand and organizations start to leverage the capabilities of those networks, the computing at 5G sites is going to need to be significant. That will require higher-density racks due to the limitations of available real estate as these networks encroach further and further into neighborhoods and urban environments.
Demand drives decision-making, and that includes the decisions related to where to place compute workloads. As demand increases for services requiring low latency and high processing power, those workloads will shift to smaller spaces at the edge of the network.
Cisco predicts there will be 12 billion mobile devices and IoT connections globally by 2022, and 82% of all IP traffic will be video. That’s a reflection of the potential of 5G and a recipe that demands powerful computing capabilities closer to end users. The networks needed to support computing demands in 2022 will almost unavoidably rely on high-density computing at the edge.
Supporting High Density
Discussions of high-density are often peppered with qualifiers that suggest the trend remains somewhere beyond the current horizon, but there are plenty of signs that these densities are becoming more accepted. According to the Uptime Institute survey, 64% of respondents indicated they have racks housing at least 10 kW of computing, and one-third have racks of 20 kW or higher. It hasn’t yet reached the kind of mass that would shift industry averages, but it’s happening more often than most think.
These pockets of higher-density racks and the seemingly inevitable migration toward higher densities as a more mainstream solution raise important questions about the infrastructure needed to support such architectures. Cooling these racks will demand entirely different approaches from the norm. The options could range from rear door heat exchangers to direct liquid cooling at the chip — two technologies currently deployed in various high-density environments.
What is the threshold for such deviations from the norm? Anything above 4 kW starts to exceed traditional air-cooling solutions and requires more targeted thermal management. When do even advanced air-cooling systems become insufficient? 10 kW? 15 kW? It’s a little early to establish such clear guidelines, but as a larger sample of high-density deployments is formed, the picture will come into focus.
Bottom line: Although previous predictions of higher rack densities have stoked “boy who cried wolf” skeptics across the industry, technological advances and consumer demands are conspiring to make higher densities unavoidable. Expect more pockets of high-density computing to accommodate specific applications and widespread adoption as users gain access to 5G networks and capabilities. Forward-thinking organizations will begin planning accordingly to ensure their network infrastructure is ready when called upon.