The term "central processing unit" (CPU) has been around since the 1950s. At the heart of every computer or smart device sits the CPU, tirelessly executing traditional applications. Then, the graphical processing unit (GPU) was introduced in the '90s — originally created for the special task of accelerating on-screen computer graphics for engineering workstations, video processing, and gaming to keep up with the extremely demanding expectations of the human visual system. We owe our survival to our ability to be immediately alert to the slightest movement or visual anomaly — and that has always presented a severe challenge to cinema, television, or computer screen design. Whereas CPUs are designed to process a linear stream of data, GPUs are designed to process a multidimensional experience with space, color, and time. In theory, CPUs could handle this, but, in practice, it would be far too slow and imprecise. The fascinating thing about GPUs is that this multidimensional processing skill turns out to be much closer to the requirements of machine learning and big data processing. Something originally created to make computer games more vivid has become a vital part of digital transformation. And, now, it’s time to introduce data processing units (DPUs). Sure, CPUs and GPUs already process data, but DPUs do it differently, managing the way data moves through the data center. And, yes, every DPU has its own CPU.
Consider a high-level international diplomatic conference. In the interests of total accountability and accuracy, one might insist that every speaker’s words be transcribed in their native language and passed to every other participant, who then has the job of translating those exact words into their own language. But, it doesn’t work like that. Instead, we use interpreters. And while translators have time to study dictionaries, interpreters do their jobs on the fly. Experienced interpreters do not simply provide accurate translations; the job requires other skills. First, there must be an understanding of national protocols. An Australian speaker might use the word “mate” as a friendly gesture, but interpreters know there will be less diplomatic friction if mate is replaced by “your esteemed highness" to the attending audience. Then, there is the understanding that phrases familiar in one culture might need explanation for others. There’s also a need to accelerate communication by editing out “ums” and “ers,” dramatic pauses, and rambling personal anecdotes. There can even be some subtle encryption — for example, “our European competitors” might be encoded as “our European allies.” Finally, in the interests of peace and global security, some utterances might simply be best forgotten. Instead of lumbering every delegate with the task of analyzing pages of transcript, a team of skillful interpreters can make sure the essential messages are conveyed in the most economical form to those who need to know while keeping a check on malicious messages. The delegates, spared a load of translating and protocol headaches, can concentrate more on high-level diplomacy. In traditional data centers, networks are no more than a web of cables linked by switches to deliver data to and from the CPUs, where all of the processing takes place. In modern data centers, networks are connected through DPUs that process the data on the fly to reduce the load on the CPUs and free them up for their intended application processing. They play a role not unlike a team of very professional interpreters. So, what is in the DPU?
DPUs are a new class of programmable processors — systems on chips (SOCs) that combine three elements:
- An industry-standard, high-performance, software-programmable, multi-core CPU.
- A high-performance network interface capable of parsing, processing, and efficiently transferring data at network speed.
- A rich set of flexible and programmable acceleration engines designed to offload networking tasks and optimize application performance for AI and machine learning, security, telecommunications, storage, etc.
That third component is especially significant because it can be compared with the skills of the perfect interpreter described above: a set of functions that, together, serve to optimize efficiency by making sure the right data goes in the right format to the right place along the fastest and most reliable route. It can also support encryption as well as identify anomalous traffic or malware and even initiate an appropriate response. Some DPU vendors may rely on proprietary processors, and others expect the CPU to do all the work, but unless all three of these elements are present, the device should not be called a DPU. DPUs are usually built into a network interface controller (NIC) to create what is called a smartNIC. SmartNICs are key components of next-generation data centers, the kind needed to power the data revolution that has been called “The Fourth Revolution.”
The Fourth Revolution
Some historians have identified three key revolutions in the evolution of humanity:
- The Cognitive Revolution, around 70,000 BCE, defined the birth of language and the ability to communicate.
- The Agricultural Revolution, around 10,000 BCE, defined the ability to domesticate farm animals and cultivate crops to support the rise of cities.
- The Scientific Revolution, during the early modern period, laid the foundation for the modern way of life with developments in math, physics, astronomy, and sciences
The Fourth Revolution is AI that can perform tasks with superhuman levels of performance. Advanced machine learning can now organize, process, and extract subtle but valuable information from mountains of data gathered from millions of users on the internet and the IoT. This marks a seminal step toward developing true AI and the extraordinary possibilities this enables. Even with the benefit of GPUs to crunch the data, progress has been held up by the extreme burden it places on the CPUs to access and share information between computers and keep the GPUs fed with data. A key development has been to make the network even faster, but this has overwhelmed CPUs with data management tasks. With DPUs, the network is smarter (like that highly professional team of interpreters), so it becomes an active agent in the overall data processing. DPUs bring the processing closer to the data itself, and they make the network act like a coprocessor, offloading labor from the central compute engine. Digital transformation is the business application of the Fourth Revolution. That is why we will soon be hearing a lot more about DPUs.