What We Forgot About Server Virtualization
Remember back 10 years ago when there was still a question as to whether you should virtualize your data center or not? Back then, there were a lot of interesting security arguments levied against virtualizing servers. Many of those arguments were the standard fear, uncertainty, and doubt surrounding any new technology adoption. However, there were also some really relevant concerns that we’ve forgotten, but probably shouldn’t have.
One of the most important of these concerns has to do with isolation. Are virtual machines (VMs) truly isolated if they are running on the same hypervisor? Does a guest VM pose a threat to the host and other VMs running on that host?
Although there have been occasional vulnerabilities discovered that allowed escape from a guest to a host, in general, this concern hasn’t manifested in any wide scale breaches. However, you should still design with this risk in mind, particularly for systems hosting confidential data and guests that bridge different security zones.
If possible, group these machines together to minimize exposure in an attack. Raw access to resources increases that risk, so be careful of granting this level of access. Most of the security concerns are not about direct jumping from guest to guest, or at least no more than the standard attacks that applied to physical hosts, but the concern is about exposure to the hypervisor. If an attacker can gain even marginal control or data from the host, then you should consider all of the guests compromised. This may sound extreme, but it’s the reality of the security model in a virtualized data center.
Since the host is so critical to your overall security architecture, it’s also important to manage it with secure protocols, limit who has network and account access, audit that access, and keep it up to date with patches. If an attacker gets full control of a single host, not only will they have full control and access to the guests running on that host, but they will likely have very broad network access, too.
Since most hosts contain VMs performing different functions, multiple VLANs are often trunked in. To make things easy and limit the requests on network teams, the full set of VLANs will often be setup, even if the current guests are only using a small subset. As network virtualization gains traction, this will only continue. Again, this is an important time to think about what access each VM actually needs and which network they need to be on. If an attacker has access to management VLANs or other sensitive networks, they have easily compromised the entire network, not just the single host.
IMAGES AND SNAPSHOTS
A second concern that wasn’t fully appreciated at the time but that has grown as a real risk is the security of images and snapshots. Although the technical threat was understood, there was little appreciation for the sheer number of these files floating around.
If an image is compromised, it’s the equivalent of someone powering off a server and walking out of your data center with it. That’s bad, and it’s much easier to do than ripping a production system off the shelf and making a run for it.
Snapshots are equally dangerous because they can contain the data running in memory at the time of the snapshot. Credentials are the biggest concern for leakage here, but any data handled by the machine is at risk.
We often talk about VM sprawl from the powered on machines perspective and forget about all of those images and snapshots lying around. You need to define a clear plan for where those files are stored, who has access, how the access is audited, and when you should delete the old snapshots. Storage costs aren’t always the driving factor for better image and snapshot management, but security certainly should be.
The third and final security issue with server virtualization to consider is how it enables insecure legacy technology to remain in your organization. This is one of the big benefits of virtualization, but needs to be managed properly. That application that only runs on Windows XP and hasn’t been or can’t be patched is a huge hole in your defenses.
If you can’t migrate to a more secure solution, make sure you have walled off such servers and applications as much as possible. You might need to use local account privileges so it doesn’t have access to the domain, and it definitely should be segmented from a network perspective as much as possible. If the machine doesn’t need internet access to function, you should not allow it to connect outbound. It might take a few extra steps for users or administrators to access, but it is well worth it given the security risk old operating systems and applications pose.
Server virtualization is not only here to stay, but it will continue to expand through the stack into fully virtualized data centers. By understanding the principles of virtualization as a technology, and recalling the initial concerns we had when server virtualization as we know it now was the trendy new technology, we can better manage and secure our virtualized data centers.
Just because it’s common now, doesn’t mean we can forget our original concerns and assume all of the problems have been solved. As we enter a time with hyper-converged data centers, remember the journey and apply those early lessons in virtualization as complexity increases.