A blog about programming topics, mostly JVM, Linux kernel, and x86 related things.

Sunday, April 11, 2010

CPU virtualization techniques

I guess most technical people have heard of virtualization by now and many are using a hypervisor such as QEMU for their daily work but don't really understand how they work. I'll try to give a brief overview of the major techniques here but please be warned that this is very x86 and Linux centric view of things.

The two fundamental questions in virtualization are: do you need to be able to run unmodified guests and how fast do you want the guest to go? Unmodified guest is, rather unsuprisingly, your out-of-the box Linux or Windows installation, for example. To be able to run them under a hypervisor, you need either hardware virtualization or hardware emulation. If you are allowed to modify the guest, you can use paravirtualization. Roughly speaking, KVM falls under the hardware virtualization category, QEMU under the hardware emulation, and Xen under paravirtualization.

Hardware emulation is the purest form of virtualization in a sense that it supports unmodified guests without any hardware assistance. Hardware emulators are actually not that different from say, the Java virtual machine. In both cases, you have an instruction set (machine code vs. bytecode) and some way to describe machine state (machine registers vs. operand stack). And much like with the JVM, hardware emulation can be pretty fast if dynamic binary translation (a form of Just-in-time compilation) is used as shown by QEMU, for example. Unfortunately on virtualization unfriendly architectures such as the x86 [1], some aspects of CPU emulation incurs serious performance overhead.

[ 1. See Section 3.1 ("x86 obstacles to virtualization") of Adams and Agesen (2006) for details. ]

Hardware virtualization also supports unmodified guests just like hardware emulation does but provides near-native performance because the hypervisor doesn't need to do binary translation. With the virtualization extensions in modern x86 CPUs, the only thing you need to do is to put the CPU in "virtualization mode" and let the guest operating system run on it. The hypervisor needs to, of course, provide I/O emulation if you want to run anything serious under it. On Linux, KVM provides the application binary interface for controlling the virtualization extensions. KVM is usually used in combination with QEMU which handles rest of the hardware emulation to implement a full hypervisor.

Paravirtualization is a technique to support high performance virtual machines on virtualization unfriendly architectures. It requires the guest operating system to be ported for the paravirtualization ABI. On Linux, this ABI is called "paravirt-ops" and it's used by the Xen hypervisor, for example. Although I tend to think Xen as a historical accident (the project started before Intel and AMD introduced their virtualization extensions), it is being used in large scale environments such as Amazon EC2 and scalability benefits seem to be real.

While the boundaries between different virtualization techniques are pretty clear on the CPU virtualization side, for real world problems they tend to be not so black and white. You almost certainly want to use paravirtualized device drivers for hardware virtualization solutions such as KVM if you're interested in high performance I/O and make sure your paravirtualization solution such as Xen takes advantage of the CPU virtualization extensions when they're available. That shouldn't come as a huge surprise as in all technology, practice beats theory every time.

No comments: