VM Migration Performance
In the context of the project "Towards an Architectural Pattern for Pro-Active Virtual Machine Migration" we investigated the performance of virtual machine (VM) migration for different virtualization solutions (VMware vSphere, Citrix XenServer, KVM) under various load situations. Thereby we concentrated on two metrics, the overall migration time and the blackout time of the VM. The migration time is the time from requesting the hypervisor to migrate the VM until it reports the successful migration. The blackout time is the time, the VM is not responsive to network I/O due to the migration. It can be significantly less than the migration time, because most hypervisors support some kind of live migration, which minimizes the blackout time by ballooning, precopying or incremetal copying the VMs state.
We developed a testing framwork consisting of several load generators and test scripts which implement the measurement procedure.
CPU load generator (CLG)
To generate a certain CPU load we use burnP6 from the cpuburn (P6 version) suite in conjunction with cpulimit (both Linux tools, also available as Debian packages).
Locked pages generator (LPG)
To increase the physical memory utilization of the VM, the tool
allocates a given amount of memory, writes garbage data to it and
locks it in physical memory using the accordant system call (such as
mlock in Linux). The
migration is started after the tool has allocated and locked the
Dirty pages generator (DPG)
This tool is used to generate dirty pages during the migration by continuous memory writes. This increases the load for live migrations and therefore influences blackout time, because the VMs state continuously changes during the migration.
All tests are run on hardware provided by the HPI FutureSOC lab:
2x FUJITSU PRIMERGY RX300 S5
- Intel Xeon E5540 @ 2.53GHz (4 cores + HT)
- 12GB RAM
- 2 Gigabit-NICs
ResultsThe following charts compare the results for:
- VMware vSphere 4.0
- Xen Server 5.6
- KVM (Kernel 2.6.32)
Migration / Blackout Time vs. CPU Load
- Based on CLG tool
Migration / Blackout Time vs. VM size
Migration / Blackout Time vs. Physical Memory Usage
- Based on LPG tool
Migration / Blackout Time vs. Dirty Pages
- Based on DPG tool
For further information about this research project, please contact Peter Tröger.