VM Migration Performance

Introduction

In the context of the project "Towards an Architectural Pattern for Pro-Active Virtual Machine Migration" we investigated the performance of virtual machine (VM) migration for different virtualization solutions (VMware vSphere, Citrix XenServer, KVM) under various load situations. Thereby we concentrated on two metrics, the overall migration time and the blackout time of the VM. The migration time is the time from requesting the hypervisor to migrate the VM until it reports the successful migration. The blackout time is the time, the VM is not responsive to network I/O due to the migration. It can be significantly less than the migration time, because most hypervisors support some kind of live migration, which minimizes the blackout time by ballooning, precopying or incremetal copying the VMs state.

We developed a testing framwork consisting of several load generators and test scripts which implement the measurement procedure.

Load Generators

CPU load generator (CLG)

To generate a certain CPU load we use burnP6 from the cpuburn (P6 version) suite in conjunction with cpulimit (both Linux tools, also available as Debian packages).

Locked pages generator (LPG)

To increase the physical memory utilization of the VM, the tool allocates a given amount of memory, writes garbage data to it and locks it in physical memory using the accordant system call (such as mlock in Linux). The migration is started after the tool has allocated and locked the memory.

Dirty pages generator (DPG)

This tool is used to generate dirty pages during the migration by continuous memory writes. This increases the load for live migrations and therefore influences blackout time, because the VMs state continuously changes during the migration.

Hardware

All tests are run on hardware provided by the HPI FutureSOC lab:
2x FUJITSU PRIMERGY RX300 S5

  • Intel Xeon E5540 @ 2.53GHz (4 cores + HT)
  • 12GB RAM
  • 2 Gigabit-NICs

Results

The following charts compare the results for:
  • VMware vSphere 4.0
  • Xen Server 5.6
  • KVM (Kernel 2.6.32)
The migrated VM is always Linux-Guest with 2GB Ram. The machine is idle, except of the given load.

Migration / Blackout Time vs. CPU Load

  • Based on CLG tool

Migration / Blackout Time vs. VM size

Migration / Blackout Time vs. Physical Memory Usage

  • Based on LPG tool

Migration / Blackout Time vs. Dirty Pages

  • Based on DPG tool


For further information about this research project, please contact Peter Tröger.