of 44

Comparison of Open Source Virtualization Technologies

Published on June 2016 | Categories: Types, Research, Internet & Technology | Downloads: 29 | Comments: 0
455 views

Comments

Content

Fernando Laudares Camargos

Gabriel Girard Benoit des Ligneris, Ph. D.
[email protected]

Comparative study of Open Source virtualization & contextualization technologies
© Révolution Linux 2008. Ce document est confidentiel.

Context (1)

     

Introduction Why virtualize the server infrastructure Virtualization technologies The experiments Explanations & anomalies Which technology is best for ... you ?

© Révolution Linux 2008. Ce document est confidentiel.

Context (2)



Research executed by Fernando L. Camargos in pursuit of his Masters degree in Computer Science under the direction of Gabriel Girard (Université de Sherbrooke) and Benoît des Ligneris (Révolution Linux) This being a research work, some questions remain unsolved... maybe you can help !



© Révolution Linux 2008. Ce document est confidentiel.

Why virtualize the server infrastructure (1) ?
Server consolidation is the most mentionned argument

© Révolution Linux 2008. Ce document est confidentiel.

Why virtualize the server infrastructure (2) ?
             

Reduction of the purchase and maintenance costs Compatibility with legacy applications and OSs Security : environment to execute untrusty applications Low cost environment for software development Centralized control/management Easy backup/restore procedures Live migration Quick server fail-over High availability Virtual appliances Controled sharing of ressources Cloud computing Hardware abstraction It's ... cool !

© Révolution Linux 2008. Ce document est confidentiel.

Virtualization technologies (1)



Hardware emulation Binary translation Classic virtualization



Full virtualization Para-virtualization









OS-level virtualization (contextualization)

© Révolution Linux 2008. Ce document est confidentiel.

Virtualization technologies (2)


Hardware emulation binary translation Classic virtualization

KQEMU VirtualBox KVM



Full virtualization







Para-virtualization OS-level virtualization (contextualization/ containers)

Xen Linux-VServer



OpenVZ

© Révolution Linux 2008. Ce document est confidentiel.

Virtualization technologies (3)

virtualization != emulation
QEMU is an emulator

© Révolution Linux 2008. Ce document est confidentiel.

Virtualization technologies (4)

Virtualization technologies

partial emulation

no emulation

KQEMU

KVM

VirtualBox

OpenVZ

Xen (Linux)

Linux-VServer

© Révolution Linux 2008. Ce document est confidentiel.

Virtualization technologies (5)
2 types of hypervisors:

Hypervisors type I: KVM, Xen  Hypervisors type II: VirtualBox, KQEMU

© Révolution Linux 2008. Ce document est confidentiel.

The experiments (1)

virtualization layer

overhead

But of how much ? To discover, we need to mesure the efficiency of the virtualization technologies where : efficiency = performance + scalability

© Révolution Linux 2008. Ce document est confidentiel.

The experiments (2)

2 types of experiments: 1) Performance (overhead): one virtual machine only 2) Scalability: several virtual machines

© Révolution Linux 2008. Ce document est confidentiel.

The experiments (3)
Virtualization solutions evaluated in this study
Virt. solution Version Host Linux-VServer Xen KVM OpenVZ KQEMU VirtualBox -2.2.0.5 3.1.0 58 5.1 1.3.0~pre11-6 Server kernel 2.6.22-14-server 2.6.22.14 2.6.22-14-xen 2.6.22-14-server 2.6.22-ovz005 VMs kernel -2.6.22.14 2.6.22-14-xen 2.6.15-26-amd64 2.6.22-ovz005

2.6.22.14-kqemu 2.6.15-26-amd64 2.6.22.14

1.5.4_OSE / 1.5.51_OSE 2.6.22-14-server

Test bed:


Intel Core 2 Duo 6300, 1.86GHz (x86_64 / VT-x) 4G Memory Hard drive SATA 80G

Chosen OSs:


Host: Ubuntu 7.10 VMs: Ubuntu 6.06







© Révolution Linux 2008. Ce document est confidentiel.

The experiment (4)
Bits & Bytes & VMs :


64 bit kernel for all technologies Use of VT extension for KVM, Xen 32 bit VM for VirtualBox Identical memory allocation per VM for every technology but Vserver : 2039 Mo







© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (1)
Methodology :


7 benchmarks (different workloads)
 

Reference : executed in the Linux host (scale = 1) executed inside the virtual machines



4 execution sets


results = the average of the 3 last sets normalized by the result obtained by the Linux host

© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (2)
Compilation of the Linux kernel


An equilibrated workload – a little bit of everything without stressing one particular ressource too much Metric: given time for the completion of the compilation
tar xvzf linux-XXX.tar.gz cd linux-XXX make defconfig # ("New config with default answer to all options") --date +%s.%N && make && date +%s.%N ... make clean date +%s.%N && make && date +%s.%N 3x ...



© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (3)

Compilation of the Linux kernel
1

0.9
Li nux-VServer Xen OpenVZ KVM Virtual Box (-hwvirtex off) KQEMU Virtual Box (-hwvirtex on) QEMU

0.8

0.7

0.6

Nomalized time (kernel vanilla)

0.5

0.4

0.3

0.2

0.1

0

© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (4)
Bzip2


Software for file compression Using option that yields maximal compression which considerably increases the memory utilisation per process Metric: given time for the completion of the compression





cd /var/tmp cp /home/fernando/Iso/ubuntu-6.06.1-server-i386.iso . date +%s.%N && bzip2 -9 ubuntu-6.06.1-server-i386.iso && date +%s.%N rm ubuntu-6.06.1-server-i386.iso.bz2 ...

4x

© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (5)
Bzip2
1.00

0.90

0.80

0.70

0.60

Xen Li nux-VServer VirtualBox (-hwvirtex on) Vir tual Box (-hwvirtex off) KVM KQEMU OpenVZ QEMU

Normalized time (kernel vanilla)

0.50

0.40

0.30

0.20

0.10

0.00

© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (6)
Dbench


Derived from the Netbench benchmark Emulates the load imposed in a file server by n Windows 95 clients n(umber of clients)=100, t(ime)=300 Metric: throughput (Mb/sec)
/usr/local/bin/dbench -t 300 -D /var/tmp 100 # 4x







© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (7)
Dbench
1.20

Normalizes throughput (kernel vanilla)

1.00
Linux-VServer Xen OpenVZ KVM KQEMU QEMU

0.80

0.60

0.40

0.20

0.00

* no results for VirtualBox
© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (8)
dd


Application for low level (bit by bit) data copy Mesures the performance of the I/O system (hard drive access) 2 tests: copy of a single big file  copy of 60G of /dev/zero to /dev/null Metric : throughput






...



dd if=/opt/iso/ubuntu-6.06.1-server-i386.iso of=/var/tmp/out.iso ... dd if=/dev/zero of=/dev/null count=117187560 # 117187560 = 60G ... rm -fr /var/tmp/* # between execution sets ...
© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (9)
dd (copy of ISO file)
1.4

1.2

1.0
Li nux-VServer Xen KVM OpenVZ

Normalized throughput (kernel vanilla)

0.8

0.6

0.4

0.2

0.0

* no results for KQEMU nor VirtualBox
© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (10)
dd (60G /dev/zero --> /dev/null)
1

0.9

0.8

0.7
Li nux-VServer KVM OpenVZ Xen

0.6

Normalized throughput (kernel vanilla)

0.5

0.4

0.3

0.2

0.1

0

* no results for KQEMU nor VirtualBox
© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (11)
Netperf


A benchmark that can be used to measure several aspects of the network performance TCP Stream test: measure the speed of the exchange of TCP packets through the network (10 sec.) Metric: throughput (bits/sec)
netserver .. netperf -H <nom du serveur> # in the server # in the client, 4x





© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (12)

Netperf (TCP Stream Test)
1.2

1

Normalized throughput (kernel vanilla)

0.8

Vir tual Box (-hwvir tex off) Vir tual Box (-hwvir tex on) Li nux-VServer Xen OpenVZ QEMU (-no-kqemu) KQEMU KQEMU (-kernel-kqemu) KVM

0.6

0.4

0.2

0

© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (13)
Rsync


Similar to Netperf's TCP Stream Test, measures the performance of file exchange through the network 2 tests:
 



ISO file: 1 big file (433M) Linux kernel tree: several small files (294M)



Metric: time (sec.)

.. date +%s.%N && rsync -av <server>::kernel /var/tmp && date +%s.%N ... date +%s.%N && rsync -av <server>::iso /var/tmp && date +%s.%N ... rm -fr /var/tmp/* # between execution sets ...
© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (14)
Rsync (kernel tree)
1.2

1.0

0.8

OpenVZ Virtual Box (-hwvirtex on) VirtualBox (-hwvirtex off) Li nux-VServer Xen QEMU KQEMU (-kernel-kqemu) KQEMU KVM

Normalized time (kernel vanilla)

0.6

0.4

0.2

0.0

© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (15)
Rsync (ISO file)
1.2

1.0

0.8

OpenVZ Virtual Box (-hwvirtex on) KQEMU VirtualBox (-hwvirtex off) Li nux-VServer KQEMU (-kernel-kqemu) Xen QEMU KVM

Normalized time (kernel vanilla)

0.6

0.4

0.2

0.0

© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (16)
Sysbench


Measures the performance of a DB server Workload centered in I/O operations in the file system Metric: throughput (transactions/sec)
# (1x) # (4x)





sysbench --test=oltp --mysql-user=root --mysql-host=localhost --debug=off prepare sysbench --test=oltp --mysql-user=root --mysql-host=localhost --debug=off run

On-Line Transaction Processing T P
OLTP test statistics: queries performed: read: write: other: total: transactions: deadlocks: read/write requests: other operations: 140000 50000 20000 210000 10000 (376.70 per sec.) 0 (0.00 per sec.) 190000 (7157.28 per sec.) 20000 (753.40 per sec.)

© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (17)

Sysbench
1 0.9

Normalized throughput (kernel vanilla)

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

Linux-VServer Xen KVM OpenVZ VirtualBox (-hwvirtex on) VirtualBox (-hwvirtex off) KQEMU KQEMU (-kernel-kqemu) QEMU

© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (18)
Conclusion :


Linux-VServer: excellent performance. Has presented minimal to no overhead when compared to Linux. Xen: has shown a great performance in all but the Dbench benchmark (I/O bound benchmark). KVM's performance was fairly good for a full virtualization solution but should be avoided to run applications that strongly rely on I/O.





© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Performance (19)
Conclusion (cont) :


OpenVZ has shown a very variable performance (from weak to excellent). Certainly because of accounting for I/O and because of some network optimization for the Network related tests. VirtualBox has presented a good performance for the file compression and network based benchmarks. Poor performance for all the other situations. KQEMU has shown a poor performance for all benchmarks. It is clear that this virtualization solution does not make a good use of the available ressources and its application in production servers should be avoided.





© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Scalability (1)
Methodology:


1 benchmark (Sysbench, kernel compilation) executed by n VMs concurrently n = 1, 2, 4, 8, 16 et 32 4 execution sets:






results = average of the last tree execution sets
n 1 2 4 8 16 32 Mémoire (Mb) 2039* 1622 811 405 202 101



Memory allocation per VM :

* 1536 Mb (KQEMU)
© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Scalability (2)

Sysbench at scale: average throughput per VM
Throughput (transactions/sec)
400 350 300 250 200 150 100 50 0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
Linux-VServer Xen KVM OpenVZ VirtualBox KQEMU

Number of VMs (n)

© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Scalability (3)
Sysbench at scale: total throughput
Throughput (transactions/sec)
700 600 500 400 300 200 100 0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
Linux-VServer Xen KVM OpenVZ VirtualBox KQEMU

Number of VMs (n)

© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Scalability (4)

Kernel compilation (VMs average)
8000 7000

time (seconds)

6000 5000 4000 3000 2000 1000 0 1 2 4 8 16 32
Host Vserver Xen

number of VMs (n)

© Révolution Linux 2008. Ce document est confidentiel.

The experiments – Scalability (5)

Kernel scalability
25 20

Kernel / Hours

15 10 5 0 1 2 4 8 16 32

Host Vserver Xen

Number of VM

© Révolution Linux 2008. Ce document est confidentiel.

Conclusion - Scalability (1)
Conclusion :


The efciency of virtualization solutions is strongly related to the number of VMs executing concurrently (scalability) Most of the time, one additional VM helps to get the maximum performance out of a given server (link with the number of CPU) More decreases performance as a bottleneck is limiting performance (CPU/core number important !) Linux-VServer has shown the best global performance for up to 5-7 VMs. Xen has proved to be the best full virtualization based solution.









© Révolution Linux 2008. Ce document est confidentiel.

Conclusion – Scalability (2)


KVM has shown a reasonable performance for a full virtualization solution. OpenVZ's performance was not what we would expect of a contextualization solution. Our hypothesis is that the accounting (beancounter) is the root cause of th overhead VirtualBox has shown an impressive performance, the total throughput has more than doubled when the number of VMs has pass from 8 to 16. However, we were unaible to execute this experiment with 32 VirtualBox VMs executing concurrently. KQEMU's performance was weak when compared to all other solutions, independently of the number of VMs in execution.







© Révolution Linux 2008. Ce document est confidentiel.

Which technology to use in each case ?
A technological point of view ...


OpenVZ : pure network related applications, thanks to the optimizations done in the network layer. Not indicated for I/O applications. Linux-VServer: all kinds of situations (a priori). Xen can also be used in all kinds of situations but requires important modifications in the guest OS kernels OR the use of the VT virtualization instructions. KVM and VirtualBox have proven to be good options for development environments. KQEMU has shown weak performance. It is indicated for development only.

 





© Révolution Linux 2008. Ce document est confidentiel.

Conclusion : Which technology to use in each case ?


Results are very different for every benchmark/technology ,so benchmark the technology you plan to use with your own mission critical application BEFORE virtualizing your servers (ex: File Servers benchmark). Only Xen is actually supported by the industry (RedHat, SuSE, Mandrake, IBM, etc.) KVM is available in the standard Linux kernel : Yeah! But poor performance overall ;-( Linux-VServer and OpenVZ: they both need a modified kernel that is not ofcially supported by the afore mentioned giants of the industry but . . .







© Révolution Linux 2008. Ce document est confidentiel.

Future / contextualisation


Since last OLS, key players like IBM, Intel, and Google are working hard to include a container based technology in the Linux kernel Lots of patches from OpenVZ gets integrated into the kernel recently and everyone expects that we will have Really Soon Now a contextualization in the linux kernel without the need for any kernel hacking We strongly believe that the integration of a contextualization/container solution is the best way to go for Linux-on-Linux virtualization needs It will offer VMWare a very strong and completely open-source competition







© Révolution Linux 2008. Ce document est confidentiel.

© Révolution Linux 2008. Ce document est confidentiel.

Sponsor Documents


Recommended

No recommend documents

Or use your account on DocShare.tips

Hide

Forgot your password?

Or register your new account on DocShare.tips

Hide

Lost your password? Please enter your email address. You will receive a link to create a new password.

Back to log-in

Close