Virtualization, GlusterFS, and automation

Finally I got some time playing with GlusterFS and libvirt.

I’m not going to step into GlusterFS setup as it’s really easy and straight forward, I did CentOS kickstart (non-attendance) installation as well since it may help me in future deployment. Note that I assume you have glusterd running the host node, other all localhost below should be replaced by the hostname that is running glusterd.

First, create the kickstart file somewhere accessible through HTTP, assuming it is http://192.168.0.1/kickstart.txt:
[raw]
install
cdrom
text
lang en_US.UTF-8
keyboard us
network –onboot yes –device eth0 –bootproto dhcp –noipv6
timezone –utc America/Los_Angeles
rootpw –iscrypted SOMETHING-COPIED-FROM-/ETC/SHADDOW
selinux –disabled
authconfig –enableshadow –passalgo=sha512
firewall –service=ssh
poweroff

zerombr yes
clearpart –linux –drives=vda
part /boot –fstype ext4 –size 512
part swap –recommended
part / –fstype ext4 –size 4096 –grow

bootloader –location=mbr –timeout=5 –append=”rhgb quiet console=ttyS0″

%packages –nobase
sudo
ypbind
nfs-utils
ntp
bind-utils
%end
[/raw]

I’m not going to explain configuration options one by one as you can get more detailed explanation from CentOS’s kickstart documentation site.

Then, create the image file (raw format) on GlusterFS volume, you can do it on fuse mounted place at this moment since there won’t be any significant amount data to be written:
[shell]
sudo truncate -s 100G /mnt/glusterfs/VM/test-vm/disk.img
[/shell]

Or you prefer qemu-img way:
[shell]
sudo qemu-img create -f raw gluster://localhost:24007/glusterfs_vol_name/VM/test-vm/disk.img 100G
[/shell]

Either one will create a sparsed file size 100G.

Now, if you’ve downloaded the installation ISO to somewhere, do this:
[shell]
sudo virt-install –name test-vm –ram 1024 –vcpus 2 –disk path=/mnt/glusterfs/VM/test-vm/disk.img,size=100 –network bridge=br1 –os-variant rhel6 –accelerate –location /path/to/CentOS-6.2-x86_64-bin-DVD1.iso –graphics none -x “ks=http://192.168.0.1/kickstart.txt console=ttyS0”
[/shell]

or if you prefer do a network installation:
[shell]
sudo virt-install –name test-vm –ram 1024 –vcpus 2 –disk path=/mnt/glusterfs/VM/test-vm/disk.img,size=100 –network bridge=br1 –os-variant rhel6 –accelerate –location http://mirror.stanford.edu/yum/pub/centos/6/os/x86_64/ –graphics none -x “ks=http://192.168.0.1/kickstart.txt console=ttyS0”
[/shell]

After the installation, the machine should be powered-off, if for whatever reason it does not, do a poweroff, then do:

sudo virsh list --all


to make sure test-vm is not running, then edit the VM’s configuration by:
[shell]
sudo virsh edit test-vm
[/shell]

The only thing needs to be changed is the location of the disk image, it should be:
[xml]

[/xml]

and need to be changed to:

[xml]

[/xml]

Remember double check ownership of the disk.img file – it should be owned by qemu:qemu, otherwise you are going to run into disk IO problem (in VM).

Now, start the VM:
[shell]
sudo virsh start –console test-vm
[/shell]

and everything should be there 🙂 .

If you have problem with the VM and want to start over, you can:
[shell]sudo virsh reset test-vm[/shell]
to power-cycle the VM,
[shell]sudo virsh destroy test-vm[/shell],
to power-off the VM
[shell]sudo virsh undefine test-vm[/shell],
to remove the VM from libvirt’s repository.

I did some performance test on disk, putting the disk image on GlusterFS makes disk I/O slow for sure, but I think it is still acceptable:
[shell]
glusterfs-fuse 1048576000 bytes (1.0 GB) copied, 192.922 s, 5.4 MB/s
glusterfs-api 1048576000 bytes (1.0 GB) copied, 33.0347 s, 31.7 MB/s
local-filesystem 1048576000 bytes (1.0 GB) copied, 5.70798 s, 184 MB/s
[/shell]

The test command was:
[shell]
dd if=/dev/zero of=/tmp/dd-test bs=1M count=1000
[/shell]