Virtual Machines working group

Members and assignment

Sander Klous - Nikhef (Chair)<br\> Ronald Starink - Nikhef<br\> Marc van Driel - NBIC<br\> Pieter van Beek - SARA<br\> Ron Trompert - SARA<br\> <br\> Charge

Meetings

Kick-off - Monday July 6, 2009: agenda (dutch), minutes (dutch)

Presentations

Sky computing - Sander@nikhef.nl Klous (Monday July 6, 2009), a summary of the CERN virtual machines workshop (see other information) and an introduction for the kick-off meeting of the BIG grid virtual machines working group.

Open Issues

• Network Address Translation - What is the load?<br\>
• Virtual Machine Isolation - Prohibit internal network connectivity with IPTables.<br\>
• Image repository - Storage Area Network or distributed over worker nodes.<br\>
• Policy document<br\>

Infrastructure

We are setting up a testbed to investigate technical issues related to virtual machine management.

Hardware and Operating Systems

• Two Dell 1950 machines, dual CPU, 4 cores per CPU
  • One machine has a CentOS-5 installation
  • One machine has a Debian-squeeze installation

Software

• CentOS-5 comes with Xen 3.0
• Debian-squeeze comes with Xen 3.3
  • Debian-squeeze Xen packages have a problem with tap:aio.

Fix:
ln -s /usr/lib/xen-3.2-1/bin/tapdisk /usr/sbin
echo xenblktap >> /etc/modules

• Opennebula has been installed (stand alone) on CentOS-5 following this guide
  • A few additional staps were needed:
    • Install rubygems and rubygem-sqlite3
    • Opennebula has to be added to the sudoers file for xm and xentop
    • Sudoers should not require a tty

wget ftp://fr.rpmfind.net/linux/EPEL/5/x86_64/rubygem-sqlite3-ruby-1.2.4-1.el5.x86_64.rpm
wget ftp://fr.rpmfind.net/linux/EPEL/5/x86_64/rubygems-1.3.1-1.el5.noarch.rpm
sudo rpm -Uvh rubygems-1.3.1-1.el5.noarch.rpm rubygem-sqlite3-ruby-1.2.4-1.el5.x86_64.rpm 

In /etc/sudoers (on all machines)
opennebula ALL = NOPASSWD: /usr/sbin/xm
opennebula ALL = NOPASSWD: /usr/sbin/xentop
#Defaults    requiretty

• Installed iSCSI target and client software for shared image repository
  • Howtos: Debian client/server, CentOS client, CentOS server
  • Maybe test later with encrypted iSCSI
  • Two new machines ordered with the required iSCSI offload
• Image repository consists of LVM volume groups
  • Performance of LVM is better than file based images
  • Each logical volume contains an image
  • This allows easy creation/deletion of new images
  • VMs can run from cloned (Copy-On-Write) images

Implementation issues

Implemented iSCSI image management for opennebula following the storage guide

• Changed oned configuration as shown below
• Added tm_iscsi configuration
• Implemented transfer manager commands
• Modified XenDriver.cc and TransferManager.cc to support LVM images

In /opt/opennebula/etc/oned.conf:
TM_MAD = [
   name       = "tm_iscsi",
   executable = "one_tm",
   arguments  = "tm_iscsi/tm_iscsi.conf",
   default    = "tm_iscsi/tm_iscsi.conf" ]

/opt/opennebula/etc/tm_iscsi/tm_iscsi.conf
/opt/opennebula/etc/tm_iscsi/tm_iscsirc

/opt/opennebula/lib/tm_commands/iscsi/tm_clone.sh
/opt/opennebula/lib/tm_commands/iscsi/tm_delete.sh
/opt/opennebula/lib/tm_commands/iscsi/tm_ln.sh
/opt/opennebula/lib/tm_commands/iscsi/tm_mkimage.sh
/opt/opennebula/lib/tm_commands/iscsi/tm_mkswap.sh
/opt/opennebula/lib/tm_commands/iscsi/tm_mv.sh

.../one-1.2.0/src/vmm/XenDriver.cc
.../one-1.2.0/src/tm/TransferManager.cc

Local caching

The clone should be made on the local disk for performance reasons and reduction of network load. Only when a clone is saved (requires explicit configuration option), it needs to be transfered back to the image repository. LVM provides the means for this kind of management. Each worker node has a partition /dev/sda3, which is assigned to the logical volume with the Virtual Machine images. The LVM clone is forced to reside on this partition. Note that the partition is not available over iSCSI, so it can only be accessed by the local machine. The Volume Group will show "PV unknown device" and "Couldn't find device with uuid ..." on any other node. When the clone is saved, it is moved from /dev/sda3 to /dev/sdb, which is the iSCSI mounted partition from the image repository. As a result, only saved images are visible in the image repository. The following changes were made to accomplish this behavior:

Added /dev/sda3 at the end of the clone line in tm_clone.sh:
exec_and_log "ssh $DST_HOST sudo /usr/sbin/lvcreate -s -L$SIZE -n $LV_PATH $SRC_PATH /dev/sda3"

Added a line in tm_mv.sh to copy a saved clone back to the image repository:
exec_and_log "ssh $DST_HOST sudo /usr/sbin/pvmove -n $DST_PATH /dev/sda3 /dev/sdb"

Other information

• CERN June 2009 workshop on virtual machines