Difference between revisions of "Xen on CentOS 5"

From PDP/Grid Wiki
Jump to navigationJump to search
 
(26 intermediate revisions by one other user not shown)
Line 1: Line 1:
−
= Centos-5.2-x86_64 xen-dom0 Installation =
+
'''This information is no longer actual, please refer to [[GSP_Virtualisation_with_Xen]] for up to date configuration and install instructions.'''
 +
 
 +
= Xen Virtualization =
 +
This description worked for us by part it is still work in progress. Any comments welcome.
 +
In the following we describe how to use Xen-Virtualisation for Quattor managed grid middleware (gLite) servers. The domUs (guests) are on local disks in logical volumes.
 +
Quattor is not necessarily needed, any other fabric management tool or simple scripts will do the same.
 +
In particular we had an existing Quattor environment and added the virtualisation framework, so you'll find here no information on howto manage system, glite-service-installation/configuration and host-monitoring, etc. with Quattor, this can be found elsewhere in this wiki. In the following Quattor manages "only" the dhcp.conf, kickstart- and pxe configuration files.
 +
 
 +
==Hardware:==
 +
PE-1950 with two 500G hard diskes, PE-2950 with four 500G hard disks.
 +
 
 +
== Software:==
 +
* Operating system host (dom0): CentOS-5.x, x.ge.2, x86-64
 +
* Virtualisation Method: Xen, stock CentOS xen kernels
 +
* Operating system guest (domU): CentOS-4.x, i386 (ParaVirtualisedMachine -guest, PVM), CentOS-3.x, i386 (Hardware (assisted) Virtual Machine, HVM)
 +
* LVM
 +
* Fabric Management: Quattor
 +
* Network installation tools: Kickstart, PXE, [https://www.cs.tcd.ie/Stephen.Childs/pypxeboot/ pypxeboot]
 +
* You have IPs for the dom0s and domUs available.
 +
 
 +
== Installation (PXE) Server ==
 +
We assume you have an installation server with Preboot eXecution Environment (PXE) available.
 +
 
 +
=== Images, Kernels ===
 +
For the dom0 network installation no additional steps are necessary as well as for the HVM domUs, for the PVM domUs you need to have the directories with the images. Example, for network installation of CentOS-4.6 i386 domUs you could do the following:
 +
mkdir -p <base directory>/xen-domU-c4.6/i386
 +
this directory should have the files initrd.img and vmlinuz, they are available from [http://mirror.liteserver.nl/pub/centos/4.6/os/i386/images/xen/ here]
 +
 
 +
=== DHCP ===
 +
Add the MAC adresses to /etc/dhcp.conf. For the dom-0 take the one from the network card. For the domU you have to choose a MAC-address. In order to avoid clashes (multiple usage of the same MAC) you can do the following: the first three hex numbers should be '''00:16:3e''' (reserved for xen domUs), the last three can be for example the hex representation of the last three numbers of the IP-address.
 +
Example:
 +
IP of the DomU is 192.'''168.1.2'''
 +
MAC:  00:16:3e:a8:01:02, the result of `printf 00:16:3e:%02x:%02x:%02x"\n" '''168 1 2'''` command
 +
 
 +
=== Create PXE Menus ===
 +
At our site this is created with Quattor, for a CentOS-4 i386 domU the cfg file looks like:
 +
  <base directory>/pxelinux.cfg/<hostname.cfg>
 +
 
 +
default Scientific Linux centos4_i386
 +
    label Scientific Linux centos4_i386
 +
    kernel xen-domU-c4.6/i386/vmlinuz
 +
    append ramdisk=32768 initrd=xen-domU-c4.6/i386/initrd.img \
 +
    ks=http://<Installation server>/<path to ks files>/<hostname>.ks ksdevice=eth0
 +
 
 +
For all hosts there should be a file like:
 +
<base directory>/pxelinux.cfg/localboot.cfg
 +
default Localdisk
 +
label Localdisk
 +
localboot 0
 +
 
 +
If you want to install/localboot a host create the links accordingly by hand or with your fabric management tool. At our site this is also done with Quattor.
 +
 
 +
== Centos-5.2-x86_64 xen-dom0 Installation ==
 
The following works for us, there is no guarantee that it will also work for you.  
 
The following works for us, there is no guarantee that it will also work for you.  
 
All settings might change in future.
 
All settings might change in future.
Line 5: Line 57:
 
* Network: 2 Interfaces, only one cabled
 
* Network: 2 Interfaces, only one cabled
  
−
== Installation of Dom-0 ==
+
=== Installation of Dom-0 ===
−
Dom0 (Host) needs < 10G for its system, tmp is not extensively used nor is var. To prevent the system to be unresponsive due to full / fs we could give 4GB /tmp and /var each. These values can be smaller if we have hardware which has some disk space restrictions.
+
Dom0 (Host) needs < 10G for its system, tmp is not extensively used nor is var (besides /var/lib/xen/save see below). To prevent the system to get unresponsive due to full filesystems you could give 4GB /tmp and /var each. These values can be smaller if you have hardware which has some disk space restrictions.
 
The saved states of the domUs, created for example during a reboot of the dom0 will be stored in /var/lib/xen/save, this might get filled and is on a separate partition.
 
The saved states of the domUs, created for example during a reboot of the dom0 will be stored in /var/lib/xen/save, this might get filled and is on a separate partition.
  
  
−
Depending on the hardware we have to set up a RAID-1 (2 disks) or we treat the (4-disks) hardware RAID-1-0 as one disk within the kickstart file.
+
Depending on the hardware we have to set up a soft RAID-1 (2 disks) or a hardware RAID-1-0 where we treat the (4-disks) as one disk within the kickstart file.
  
 
=== Filesystem layout Soft-RAID-1 ===
 
=== Filesystem layout Soft-RAID-1 ===
Line 21: Line 73:
 
  Volumegroup System, Size: rest of available space
 
  Volumegroup System, Size: rest of available space
 
  I System-Volumes-Dom0 [/, /tmp] fs: ext3
 
  I System-Volumes-Dom0 [/, /tmp] fs: ext3
−
  II DomU-lvms added when needed
+
  II DomU-lvms added when needed.
  
  
Line 45: Line 97:
 
  logvol /var --fstype=ext3      --vgname=systemvg      --size=4096    --name=sysvar
 
  logvol /var --fstype=ext3      --vgname=systemvg      --size=4096    --name=sysvar
 
  logvol /var/lib/xen/save --fstype=ext3      --vgname=systemvg      --size=10240    --name=sysxensave
 
  logvol /var/lib/xen/save --fstype=ext3      --vgname=systemvg      --size=10240    --name=sysxensave
 +
 +
For reinstallation of the host OS leaving the domUs untouched the following might work: ''' Try at own risk!'''
 +
# make sure to have deleted or put a comment on the line saying: '''clearpart --all --initlabel'''
 +
#part raid.01 --size=512  --ondisk=sda --asprimary
 +
#part raid.02 --size=8192 --ondisk=sda --asprimary
 +
#part raid.03 --size=24576  --ondisk=sda
 +
#part raid.04 --size=1 --ondisk=sda --grow
 +
#part raid.11 --size=512  --ondisk=sdb --asprimary
 +
#part raid.12 --size=8192 --ondisk=sdb --asprimary
 +
#part raid.13 --size=24576  --ondisk=sdb
 +
#part raid.14 --size=1 --ondisk=sdb --grow
 +
raid /boot    --level=RAID1 --device=md0 --fstype=ext2  --useexisting
 +
raid /        --level=RAID1 --device=md1 --fstype=ext3  --useexisting
 +
raid pv.01 --fstype "physical volume (LVM)" --level=RAID1 --device=md2  --useexisting
 +
raid pv.02 --fstype "physical volume (LVM)" --level=RAID1 --device=md3  --useexisting
 +
volgroup systemvg      pv.01 --useexisting
 +
volgroup xenvg          pv.02 --noformat
 +
logvol swap --fstype=swap      --vgname=systemvg      --size=2048    --name=sysswap --useexisting
 +
logvol /tmp --fstype=ext3      --vgname=systemvg      --size=4096    --name=systmp --useexisting
 +
logvol /var --fstype=ext3      --vgname=systemvg      --size=4096    --name=sysvar --useexisting
 +
logvol /var/lib/xen/save --fstype=ext3      --vgname=systemvg      --size=10240    --name=sysxensave --useexisting
  
 
=== Filesystem layout Harware-RAID-10 ===
 
=== Filesystem layout Harware-RAID-10 ===
Line 51: Line 124:
 
  clearpart --all --initlabel
 
  clearpart --all --initlabel
 
  part /boot --size=256 --asprimary --fstype=ext2 --ondisk=sda
 
  part /boot --size=256 --asprimary --fstype=ext2 --ondisk=sda
−
  part / --size=12288 --asprimary --fstype=ext3 --ondisk=sda
+
  part pv.01 --size=73728 --asprimary --fstype=ext3 --ondisk=sda
−
  part swap --size=8192 --asprimary --fstype=swap --ondisk=sda
+
  part pv.02 --size=1 --grow --ondisk=sda
−
  part pv.01 --size=1 --grow --ondisk=sda
+
volgroup systemvg pv.01
−
  volgroup system pv.01
+
volgroup xenvg pv.02
−
  logvol /var --fstype ext3 --size=8192    --name=var --vgname=system
+
logvol / --fstype ext3 --size=8192    --name=sysroot  --vgname=systemvg
−
  logvol /tmp --fstype ext3 --size=8192     --name=tmp --vgname=system
+
logvol /var/lib/xen/save --fstype ext3 --size=16384    --name=sysxensave  --vgname=systemvg
−
  logvol /var/lib/xen/save --fstype=ext3      --vgname=systemvg       --size=10240     --name=sysxensave
+
logvol /var --fstype ext3 --size=4096    --name=sysvar  --vgname=systemvg
 +
logvol /tmp --fstype ext3 --size=2048    --name=systmp  --vgname=systemvg
 +
logvol swap --fstype swap --size=4096    --name=sysswap  --vgname=systemvg
 +
 
 +
For reinstalling the host OS and keep the domUs you could try ''' at own risk!''':
 +
# make sure to have deleted or put a comment on the line saying: '''clearpart --all --initlabel'''
 +
#part /boot --size=256 --asprimary --fstype=ext2 --ondisk=sda
 +
  #part pv.01 --size=73728 --asprimary --fstype=ext3 --ondisk=sda
 +
#part pv.02 --size=1 --grow --ondisk=sda
 +
  volgroup systemvg pv.01 --useexisting
 +
volgroup xenvg pv.02 --noformat
 +
  logvol / --fstype ext3 --size=8192    --name=sysroot --vgname=systemvg --useexisting
 +
  logvol /var/lib/xen/save --fstype ext3 --size=16384     --name=sysxensave --vgname=systemvg --useexisting
 +
  logvol /var --fstype ext3 --size=4096    --name=sysvar  --vgname=systemvg --useexisting
 +
logvol /tmp --fstype ext3 --size=2048    --name=systmp  --vgname=systemvg --useexisting
 +
logvol swap --fstype swap --size=4096     --name=sysswap  --vgname=systemvg --useexisting
  
 
=== Software ===
 
=== Software ===
Line 78: Line 166:
 
  xen
 
  xen
 
  kernel-xen
 
  kernel-xen
 +
lvm2
 +
 +
== DomU Kickstart file ==
 +
Here the whole lvm created on the dom0 will be used as one disk (xvda), the size of the root partition will then depend on the lvm size. For a basic system the following snipped of a ks file could be useful:
 +
clearpart --all --initlabel
 +
part swap  --size=1024 --asprimary --ondisk=xvda
 +
part /tmp  --size=4096 --ondisk=xvda
 +
part /var  --size=4096 --ondisk=xvda
 +
part /    --size=8192 --grow --asprimary --ondisk=xvda
 +
.
 +
.
 +
.
 +
zerombr yes
 +
%packages --resolvedeps --ignoremissing
 +
openssh
 +
openssh-server
 +
wget
 +
perl-libnet
 +
perl-MIME-Base64
 +
perl-URI
 +
perl-Digest-MD5
 +
perl-libwww-perl
 +
perl-XML-Parser
 +
perl-DBI
 +
perl-Crypt-SSLeay
 +
lsof
 +
curl
 +
  
−
==== Addidtional Software needed to pxeboot domUs ====
 
−
See [https://www.cs.tcd.ie/Stephen.Childs/pypxeboot/ S. Childs]
 
  
−
= Xen dom-0 Configuration =
+
== Xen dom-0 Configuration ==
−
== Dom0 grub ==
+
=== Dom0 grub ===
−
For performance its best to limit the Memory of the dom0 to 512 MB and to prevent it from ballooning (i.e.dom-0 takes all available memory), and to restrict the dom0 to one cpu.
+
We limit the memory of the dom0 to 512 MB and to prevent it from ballooning (i.e.dom-0 takes all available memory), and to restrict the dom0 to one cpu.
 
Use some monitoring to make sure that these settings are sufficient.
 
Use some monitoring to make sure that these settings are sufficient.
  
−
Therefore modify /boot/grub/menu.lst, the important entries are in bold:
+
Therefore modify the dom0 /boot/grub/menu.lst, the important entries are in bold:
  
 
  title CentOS (2.*xen)
 
  title CentOS (2.*xen)
Line 95: Line 209:
 
         module /initrd-2.6.18-92.1.6.el5xen.img
 
         module /initrd-2.6.18-92.1.6.el5xen.img
  
−
== xend configuration ==
+
=== xend configuration ===
 
Set in /etc/xen/xend-config.sxp:
 
Set in /etc/xen/xend-config.sxp:
 
  (dom0-min-mem 0)
 
  (dom0-min-mem 0)
  
−
= Installation of a domU using pypxeboot =
+
== Installation of a domU using pypxeboot ==
 +
On the dom0 pypxeboot and the patched version of udhcp has to be available, see [https://www.cs.tcd.ie/Stephen.Childs/pypxeboot/ here].
 +
 
 +
On the dom0 hosting the domU, do the following:
 +
# Create a lvm for the domU, for the size check the domU kickstart file you created above. ''' lvcreate -n <domU name> -L<SIZE>G xenvg '''
 +
# Create domU config file:
 +
You'll have to provide the domU MAC-address you created above for the dhcp config on the installation server.
  
−
== Create domU config file ==
+
  <dom0>:/etc/xen/<domU name>
−
You'll have to provide a MAC-address.
+
  name ='<domU name>'
−
Calculate last 3 hex numbers from IP (example 192.168.96.43)with: printf ":%02x:%02x:%02x\n" {192,168,43}
 
−
this shoul not clash with the MAC-Address of other domUs.
 
−
create domU config file
 
−
  <dom0>:/etc/xen/ullr
 
−
  name ='ullr'
 
 
  memory ='1024'
 
  memory ='1024'
 
  bootloader="/usr/bin/pypxeboot"
 
  bootloader="/usr/bin/pypxeboot"
 
  vif=['mac=00:16:3e:XX:XX:XX']
 
  vif=['mac=00:16:3e:XX:XX:XX']
 
  bootargs=vif[0]
 
  bootargs=vif[0]
−
  disk = [ "phy:xenvg/<DOMU-NAME>,xvda,w" ]
+
  disk = [ "phy:xenvg/<domU-name>,xvda,w" ]
 
  root="/dev/xvda1 ro"
 
  root="/dev/xvda1 ro"
 
  on_poweroff = 'destroy'
 
  on_poweroff = 'destroy'
Line 118: Line 233:
 
  on_crash    = 'restart'
 
  on_crash    = 'restart'
  
−
== Create logical volume for this domU ==
 
−
lvcreate -L XG -n <DOMU-NAME xenvg
 
  
−
== Create DomU ==
+
=== Create DomU ===
 
on dom0:
 
on dom0:
−
  xm create <DOMU-NAME>
+
  xm create <domU name>
−
= On Installationserver =
 
  
−
== /tftpboot/pxelinux.cfg/netinstall-CentOS-4-i386-xenU==
+
should give you something like: ... to be continued
−
should read like
 
−
# You need the xen-kernels for the DomU, get them from
 
−
default centos4.6_32_xenU
 
−
label centos4.6_32_xenU
 
−
  kernel vmlinuz-CentOS-4.6_i386_xenU
 
−
  append initrd=initrd.img-CentOS-4.6_i386_xenU
 
−
== Create kickstart file for DomU ==
 

Latest revision as of 11:36, 16 June 2012

This information is no longer actual, please refer to GSP_Virtualisation_with_Xen for up to date configuration and install instructions.

Xen Virtualization

This description worked for us by part it is still work in progress. Any comments welcome. In the following we describe how to use Xen-Virtualisation for Quattor managed grid middleware (gLite) servers. The domUs (guests) are on local disks in logical volumes. Quattor is not necessarily needed, any other fabric management tool or simple scripts will do the same. In particular we had an existing Quattor environment and added the virtualisation framework, so you'll find here no information on howto manage system, glite-service-installation/configuration and host-monitoring, etc. with Quattor, this can be found elsewhere in this wiki. In the following Quattor manages "only" the dhcp.conf, kickstart- and pxe configuration files.

Hardware:

PE-1950 with two 500G hard diskes, PE-2950 with four 500G hard disks.

Software:

  • Operating system host (dom0): CentOS-5.x, x.ge.2, x86-64
  • Virtualisation Method: Xen, stock CentOS xen kernels
  • Operating system guest (domU): CentOS-4.x, i386 (ParaVirtualisedMachine -guest, PVM), CentOS-3.x, i386 (Hardware (assisted) Virtual Machine, HVM)
  • LVM
  • Fabric Management: Quattor
  • Network installation tools: Kickstart, PXE, pypxeboot
  • You have IPs for the dom0s and domUs available.

Installation (PXE) Server

We assume you have an installation server with Preboot eXecution Environment (PXE) available.

Images, Kernels

For the dom0 network installation no additional steps are necessary as well as for the HVM domUs, for the PVM domUs you need to have the directories with the images. Example, for network installation of CentOS-4.6 i386 domUs you could do the following: 

mkdir -p <base directory>/xen-domU-c4.6/i386

this directory should have the files initrd.img and vmlinuz, they are available from here

DHCP

Add the MAC adresses to /etc/dhcp.conf. For the dom-0 take the one from the network card. For the domU you have to choose a MAC-address. In order to avoid clashes (multiple usage of the same MAC) you can do the following: the first three hex numbers should be 00:16:3e (reserved for xen domUs), the last three can be for example the hex representation of the last three numbers of the IP-address. 

Example: 
IP of the DomU is 192.168.1.2
MAC:  00:16:3e:a8:01:02, the result of `printf 00:16:3e:%02x:%02x:%02x"\n" 168 1 2` command

Create PXE Menus

At our site this is created with Quattor, for a CentOS-4 i386 domU the cfg file looks like: 

 <base directory>/pxelinux.cfg/<hostname.cfg>

default Scientific Linux centos4_i386
   label Scientific Linux centos4_i386
   kernel xen-domU-c4.6/i386/vmlinuz
   append ramdisk=32768 initrd=xen-domU-c4.6/i386/initrd.img \
   ks=http://<Installation server>/<path to ks files>/<hostname>.ks ksdevice=eth0

For all hosts there should be a file like: 

<base directory>/pxelinux.cfg/localboot.cfg
default Localdisk
label Localdisk
localboot 0

If you want to install/localboot a host create the links accordingly by hand or with your fabric management tool. At our site this is also done with Quattor.

Centos-5.2-x86_64 xen-dom0 Installation

The following works for us, there is no guarantee that it will also work for you. All settings might change in future.

  • Hardware: Dell PE-1950, HD: 2X500G Soft-RAID-1, PE-2950, HD: 4X500G Hardware-RAID-1-0.
  • Network: 2 Interfaces, only one cabled

Installation of Dom-0

Dom0 (Host) needs < 10G for its system, tmp is not extensively used nor is var (besides /var/lib/xen/save see below). To prevent the system to get unresponsive due to full filesystems you could give 4GB /tmp and /var each. These values can be smaller if you have hardware which has some disk space restrictions. The saved states of the domUs, created for example during a reboot of the dom0 will be stored in /var/lib/xen/save, this might get filled and is on a separate partition.


Depending on the hardware we have to set up a soft RAID-1 (2 disks) or a hardware RAID-1-0 where we treat the (4-disks) as one disk within the kickstart file.

Filesystem layout Soft-RAID-1

Principle layout:

  • RAID-1
  • Primary Partitions
/boot 512MB
/swap 4096MB
  • LVM
Volumegroup System, Size: rest of available space
I System-Volumes-Dom0 [/, /tmp] fs: ext3
II DomU-lvms added when needed.


To have the possibility to reinstall the dom0 with the fabric management tool (uses pxeboot/kickstart) we postprocess the ks file after first installation. See appendix (ks2reinst.sh). 

clearpart --all --initlabel
part raid.01 --size=512   --ondisk=sda --asprimary
part raid.02 --size=8192 --ondisk=sda --asprimary
part raid.03 --size=24576  --ondisk=sda
part raid.04 --size=1 --ondisk=sda --grow
part raid.11 --size=512   --ondisk=sdb --asprimary
part raid.12 --size=8192 --ondisk=sdb --asprimary
part raid.13 --size=24576  --ondisk=sdb
part raid.14 --size=1 --ondisk=sdb --grow
raid /boot     --level=RAID1 --device=md0 --fstype=ext2 raid.01 raid.11
raid /         --level=RAID1 --device=md1 --fstype=ext3 raid.02 raid.12
raid pv.01 --fstype "physical volume (LVM)" --level=RAID1 --device=md2 raid.03 raid.13
raid pv.02 --fstype "physical volume (LVM)" --level=RAID1 --device=md3 raid.04 raid.14
volgroup systemvg       pv.01
volgroup xenvg          pv.02
logvol swap --fstype=swap       --vgname=systemvg       --size=2048     --name=sysswap
logvol /tmp --fstype=ext3       --vgname=systemvg       --size=4096     --name=systmp
logvol /var --fstype=ext3       --vgname=systemvg       --size=4096     --name=sysvar
logvol /var/lib/xen/save --fstype=ext3       --vgname=systemvg       --size=10240     --name=sysxensave

For reinstallation of the host OS leaving the domUs untouched the following might work: Try at own risk! 

# make sure to have deleted or put a comment on the line saying: clearpart --all --initlabel
#part raid.01 --size=512   --ondisk=sda --asprimary
#part raid.02 --size=8192 --ondisk=sda --asprimary
#part raid.03 --size=24576  --ondisk=sda
#part raid.04 --size=1 --ondisk=sda --grow
#part raid.11 --size=512   --ondisk=sdb --asprimary
#part raid.12 --size=8192 --ondisk=sdb --asprimary
#part raid.13 --size=24576  --ondisk=sdb
#part raid.14 --size=1 --ondisk=sdb --grow
raid /boot     --level=RAID1 --device=md0 --fstype=ext2  --useexisting
raid /         --level=RAID1 --device=md1 --fstype=ext3  --useexisting
raid pv.01 --fstype "physical volume (LVM)" --level=RAID1 --device=md2  --useexisting
raid pv.02 --fstype "physical volume (LVM)" --level=RAID1 --device=md3  --useexisting
volgroup systemvg       pv.01 --useexisting
volgroup xenvg          pv.02 --noformat
logvol swap --fstype=swap       --vgname=systemvg       --size=2048     --name=sysswap --useexisting
logvol /tmp --fstype=ext3       --vgname=systemvg       --size=4096     --name=systmp --useexisting
logvol /var --fstype=ext3       --vgname=systemvg       --size=4096     --name=sysvar --useexisting
logvol /var/lib/xen/save --fstype=ext3       --vgname=systemvg       --size=10240     --name=sysxensave --useexisting

Filesystem layout Harware-RAID-10

Here the disks appear as one and the respective section in the kickstart file gets a lot simpler: 

clearpart --all --initlabel
part /boot --size=256 --asprimary --fstype=ext2 --ondisk=sda
part pv.01 --size=73728 --asprimary --fstype=ext3 --ondisk=sda
part pv.02 --size=1 --grow --ondisk=sda
volgroup systemvg pv.01
volgroup xenvg pv.02
logvol / --fstype ext3 --size=8192     --name=sysroot  --vgname=systemvg
logvol /var/lib/xen/save --fstype ext3 --size=16384     --name=sysxensave  --vgname=systemvg
logvol /var --fstype ext3 --size=4096     --name=sysvar  --vgname=systemvg
logvol /tmp --fstype ext3 --size=2048     --name=systmp  --vgname=systemvg
logvol swap --fstype swap --size=4096     --name=sysswap  --vgname=systemvg

For reinstalling the host OS and keep the domUs you could try at own risk!: 

# make sure to have deleted or put a comment on the line saying: clearpart --all --initlabel
#part /boot --size=256 --asprimary --fstype=ext2 --ondisk=sda
#part pv.01 --size=73728 --asprimary --fstype=ext3 --ondisk=sda
#part pv.02 --size=1 --grow --ondisk=sda
volgroup systemvg pv.01 --useexisting
volgroup xenvg pv.02 --noformat
logvol / --fstype ext3 --size=8192     --name=sysroot  --vgname=systemvg --useexisting
logvol /var/lib/xen/save --fstype ext3 --size=16384     --name=sysxensave  --vgname=systemvg --useexisting
logvol /var --fstype ext3 --size=4096     --name=sysvar  --vgname=systemvg --useexisting
logvol /tmp --fstype ext3 --size=2048     --name=systmp  --vgname=systemvg --useexisting
logvol swap --fstype swap --size=4096     --name=sysswap  --vgname=systemvg --useexisting

Software

We install stock CentOS packages, the following excerpt from the kickstart file should be sufficient. 

%packages --resolvedeps --ignoremissing
openssh
openssh-server
wget
perl-libnet
perl-MIME-Base64
perl-URI
perl-Digest-MD5
perl-libwww-perl
perl-XML-Parser
perl-DBI
perl-Crypt-SSLeay
lsof
curl
xen
kernel-xen
lvm2

DomU Kickstart file

Here the whole lvm created on the dom0 will be used as one disk (xvda), the size of the root partition will then depend on the lvm size. For a basic system the following snipped of a ks file could be useful: 

clearpart --all --initlabel
part swap  --size=1024 --asprimary --ondisk=xvda
part /tmp  --size=4096 --ondisk=xvda
part /var  --size=4096 --ondisk=xvda
part /     --size=8192 --grow --asprimary --ondisk=xvda
.
.
.
zerombr yes
%packages --resolvedeps --ignoremissing
openssh
openssh-server
wget
perl-libnet
perl-MIME-Base64
perl-URI
perl-Digest-MD5
perl-libwww-perl
perl-XML-Parser
perl-DBI
perl-Crypt-SSLeay
lsof
curl


Xen dom-0 Configuration

Dom0 grub

We limit the memory of the dom0 to 512 MB and to prevent it from ballooning (i.e.dom-0 takes all available memory), and to restrict the dom0 to one cpu. Use some monitoring to make sure that these settings are sufficient.

Therefore modify the dom0 /boot/grub/menu.lst, the important entries are in bold: 

title CentOS (2.*xen)
       root (hd0,0)
       kernel /xen.gz-2.6.18-92.1.6.el5 dom0_mem=512M
       module /vmlinuz-2.6.18-92.1.6.el5xen ro root=/dev/systemvg/sysroot nosmp
       module /initrd-2.6.18-92.1.6.el5xen.img

xend configuration

Set in /etc/xen/xend-config.sxp: 

(dom0-min-mem 0)

Installation of a domU using pypxeboot

On the dom0 pypxeboot and the patched version of udhcp has to be available, see here.

On the dom0 hosting the domU, do the following:

  1. Create a lvm for the domU, for the size check the domU kickstart file you created above. lvcreate -n <domU name> -L<SIZE>G xenvg
  2. Create domU config file:
You'll have to provide the domU MAC-address you created above for the dhcp config on the installation server.

<dom0>:/etc/xen/<domU name>
name ='<domU name>'
memory ='1024'
bootloader="/usr/bin/pypxeboot"
vif=['mac=00:16:3e:XX:XX:XX']
bootargs=vif[0]
disk = [ "phy:xenvg/<domU-name>,xvda,w" ]
root="/dev/xvda1 ro"
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'


Create DomU

on dom0: 

xm create <domU name>

should give you something like: ... to be continued

Retrieved from "https://wiki.nikhef.nl/grid/index.php?title=Xen_on_CentOS_5&oldid=8043"