|
|
| Line 232: |
Line 232: |
| | The other Xen machines, toom and kudde, run Xen 3.1. On these machines the creation and destruction of virtual machines is best left to the generate-machine and destroy-machine scripts, part of the [https://gforge.vl-e.nl/plugins/scmcvs/cvsweb.php/nl.vl-e.poc.ctb.mktestbed/?cvsroot=build nl.vl-e.poc.ctb.mktestbed] software package. | | The other Xen machines, toom and kudde, run Xen 3.1. On these machines the creation and destruction of virtual machines is best left to the generate-machine and destroy-machine scripts, part of the [https://gforge.vl-e.nl/plugins/scmcvs/cvsweb.php/nl.vl-e.poc.ctb.mktestbed/?cvsroot=build nl.vl-e.poc.ctb.mktestbed] software package. |
| | | | |
| − |
| |
| − | == Operational procedures ==
| |
| − |
| |
| − | The testbed is not too tightly managed, but here's an attempt to keep our sanity.
| |
| − |
| |
| − | === Server certificates ===
| |
| − |
| |
| − | Host or server SSL certificates for volatile machines in the testbed are kept on span.nikhef.nl:/var/local/hostkeys. The FQDN of the host determines which CA should be used:
| |
| − | * for *.nikhef.nl, the TERENA eScience SSL CA should be used,
| |
| − | * for *.testbed, the testbed CA should be used.
| |
| − |
| |
| − | ==== Generating certificate requests for the TERENA eScience SSL CA ====
| |
| − |
| |
| − | * Go to bleek.nikhef.nl:/var/local/hostkeys/pem/
| |
| − | * Generate a new request by running ../[https://ndpfsvn.nikhef.nl/cgi-bin/viewvc.cgi/pdpsoft/trunk/agiletestbed/make-terena-req.sh?view=co make-terena-req.sh] ''hostname''. This will create a directory for the hostname with the key and request in it.
| |
| − | * Send the resulting newrequest.csr file to the local registrar (Paul or Elly).
| |
| − | * When the certificate file comes back, install it in /var/local/hostkeys/pem/''hostname''/.
| |
| − |
| |
| − | ==== Requesting certificates from the testbed CA ====
| |
| − |
| |
| − | Kindly ask Dennis. The CA key is on his eToken, which means no one else can generate host certificates. Some time in the future this will be replaced by a simple CA setup on the testbed itself.
| |
| − |
| |
| − |
| |
| − | === Logging of changes ===
| |
| − |
| |
| − | All changes need to be communicated by e-mail to [mailto:CTB-changelog@nikhef.nl CTB-changelog@nikhef.nl].
| |
| − |
| |
| − | (This replaces the earlier [[CTB Changelog]].)
| |
| − |
| |
| − | === adding a new machine ===
| |
| − |
| |
| − | ==== preparations on bleek ====
| |
| − | * edit
| |
| − | /etc/hosts
| |
| − | /etc/ethers
| |
| − | to add the new machine, and hardware address.
| |
| − | * Restart dnsmasq
| |
| − | /etc/init.d/dnsmasq restart
| |
| − | * on span.nikhef.nl, run
| |
| − | /usr/local/bin/keygen <hostname>
| |
| − | to pre-generate ssh keys.
| |
| − | * on span, run
| |
| − | /var/local/hostkeys/generate-knownhosts.sh
| |
| − | * on all machines, do
| |
| − | cp /var/local/hostkeys/ssh_known_hosts /etc/ssh/ssh_known_hosts
| |
| − | * (optional) generate or request an X509 host certificate. For local machines in the .testbed domain, Dutchgrid certificates won't be issued, but a testbed-wide CA is in use, ask Dennis. The certificate and key are stored in
| |
| − | /var/local/hostkeys/pem/<hostname>/hostcert.pem
| |
| − | /var/local/hostkeys/pem/<hostname>/hostkey.pem
| |
| − | * place a 'firstboot' script on span in
| |
| − | /var/local/xen/firstboot/<hostname>
| |
| − | (it will be downloaded and run the first time after installation of the machine.)
| |
| − |
| |
| − | ==== starting the installation on the dom0 ====
| |
| − |
| |
| − | The old generate-machine script still works for Xen Dom0. For blade13 and blade14 (and other libvirt machines) this script sort of works:
| |
| − |
| |
| − | virt-install --name my-new-machine.testbed --ram 1024 --disk pool=vmachines,size=10 \
| |
| − | --network bridge=br0,mac=52:54:00:48:22:32 --os-type linux --os-variant rhel5.4 \
| |
| − | --location http://spiegel.nikhef.nl/mirror/centos/5/os/x86_64 \
| |
| − | --extra ks=http://bleek.nikhef.nl/mktestbed/kickstart/centos5-kvm.ks
| |
| − |
| |
| − | Take into account that the mac address is what is configured in /etc/ethers on bleek.nikhef.nl; the disk pool points to the Compellent Fiber Channel pool.
| |
| − |
| |
| − |
| |
| − | A debian installation is similar, but uses preseeding instead of kickstart. The preseed configuration is on bleek.
| |
| − |
| |
| − | virt-install --name debian-builder.testbed --ram 1024 --disk pool=vmachines,size=10 \
| |
| − | --network bridge=br0,mac=52:54:00:fc:18:dd --os-type linux --os-variant debiansqueeze \
| |
| − | --location http://ftp.nl.debian.org/debian/dists/squeeze/main/installer-amd64/ \
| |
| − | --extra 'auto=true priority=critical url=http://bleek.testbed/d-i/squeeze/preseed.cfg'
| |
| − |
| |
| − | A few notes:
| |
| − | * The network autoconfiguration seems to happen too soon; the host bridge configuration doesn't pass the DHCP packets for a while after creating the domain which systematically causes the Debian installer to complain. Fortunately the configuration can be retried from the menu. The second time around the configuration is ok.
| |
| − | * Alternative installation kickstart files are available; e.g. http://bleek.nikhef.nl/mktestbed/kickstart/centos6-64-kvm.ks for CentOS 6.
| |
| − | ** With Debian preseeding, this may be automated by either setting <tt>d-i netcfg/dhcp_options select Retry network autoconfiguration</tt> or <tt>d-i netcfg/dchp_timeout string 60</tt>.
| |
| − | * Sometimes, a storage device is re-used (especially when recreating a domain after removing it '''and''' the associated storage). The re-use may cause the partitioner to see an existing LVM definition and fail, complaining that the partition already exists; you can re-use an existing LVM volume by using the argument: <tt>--disk vol=vmachines/blah</tt>.
| |
| − |
| |
| − | ==== Automatic configuration of machines ====
| |
| − |
| |
| − | The default kickstart scripts for testbed VMs will download a 'firstboot' script at the end of the install cycle, based on the name they've been given by DHCP. Look in span.nikhef.nl:/usr/local/mktestbed/firstboot for the files that are used, but be aware that these are managed with git (gitosis on span).
| |
| − |
| |
| − | === Configuration of LDAP authentication ===
| |
| − |
| |
| − | ==== Fedora Core 14 ====
| |
| − |
| |
| − | The machine fc14.testbed is configured for LDAP authn against ldap.nikhef.nl. Some notes:
| |
| − | * /etc/nslcd.conf:
| |
| − | uri ldaps://ldap.nikhef.nl ldaps://hooimijt.nikhef.nl
| |
| − | base dc=farmnet,dc=nikhef,dc=nl
| |
| − | ssl on
| |
| − | tls_cacertdir /etc/openldap/cacerts
| |
| − |
| |
| − | * /etc/openldap/cacerts is symlinked to /etc/grid-security/certificates.
| |
| − |
| |
| − | ==== Debian 'Squeeze' ====
| |
| − |
| |
| − | Debian is a bit different; the nslcd daemon is linked against GnuTLS instead of OpenSSL. Due to a bug (so it would seem) one cannot simply point to a directory of certificates. Debian provides a script to collect all the certificates in one big file. Here is the short short procedure:
| |
| − |
| |
| − | mkdir /usr/share/ca-certificates/igtf
| |
| − | for i in /etc/grid-security/certificates/*.0 ; do ln -s $i /usr/share/ca-certificates/igtf/`basename $i`.crt; done
| |
| − | update-ca-certificates
| |
| − |
| |
| − | The file /etc/nsswitch.conf needs to include these lines to use ldap:
| |
| − | passwd: compat ldap
| |
| − | group: compat ldap
| |
| − |
| |
| − | This file can be used in /etc/nslcd.conf:
| |
| − | uid nslcd
| |
| − | gid nslcd
| |
| − | base dc=farmnet,dc=nikhef,dc=nl
| |
| − | ldap_version 3
| |
| − | ssl on
| |
| − | uri ldaps://ldap.nikhef.nl ldaps://hooimijt.nikhef.nl
| |
| − | tls_cacertfile /etc/ssl/certs/ca-certificates.crt
| |
| − | timelimit 120
| |
| − | bind_timelimit 120
| |
| − | nss_initgroups_ignoreusers root
| |
| − |
| |
| − | The libpam-ldap package needs to be installed as well, with the following in /etc/pam_ldap.conf
| |
| − | base dc=farmnet,dc=nikhef,dc=nl
| |
| − | uri ldaps://ldap.nikhef.nl/ ldaps://hooimijt.nikhef.nl/
| |
| − | ldap_version 3
| |
| − | pam_password md5
| |
| − | ssl on
| |
| − | tls_cacertfile /etc/ssl/certs/ca-certificates.crt
| |
| − |
| |
| − | At least this seems to work.
| |
| | | | |
| | == Services index == | | == Services index == |
