Upgrading Quattor managed glite servers

Upgrading to a new glite release

The following notes were taken during the upgrade from glite-3.1-update-29 to glite-3.1-update-31.

As ndpfmgr@stal

1. Run the skript ~/bin/mirror-glite-3.1

This creates a Quattor template file in /tmp which can be used to create a cern-mirror of the latest glite in /project/quattor/www/html/mirror/glite (fetch release from: http://glitesoft.cern.ch/EGEE/gLite/).

As normal-user@stal

Assuming you have a working Quattor environment, in particular $L points to a usable Quattor checkout. First argument of rpmUpdates.pl script is the directory of the mirror created as ndpfmgr before.

1. $L/..//bin/rpmUpdates.pl /project/quattor/www/html/mirror/glite/3.1/generic/sl4/x86_64/updates/ > /tmp/31-i386
2. $L/../bin/rpmUpdates.pl /project/quattor/www/html/mirror/glite/3.1/generic/sl4/x86_64/updates/ > /tmp/31-x86-64
3. cd $L/cfg
4. cp /tmp/glite-3.1-update-31.tpl ./
5. cd $L/cfg/grid/glite-3.1/update/
6. cp -a 29 31
7. find $L/cfg/grid/glite-3.1/update/31 -type d -name CVS -exec rm -rf {} \; (delete the CVS files, this command will change when moving to subversion)
8. for file in `grep -H -r "/29/" 31/* |awk -F : '{print $1}'`; do sed -i "s/\/29\//\/31\//g" $file; done (replace the old update nr with the new one).
9. replace the entries in $L/cfg/grid/glite-3.1/update/31/{i386,x86_64}/rpms.tpl with entries in glite-3.1-update-31.tpl, take care to uncomment torque packages and that the manualy added packages are appended.
10. rm $L/cfg/glite-3.1-update-31.tpl (file copied above, will not pass quattor syntax checks.)
11. edit $L/cfg/grid/glite-3.1/glite/defaults.tpl replace update version nr to the actual one (set new update to default version)
12. makexprof -u -f itb tbn09 (try to compile it for some testbed node)
13. check if the changes for 64 bit are also OK, for example: makexprof -f prd hooi-ei-12
14. if OK, submit the changes to CVS

Again as ndpfmgr@stal

Get the updates commited as normal user above, confirm they compile, push profile to a testbed node, here are the steps:

1. cd $/cfg
2. cvs upd -APd
3. makexprof -u -f itb tbn09
4. pushxprof -u -f itb tbn09
5. if OK, push profiles to itb: makexprof -f itb
6. pushxprof -f itb

And finaly, since Quattor somehow has to add its ideas of package managing

If you get noticed, preferably by a monitoring system (if you don't have one -good luck), that something does not work after carefully doing these simple some 20 steps above. You could try the following:

1. Logon the host who shows problems
2. tail /var/log/ncm/ncd.log

2008/10/02-15:25:30 [INFO] Errors while configuring spma (1)
2008/10/02-15:25:30 [ERROR] 1 errors, 0 warnings executing configure

This tells us, that we actualy have to look into

1. tail /var/log/spma.log

2008/10/02-15:25:28 [WARN] Errors found:
depcheck: package glite-UI 3.1.19-0 needs glite-amga-api-python >= 1.3.0-1
depcheck: package glite-UI 3.1.19-0 needs glite-amga-cli >= 1.3.0-4
there were 2 dependency problem(s) and 0 conflict(s)

With this information now simply visit the web page which has the rpm list for this particular server, in our case: http://glite.web.cern.ch/glite/packages/R3.1/deployment/glite-UI/3.1.20-0/glite-UI-3.1.20-0.html

These packages have to be added to the respective rpm list, don't expect something fancy or advanced as in the package managers (rpm, apt,...) which are around, its a simple text file, but stepwise:

Again as normal-user@stal

Here the package managing for the ui is wrong. Note that you don't add it to the updates directory, this goes to the "base list".

1. In our example the following should work:

cat << EOF >> $L/cfg/grid/glite-3.1/glite/ui/rpms.tpl
"/software/packages"=pkg_repl("glite-amga-cli","1.3.0-4","i386");
"/software/packages"=pkg_repl("glite-amga-api-python","1.3.0-1","noarch");
EOF

2. vi $L/cfg/grid/glite-3.1/glite/ui/rpms.tpl