Upgrading Quattor managed glite servers

Upgrading to a new glite release in not even 30 simple steps

The following notes were taken during the upgrade from glite-3.1-update-29 to glite-3.1-update-31.

As ndpfmgr@stal

• Run the skript ~/bin/mirror-glite-3.1

This creates a Quattor template file in /tmp which can be used to create a cern-mirror of the latest glite in /project/quattor/www/html/mirror/glite (fetch release from: http://glitesoft.cern.ch/EGEE/gLite/).

As normal-user@stal

Assuming you have a working Quattor environment, in particular $L points to a usable Quattor checkout. First argument of rpmUpdates.pl script is the directory of the mirror created as ndpfmgr before.

• $L/..//bin/rpmUpdates.pl /project/quattor/www/html/mirror/glite/3.1/generic/sl4/x86_64/updates/ > /tmp/31-i386
• $L/../bin/rpmUpdates.pl /project/quattor/www/html/mirror/glite/3.1/generic/sl4/x86_64/updates/ > /tmp/31-x86-64
• cd $L/cfg
• cp /tmp/glite-3.1-update-31.tpl ./
• cd $L/cfg/grid/glite-3.1/update/
• cp -a 29 31
• find $L/cfg/grid/glite-3.1/update/31 -type d -name CVS -exec rm -rf {} \; (delete the CVS files, this command will change when moving to subversion)
• for file in `grep -H -r "/29/" 31/* |awk -F : '{print $1}'`; do sed -i "s/\/29\//\/31\//g" $file; done (replace the old update nr with the new one).
• replace the entries in $L/cfg/grid/glite-3.1/update/31/{i386,x86_64}/rpms.tpl with entries in glite-3.1-update-31.tpl, take care to uncomment torque packages and that the manualy added packages are appended.
• rm $L/cfg/glite-3.1-update-31.tpl (file copied above, will not pass quattor syntax checks.)
• edit $L/cfg/grid/glite-3.1/glite/defaults.tpl replace update version nr to the actual one (set new update to default version)
• makexprof -u -f itb tbn09 (try to compile it for some testbed node)
• check if the changes for 64 bit are also OK, for example: makexprof -f prd hooi-ei-12
• if OK, submit the changes to CVS

Again as ndpfmgr@stal

Get the updates commited as normal user above, confirm they compile, push profile to a testbed node, here are the steps:

• cd $/cfg
• cvs upd -APd
• makexprof -u -f itb tbn09
• pushxprof -u -f itb tbn09
• if OK, push profiles to itb: makexprof -f itb
• pushxprof -f itb

And finaly-A, since Quattor somehow has to add its ideas of package managing

If you get noticed, preferably by a monitoring system (if you don't have one -good luck), that something does not work after carefully doing these simple some 20 steps above. You could try the following:

• Logon the host who shows problems
• tail /var/log/ncm/ncd.log

2008/10/02-15:25:30 [INFO] Errors while configuring spma (1)
2008/10/02-15:25:30 [ERROR] 1 errors, 0 warnings executing configure

This tells us, that we actualy have to look into

• tail /var/log/spma.log

2008/10/02-15:25:28 [WARN] Errors found:
depcheck: package glite-UI 3.1.19-0 needs glite-amga-api-python >= 1.3.0-1
depcheck: package glite-UI 3.1.19-0 needs glite-amga-cli >= 1.3.0-4
there were 2 dependency problem(s) and 0 conflict(s)

With this information now simply visit the web page which has the rpm list for this particular server, in our case: http://glite.web.cern.ch/glite/packages/R3.1/deployment/glite-UI/3.1.20-0/glite-UI-3.1.20-0.html

These packages have to be added to the respective rpm list, don't expect something fancy or advanced as in the package managers (rpm, apt,...) which are around, its a simple text file, but stepwise:

And finaly-B, again as normal-user@stal

Here the package managing for the ui is wrong. Note that you don't add it to the updates directory, this goes to the "base list".

• In our example the following should work:

cat << EOF >> $L/cfg/grid/glite-3.1/glite/ui/rpms.tpl
"/software/packages"=pkg_repl("glite-amga-cli","1.3.0-4","i386");
"/software/packages"=pkg_repl("glite-amga-api-python","1.3.0-1","noarch");
EOF

• makexprof -f prd bosui
• If compilation is successful proceed with:

cvs commit

And finaly-C, again as ndpfmgr@stal

Get the updates commited as normal user above, confirm that they compile, push profile to the node that misses packets (here bosui, an UI)

• cd $L/cfg/grid/glite-3.1/glite/ui
• cvs update rpms.tpl
• makexprof -u -f prd bosui
• if OK, pushxprof -u -f prd bosui
• check monitoring system