VO-specific software and modules

Introduction

VOs sometimes have a need to deploy their own software at a particular site. For this the Software Group Manager VOMS role is available. Not all VOs have this role and not all VOs with this role are supported at each site. If you are a Software Group Manager for a particular VO then the VOMS role will have been assigned to you and you can generate a SGM-specific proxy.

For these Software Group Managers this HOWTO will explain how you can deploy your own software and add your own module to the available modules for the modules command.

VOMS Role

The HEP VOs have an SGM role, usually of the form

 /Role=lcgadmin

However, in this HOWTO the VO vlemed was chosen as the example. This VO has a role

/Role=sgm

available, which gives users who possess that role the right to install software in the VO specific software area. To generate an SGM-proxy use

$ voms-proxy-init --voms vlemed:/vlemed/Role=sgm

You can view your current VOMS roles using

$ voms-proxy-info -all | sed 's/^/ /'
subject   : /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser/CN=proxy
issuer    : /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser
identity  : /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser
type      : proxy
strength  : 1024 bits
path      : /tmp/x509up_u7651
timeleft  : 11:59:33
=== VO vlemed extension information ===
VO        : vlemed
subject   : /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser
issuer    : /O=dutchgrid/O=hosts/OU=sara.nl/CN=voms.grid.sara.nl
attribute : /vlemed/Role=sgm/Capability=NULL
attribute : /vlemed/Role=NULL/Capability=NULL
timeleft  : 11:59:33
uri       : voms.grid.sara.nl:30003

VO-specific software area

The VO-specific software area is denoted using the environment variable

VO_<VO>_SW_DIR

e.g. for vlemed it is VO_VLEMED_SW_DIR.

At Nikhef the $VO_VLEMED_SW_DIR has the following permissions:

$ cd $VO_VLEMED_SW_DIR
$ ls -ald .
drwxrwsr-t 4 root vlemedsm 4096 Dec 11 16:17 .

This means the directory is writable only for members of the Unix group vlemedsm. When submitting a job to Nikhef using the SGM-proxy a special pool-account is chosen:

$ id vlemsm00
uid=70960(vlemsm00) gid=2058(vlemedsm) groups=2024(vlemed),2058(vlemedsm)

So the SGM-proxy causes a mapping to a pool-account with access rights to install software in $VO_VLEMED_SW_DIR.

Just to verify: a "regular" vlemed poolaccount does not have these permissions:

$ id vlemed00
uid=53200(vlemed00) gid=2024(vlemed) groups=2024(vlemed)

Note Checking this before proceeding is a very good practice, as sites misconfigurations at this level occur quite frequently.

Building the software

Before deploying new software we need to build and package it first on a local system:

• Notes on building
  • The target platform is a RHEL5 64bit compatible system. The easiest approach is to build the software on such a system (e.g. ui.grid.sara.nl)
  • It is not possible to hardcode paths into the software: the VO_<VO>_SW_DIR points to different directories on different clusters. Try to ensure that your software is relocatable using environment variables. Most software allows for this.
• Notes on packaging
  • In this HOWTO all software is packaged as a single tarball
  • The installation job then only has to download and extract the tarball in the right location.

For this HOWTO guide we will package a non-existing version 5.0 of the fsl package. The following directory structure has been set up for this package:

<local-dir>/fsl-5.0
<local-dir>/fsl-5.0/bin
<local-dir>/fsl-5.0/bin/fsl

where fsl-5.0/bin/fsl is a dummy script.

Packaging this software is very easy

cd <local-dir>
tar czvf ~/my-fsl-5.0.tar.gz fsl-5.0

We then upload this tarball to a public webserver (or gridftp server):

scp ~/my-fsl-5.0.tar.gz ~/public_html

Deploying the software

To deploy the software we use this JDL file:

Executable = "deploy.sh";
Stdoutput = "stdout";
StdError = "stderr";
InputSandbox = {"deploy.sh"};
OutputSandbox = {"stdout","stderr"};

with this deploy.sh script:

#!/bin/bash
cd $VO_VLEMED_SW_DIR
wget http://www.nikhef/~janjust/my-fsl-5.0.tar.gz || exit 1
tar xzf my-fsl-5.0.tar.gz || exit 2
# List the directory afterwards for inspection
ls -l

Run this job with an SGM-proxy like any other job at the cluster where you want to install it, e.g.

glite-wms-job-submit -d janjust.sgm -r gazon.nikhef.nl:2119/jobmanager-pbs-short deploy.jdl

Wait for completion and check the stdout and stderr files for any errores.

Adding a module

All VO-specific modules need to be installed in

VO_<VO>_SW_DIR/modules

Only modulefiles installed in this directory will be automagically picked up by the worker node login scripts.

Here is the listing of a sample modulefile

#%Module1.0#####################################################################
##
## fsl 5.0 modulefile
##

proc ModulesHelp { } {
        global fslversion

        puts stderr "\tSet up the environment for FSL"
        puts stderr "\n\tVersion $fslversion\n"
}

module-whatis   "sets FSL environment"

# for Tcl script use only
set     fslversion      5.0

set fsldir      "$env(VO_VLEMED_SW_DIR)/fsl-5.0"

prepend-path    PATH    "$fsldir/bin"
setenv          FSLDIR  "$fsldir"

which adds support for a (non-existing) version 5.0 of the package fsl.

Note especially the

$env(VO_VLEMED_SW_DIR)

in this modulefile: all software needs be installed (and thus, relocatable!) relative to this directory. The $env(...) command is the modules (TCL) method to import an environment variable.

It is by far the easiest to develop and test new modulefiles on a local system. After making sure that the modulefile works you can then include it in the software tarball or create a new tarball, e.g. mypackage-X.Y-module.tar.gz.

For this HOWTO we will do the latter:

$ cd <local-dir>
$ ls 
fsl-5.0  modules

mkdir -p $VO_VLEMED_SW_DIR/modules/mypackage
cd $VO_VLEMED_SW_DIR/modules/mypackage
wget modulefile

Use the module

• launch normal job
• check MODULEPATH

$ echo $MODULEPATH
[SNIP]:/etc/opt/vl-e/modulefiles::/data/esia/vlemed/modules

• check available modules

$ module avail
[SNIP]
------------------------------------ /data/esia/vlemed/modules -------------------------------------
fsl/5.0       mypackage/5.0

• use it:

$ module load fsl

show that it does not pick up the vlemed version:

$ module list
Currently Loaded Modulefiles:
  1) fsl/4.1.4

use

module load fsl/5.0

for that

• new packages are picked up automatically:

$ module load mypackage
$ module list
Currently Loaded Modulefiles:
  1) fsl/4.1.4       2) mypackage/5.0