Difference between revisions of "How to setup the qsub tunnel for use with Ganga and the LHCb software"
Jump to navigation
Jump to search
(One intermediate revision by the same user not shown) | |||
Line 7: | Line 7: | ||
* Make sure that you are using Ganga version 5.5.16 (v505r16) | * Make sure that you are using Ganga version 5.5.16 (v505r16) | ||
− | == Preparing your | + | == Preparing your environment on the submission machine == |
− | * To make | + | * To make your grid proxy available on the worker node, set the following environment variable before you create your proxy. It is easiest to put this in your login scripts. |
* The pbs server must also be known (Set this back to allier.nikhef.nl if you want to submit to stoomboot): | * The pbs server must also be known (Set this back to allier.nikhef.nl if you want to submit to stoomboot): | ||
* For bash-like shells: | * For bash-like shells: | ||
Line 21: | Line 21: | ||
</pre> | </pre> | ||
− | == Preparing the environment == | + | == Preparing the environment for the worker nodes == |
The LHCb software available on the Tier1 worker nodes is not the same installation as is available in the /project/bfys area. The worker nodes have all versions available which are distributed to the Grid by the LHCb production and software management team. Login scripts need to be prepared: | The LHCb software available on the Tier1 worker nodes is not the same installation as is available in the /project/bfys area. The worker nodes have all versions available which are distributed to the Grid by the LHCb production and software management team. Login scripts need to be prepared: | ||
* Create a file called .bash_profile in /data/tunnel/user/yourusername. | * Create a file called .bash_profile in /data/tunnel/user/yourusername. |
Latest revision as of 17:36, 22 December 2010
General information
The qsub tunnel is a new service supplied by the PDP group that allows Nikhef users to submit directly to the "medium" queue of the Nikhef Tier1. For general information and instructions, see this twiki page: qsub tunnel wiki pagina.
First steps
- Make sure that you are included in the list of people who are allowed to submit jobs using the qsub tunnel by asking the PDP group if you are.
- Make sure that you have a grid proxy if you want to use the Oracle conditions database or if you want to access files on the Nikhef or Sara storage elements.
- Make sure that you are using Ganga version 5.5.16 (v505r16)
Preparing your environment on the submission machine
- To make your grid proxy available on the worker node, set the following environment variable before you create your proxy. It is easiest to put this in your login scripts.
- The pbs server must also be known (Set this back to allier.nikhef.nl if you want to submit to stoomboot):
- For bash-like shells:
export X509_USER_PROXY=/data/tunnel/user/yourusername/.x509tmp export PBS_SERVER=stro.nikhef.nl
- For csh-like shells:
setenv X509_USER_PROXY /data/tunnel/user/yourusername/.x509tmp setenv PBS_SERVER stro.nikhef.nl
Preparing the environment for the worker nodes
The LHCb software available on the Tier1 worker nodes is not the same installation as is available in the /project/bfys area. The worker nodes have all versions available which are distributed to the Grid by the LHCb production and software management team. Login scripts need to be prepared:
- Create a file called .bash_profile in /data/tunnel/user/yourusername.
- Put the following lines into it:
if [ -f ~/.bashrc ]; then . ~/.bashrc fi
- Create a file named .bashrc in /data/tunnel/user/yourusername.
- Put the following lines into it:
export X509_USER_PROXY=/data/tunnel/user/yourusername/.x509tmp . $VO_LHCB_SW_DIR/group_login.sh
Ganga configuration
Edit your ~/.gangarc file and look for the [PBS] section, put in the following bits:
postexecute = import os env = os.environ tmpdir = None if "TMPDIR" in env: tmpdir = env[ "TMPDIR" ].rstrip( "/" ) else: tmpdir = "." jobnumid = env["PBS_JOBID"] dir = "%s/%s" % ( tmpdir, jobnumid ) os.chdir( tmpdir ) os.system("rm -rf %s" % dir )
preexecute = import os env = os.environ jobnumid = env["PBS_JOBID"] tmpdir = None if "TMPDIR" in env: tmpdir = env[ "TMPDIR" ].rstrip( "/" ) else: tmpdir = "." dir = "%s/%s" % ( tmpdir, jobnumid ) os.system("mkdir %s" % dir ) os.chdir( "%s" % dir ) os.environ["PATH"]+=":."
submit_res_pattern = ^(?P<id>\d*)\.\w+\.nikhef\.nl
When using Ganga
- When submitting jobs, use the PBS backend with the following settings:
yourjob.backend = PBS( queue = 'medium' )