AMA on Stoomboot 14 2 22

From Atlas Wiki
Jump to navigation Jump to search

Running at Nikhef on SARA ESDs

Below is a (personal) recipe to run on ESDs at Nikhef. The instructions are when logged on interactively to Stoomboot. Instructions to submit a job to the Stoomboot batch can be found at the bottom of the page.

From your local desktop log in to Stoomboot:

qsub -X -I -q qlong

Go to your working area and make a working directory

cd /project/atlas/users/egge/testarea/
mkdir twiki
cd twiki
mkdir 14.2.22

Create a requirements file, for example

set SITEROOT /data/atlas/offline/14.2.22
macro ATLAS_TEST_AREA /project/atlas/users/egge/testarea/twiki
apply_tag opt
apply_tag setup
apply_tag simpleTest
apply_tag 14.2.22
apply_tag 32
apply_tag runtime
use AtlasLogin AtlasLogin-* $(ATLAS_DIST_AREA)
set CMTCONFIG i686-slc4-gcc34-opt


source /data/atlas/offline/14.2.22/CMT/v1r20p20080222/mgr/
cmt config

The following two lines should be done every next time to set up 14.2.22

cd /project/atlas/users/egge/testarea/twiki/14.2.22/

To be able to checkout AMA from CERN

/usr/kerberos/bin/kinit -5 egge@CERN.CH

Then actually checkout AMA

cmt co -r AMACore-00-00-02-01 PhysicsAnalysis/AnalysisCommon/AMA/AMACore
cmt co -r AMAAthena-00-00-07 PhysicsAnalysis/AnalysisCommon/AMA/AMAAthena

and compile

cd PhysicsAnalysis/AnalysisCommon/AMA/AMAAthena/cmt
cmt br gmake

In ANOTHER shell: set up dq2

source /project/atlas/nikhef/dq2/
voms-proxy-init -voms atlas

Check if all is ok with

voms-proxy-info -all

To see what SARA destinations exist

dq2-destinations | grep "SARA"

To my knowledge SARA-MATRIX_DATADISK is the one to use (experts, please comment). To see the available (complete) datasets on SARA-MATRIX_DATADISK

dq2-list-dataset-site -c SARA-MATRIX_DATADISK

Let's take "data08_cosmag.00090721.physics_MBTS_BCM_LUCID.recon.ESD.o4_f70" as an example

cd /project/atlas/users/egge/testarea/twiki/14.2.22/PhysicsAnalysis/AnalysisCommon/AMA/AMAAthena/
define_dq2_sample -n cosmag90721 data08_cosmag.00090721.physics_MBTS_BCM_LUCID.recon.ESD.o4_f70 SARA-MATRIX_DATADISK

BACK in Stoomboot shell. Also setup Grid here

cd /project/atlas/users/egge/testarea/twiki/14.2.22/PhysicsAnalysis/AnalysisCommon/AMA/AMAAthena/
source /project/atlas/nikhef/dq2/
voms-proxy-init -voms atlas

The warning "bash: /afs/ No such file or directory" seems to be harmless.

AFTER setting up Grid on Stoomboot ALWAYS do

source /project/atlas/users/egge/testarea/twiki/
export TMPDIR=/tmpdir

To make sure you have correct (14.2.22) Database settings please do

cp /project/atlas/users/egge/testarea/14.2.22/PhysicsAnalysis/AnalysisCommon/AMA/AMAAthena/ .

Then, modify the following in "share/"

EvtMax = 1000
sampleFile = "samples/cosmag90721.def" # File with input collections

Then copy "share/input_FileStagerRFCP" to "share/" and implement the following in "share/"

#stagetool.CpCommand = "rfcp"
#stagetool.CpArguments = []
#stagetool.OutfilePrefix = ""
stagetool.checkGridProxy = False


svcMgr.EventSelector.SkipBadFiles = True

Then all should be ready to run the job

cd /project/atlas/users/egge/testarea/twiki/14.2.22/PhysicsAnalysis/AnalysisCommon/AMA/AMAAthena
athena -s share/

Submitting in stoomboot batch mode

If you remember from above instruction, setting up the Grid on Stoomboot results in an error: "bash: /afs/ No such file or directory"

This can be overcome by using only the following line from /project/atlas/nikhef/dq2/

source /global/ices/lcg/current/etc/profile.d/

When submitting the job to Stoomboot you need to have your Grid proxy available on Stoomboot. To accomplish this do (thanks to Max: FileStager twiki) in a clean shell on your local desktop

source /global/ices/lcg/current/etc/profile.d/
voms-proxy-init -voms atlas -out $HOME/.globus/gridproxy.cert
export X509_USER_PROXY=${HOME}/.globus/gridproxy.cert

And then start your submit-job with

## script for restarting grid proxy certificate
source /global/ices/lcg/current/etc/profile.d/
export X509_USER_PROXY=${HOME}/.globus/gridproxy.cert
voms-proxy-init -voms atlas -noregen

#plus rest of instructions

followed by the 'rest of the instructions'. As a personal example I used the following


source /global/ices/lcg/current/etc/profile.d/
export X509_USER_PROXY=${HOME}/.globus/gridproxy.cert
voms-proxy-init -voms atlas -noregen

cd /project/atlas/users/egge/testarea/14.2.22/

export TMPDIR=/tmpdir
mkdir /tmpdir/egge

cd /project/atlas/users/egge/testarea/14.2.22/PhysicsAnalysis/AnalysisCommon/AMA/AMAAthena/


athena -s

Which is submitted (to qlong que!) from local desktop by

qsub -V -q qlong

Have fun!