Using RFIO or DCAP to access files on Nikhef Grid disks

Since in principle all AODs will be stored in Nikhef's basement, it is convenient to use a system that allows us to access these files from our desktops, or the stoomboot cluster. NFS will not work, because it cannot handle many parallel large data transfers. Different protocols have been developed to accomodate this. The nikhef DPM (Disk Pool Manager) uses RFIO, sara uses DCAP(Data Link Switching Client Access Protocol). To access these files, the root GFAL (Grid File Access Library) plugin can be used. GFAL determines whcih protocol to use. Access using RFIO/DCAP is simple, but there are a few issues.

Setting up use of DCAP/RFIO: Shell

A few environment variables have to be set. In bash:

source /global/ices/lcg/current/etc/profile.d/grid_env.sh
export LFC_HOST='lfc-atlas.grid.sara.nl'
export LCG_CATALOG_TYPE=lfc
export LCG_RFIO_TYPE=dpm
export DCACHE_CLIENT_ACTIVE=1
export X509_USER_PROXY=$HOME/.globus/gridproxy.pem

Where the last cariable is set to store look for your grid proxy in a place accessible from stoomboot. Use the following options with "voms-proxy-init":

voms-proxy-init -voms atlas -order /atlas/nl -out $HOME/.globus/gridproxy.pem

In csh you type:

source /global/ices/lcg/current/etc/profile.d/grid_env.csh
setenv LFC_HOST 'lfc-atlas.grid.sara.nl'
setenv LCG_CATALOG_TYPE lfc
setenv LCG_RFIO_TYPE dpm 
setenv DCACHE_CLIENT_ACTIVE 1
setenv X509_USER_PROXY $HOME/.globus/gridproxy.pem

Setting up use of DCAP/RFIO: Root

All precompiled versions of ROOT support the GFAL plugin. If you compiled your own version, you might have to recompile using the --with-dcap-libdir=, --with-dcap-incdir= and if you are on SLC3 also --with-gfal-libdir= and --with-gfal-incdir= flags set. There is a small issue that has to be solved, which is that the 64-bit ROOT version used by the ATLAS 13.0.30 Software for SLC4 64-bit installed on stoomboot does not seem to support the GFAL plugin or the RFIO protocol. This is as yet not solved. What does work is to use the 32-bits version of the atlas software and ROOT.

To open a file on the grid you need it's Logical File Name, or LFN. An LFN is stored on the Local File Catalogue (LFC), and it points to one or more duplicates of the file you want to open.

For instance, LFN's to all the produced TVNtuples can be found by typing:

lfc-ls /grid/atlas/users/nikhef/datasets/

You will see a list of all datasets, the AODs and the TVNtuples produced from them. If you want to open one of these files in ROOT, you type:

TFile* f=TFile::Open("gfal:lfn:///grid/atlas/users/nikhef/datasets/<datasetname>/<filename>")

You can then use it as any other root file. To open AOD's using the AthenaROOTAccess package, you can use the same format for the filename:

filename="gfal:lfn:/// <The LFN that points to your file"

And that's it!