Difference between revisions of "Access to data at NIKHEF tier"
Jump to navigation
Jump to search
m (→'''LFC''') |
|||
| (5 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| + | <i>NOTE: For all this a Grid certificate [http://www.nikhef.nl/grid] is required.</i> | ||
| + | |||
| + | =='''Introduction'''== | ||
There are maybe multiple ways to access data on the LCG. Though the two methods explained here are via LFC (LCG File Catalog) and SRM (Storage Resource Management). I'm just documenting what I didn't know. So please, correct things if they are wrong! | There are maybe multiple ways to access data on the LCG. Though the two methods explained here are via LFC (LCG File Catalog) and SRM (Storage Resource Management). I'm just documenting what I didn't know. So please, correct things if they are wrong! | ||
<ul> | <ul> | ||
| − | <li>LFC is | + | <li>The LFC is the file catalog at a Grid site that stores the information on which site a file can be found. This means, for example, that a file registered in the NIKHEF LFC, might be stored for at a CERN SE (Storage Element).</li> |
| − | <li>SRM is a procotol that enables file access only at a specific SE.</li> | + | <li>SRM is a procotol that enables file access only at a specific SE. At NIKHEF, DPM (Disk Pool Manager) is used for the SRM interface. (dCache being the other solution, used at eg. Fermilab).</li> |
</ul> | </ul> | ||
| − | |||
| − | |||
=='''LFC'''== | =='''LFC'''== | ||
| − | |||
Everytime you start a new session, you need to setup the environment for Grid access: | Everytime you start a new session, you need to setup the environment for Grid access: | ||
<pre> | <pre> | ||
source /global/ices/lcg/current/etc/profile.d/grid_env.csh | source /global/ices/lcg/current/etc/profile.d/grid_env.csh | ||
| + | grid-proxy-init | ||
</pre> | </pre> | ||
| + | The next two variables are mandatory to set: | ||
| + | <pre> | ||
| + | setenv LFC_HOST lfc03.nikhef.nl | ||
| + | setenv VO_DEFAULT_SE tbn18.nikhef.nl | ||
| + | </pre> | ||
| + | The CERN Atlas LFC host <tt>prod-lfc-atlas-central.cern.ch</tt> or SARA <tt>mu11.matrix.sara.nl</tt> could be used instead. <tt>lfc-atlas-test.cern.ch</tt> seems to be discouraged by the Grid people. <i><font color=red>need to check what's best</font></i>. | ||
| + | Details on the available commands can be found in [http://egee.itep.ru/User_Guide.html here]. | ||
| + | <ul> | ||
| + | <li>show the contents of a directory:<br> | ||
| + | <tt>lfc-ls /grid/atlas</tt> | ||
| + | <li>copy file from Grid to local file:<br> | ||
| + | <tt>lcg-cp -v --vo atlas lfn:/grid/atlas/MYFILE file:/tmp/MYFILE</tt> | ||
| + | </ul> | ||
| − | =='''DPM/DPNS | + | =='''SRM/DPM/DPNS'''== |
| − | + | To access files on the NIKHEF SE directly, the following two variables should be set: | |
| − | |||
<pre> | <pre> | ||
setenv DPM_HOST tbn18.nikhef.nl | setenv DPM_HOST tbn18.nikhef.nl | ||
| Line 24: | Line 37: | ||
</pre> | </pre> | ||
| − | dpns-ls -l /dpm/nikhef.nl/home/atlas/MYFILE | + | The next two commands are handy: |
| − | + | <ul> | |
| − | srmcp srm://tbn18.nikhef.nl:8443/dpm/nikhef.nl/home/atlas/MYFILE file:////tmp/MYFILE | + | <li>To have a directory listing:<br> |
| + | <tt>dpns-ls -l /dpm/nikhef.nl/home/atlas/MYFILE</tt></li> | ||
| + | <li>To copy a file:<br> | ||
| + | <tt>srmcp srm://tbn18.nikhef.nl:8443/dpm/nikhef.nl/home/atlas/MYFILE file:////tmp/MYFILE</tt><br> | ||
| + | <i>Note the four slashes!</i></li> | ||
| + | </ul> | ||
Latest revision as of 13:45, 3 May 2007
NOTE: For all this a Grid certificate [1] is required.
Introduction
There are maybe multiple ways to access data on the LCG. Though the two methods explained here are via LFC (LCG File Catalog) and SRM (Storage Resource Management). I'm just documenting what I didn't know. So please, correct things if they are wrong!
- The LFC is the file catalog at a Grid site that stores the information on which site a file can be found. This means, for example, that a file registered in the NIKHEF LFC, might be stored for at a CERN SE (Storage Element).
- SRM is a procotol that enables file access only at a specific SE. At NIKHEF, DPM (Disk Pool Manager) is used for the SRM interface. (dCache being the other solution, used at eg. Fermilab).
LFC
Everytime you start a new session, you need to setup the environment for Grid access:
source /global/ices/lcg/current/etc/profile.d/grid_env.csh grid-proxy-init
The next two variables are mandatory to set:
setenv LFC_HOST lfc03.nikhef.nl setenv VO_DEFAULT_SE tbn18.nikhef.nl
The CERN Atlas LFC host prod-lfc-atlas-central.cern.ch or SARA mu11.matrix.sara.nl could be used instead. lfc-atlas-test.cern.ch seems to be discouraged by the Grid people. need to check what's best.
Details on the available commands can be found in here.
- show the contents of a directory:
lfc-ls /grid/atlas - copy file from Grid to local file:
lcg-cp -v --vo atlas lfn:/grid/atlas/MYFILE file:/tmp/MYFILE
SRM/DPM/DPNS
To access files on the NIKHEF SE directly, the following two variables should be set:
setenv DPM_HOST tbn18.nikhef.nl setenv DPNS_HOST tbn18.nikhef.nl
The next two commands are handy:
- To have a directory listing:
dpns-ls -l /dpm/nikhef.nl/home/atlas/MYFILE - To copy a file:
srmcp srm://tbn18.nikhef.nl:8443/dpm/nikhef.nl/home/atlas/MYFILE file:////tmp/MYFILE
Note the four slashes!