Difference between revisions of "DQ2Client quick usage"

From Atlas Wiki
Jump to navigation Jump to search
Line 72: Line 72:
 
|}
 
|}
  
−
== List files in PFNs ==
+
== List dataset files in PFNs ==
 
The following command lists the physical SURLs of the files of a particular dataset.
 
The following command lists the physical SURLs of the files of a particular dataset.
 
<pre>
 
<pre>

Revision as of 15:30, 10 September 2008

Introduction

This page aims to help ATLAS users to quickly browse, search, retrieve ATLAS datasets for analysis. The page is organized by several use cases.

Setup environment

% source /project/atlas/nikhef/dq2/dq2_setup.(c)sh.NIKHEF
% voms-proxy-init -voms atlas

List datasets

The following command lists the merged FDRII AOD datasets of the Muon stream. 

% dq2-ls fdr08_run2*Muon*merge*AOD*
...
fdr08_run2.0052280.physics_Muon.merge.AOD.o3_r508_t45
fdr08_run2.0052301.physics_Muon.merge.AOD.o3_r508_t45/
fdr08_run2.0052283.physics_Muon.merge.AOD.o3_r508_t45
fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10
fdr08_run2.0052290.physics_Muon.merge.AOD.o3_f8_m10
fdr08_run2.0052293.physics_Muon.merge.AOD.o3_r508_t45_tid024345
fdr08_run2.0052283.physics_Muon.merge.AOD.o3_r508_t45/
fdr08_run2.0052300.physics_Muon.merge.AOD.o3_f26_m15
...

The one with "/" at the end indicates a dataset container.

List dataset locations

The following command lists the replica locations of dataset fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10 

% dq2-ls -r fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10
 INCOMPLETE:
   RAL-LCG2_DATADISK
 COMPLETE:
   PRAGUELCG2_DATADISK
   GOEGRID_DATADISK
   TRIUMF-LCG2_DATADISK
   ...
   NIKHEF-ELPROD_DATADISK
   ...

List dataset files

The following command lists the files in a particular dataset 

% dq2-ls -l fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10

fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10
[X]	fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0004.1	9C32CDB7-7638-DD11-8B46-0030487C782A	ad:6906bd22	3044690419
[X]	fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0007.1	8AEBE6D3-7638-DD11-A9B9-0030487CD836	ad:72aacdf0	3013049170
[X]	fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0002.1	96E34CB5-7638-DD11-B584-0019DB29C602	ad:be45ed57	2990039386
[ ]	fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0003.1	5C4857B2-7638-DD11-87BB-0030487CD762	ad:19ebf510	2991127754
[X]	fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0009.1	5E0F0BCC-7638-DD11-B69A-0030487CD938	ad:e96929bb	2008226831
[ ]	fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0005.1	526C74BC-7638-DD11-9959-0030487CD16C	ad:72859987	2994945608
...
Meaning of the returned table
column # example meaning
1 [X] or [ ] existence of the file in local site. Cross means it exists.
2 fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0004.1 logical file name
3 9C32CDB7-7638-DD11-8B46-0030487C782A Global Unique Identifier (GUID)
4 ad:6906bd22 checksum
5 3044690419 file size in byte

List dataset files in PFNs

The following command lists the physical SURLs of the files of a particular dataset. 

% dq2-ls -p -f fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10

fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10
srm://tbn18.nikhef.nl/dpm/nikhef.nl/home/atlas/atlasdatadisk/fdr08_run2/AOD/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0006.1
srm://tbn18.nikhef.nl/dpm/nikhef.nl/home/atlas/atlasdatadisk/fdr08_run2/AOD/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0008.1
srm://tbn18.nikhef.nl/dpm/nikhef.nl/home/atlas/atlasdatadisk/fdr08_run2/AOD/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0001.1
srm://tbn18.nikhef.nl/dpm/nikhef.nl/home/atlas/atlasdatadisk/fdr08_run2/AOD/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0005.1
srm://tbn18.nikhef.nl/dpm/nikhef.nl/home/atlas/atlasdatadisk/fdr08_run2/AOD/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0009.1
...

It returns a list of physical files located at your local site defined by $DQ2_LOCAL_SITE_ID environment variable.

You can however list the physical files located at another site by using the -L option: 

% dq2-ls -L CERN-PROD_DATADISK -p -f fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10

fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10
srm://srm-atlas.cern.ch/castor/cern.ch/grid/atlas/tzero/atlasdatadisk/fdr08_run2/AOD/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0006.1
srm://srm-atlas.cern.ch/castor/cern.ch/grid/atlas/tzero/atlasdatadisk/fdr08_run2/AOD/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0008.1
srm://srm-atlas.cern.ch/castor/cern.ch/grid/atlas/tzero/atlasdatadisk/fdr08_run2/AOD/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0001.1
srm://srm-atlas.cern.ch/castor/cern.ch/grid/atlas/tzero/atlasdatadisk/fdr08_run2/AOD/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0005.1
srm://srm-atlas.cern.ch/castor/cern.ch/grid/atlas/tzero/atlasdatadisk/fdr08_run2/AOD/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0009.1
srm://srm-atlas.cern.ch/castor/cern.ch/grid/atlas/tzero/atlasdatadisk/fdr08_run2/AOD/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10/fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10._0003.1
...

Get dataset

The following command downloads a dataset from your local site to your current working directory. 

% dq2-get fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10

The following command downloads a dataset from a specific remote site to your current working directory. 

% dq2-get -s CERN-PROD_DATADISK fdr08_run2.0052293.physics_Muon.merge.AOD.o3_f8_m10
Retrieved from "https://wiki.nikhef.nl/atlas/index.php?title=DQ2Client_quick_usage&oldid=2418"