Difference between revisions of "AMA in Athena 14 1 0"

From Atlas Wiki
Jump to navigation Jump to search
 
(30 intermediate revisions by 5 users not shown)
Line 1: Line 1:
 
=Setting up ATLAS Release 14.1.0 at NIKHEF =
 
=Setting up ATLAS Release 14.1.0 at NIKHEF =
 
= WVE WARNING UNDER RECONSTRUCTION WILL UPDATE TONIGHT (Fri June 6) =
 
  
 
Only once:
 
Only once:
Line 34: Line 32:
 
in <tt>/project/atlas/users/<username>/14.1.0/PhysicsAnalysis</tt>:
 
in <tt>/project/atlas/users/<username>/14.1.0/PhysicsAnalysis</tt>:
 
<pre>
 
<pre>
$ cvs -d /project/atlas/cvs/Athena co -r V00-04-06 AtlasModularAnalysis
+
$ cvs -d /project/atlas/cvs/Athena co -r V00-04-13 AtlasModularAnalysis
 +
</pre>
 +
Or, if you want to update from a previous version, in your PhysicsAnalysis/AtlasModularAnalysis directory type:
 +
<pre>
 +
$ cvs update -r V00-04-13
 +
</pre>
 +
 
 +
Compile the code as follows
 +
<pre>
 +
$ cd AtlasModularAnalysis/cmt
 +
$ cmt config
 +
$ . setup.sh
 +
$ gmake
 
</pre>
 
</pre>
  
Line 42: Line 52:
 
  <li>Standalone (no AODs)
 
  <li>Standalone (no AODs)
 
  <li>with ARA (all formats)
 
  <li>with ARA (all formats)
  <li>in Athena (only AODs)
+
  <li>in Athena (only AODs) <b>Use this for FDR-II</b>
 
</ul>
 
</ul>
  
For FDR-II: try running AMA in Athena.
 
  
 
== Using AMA within Athena (AMAAthena) ==
 
== Using AMA within Athena (AMAAthena) ==
  
<i>NB: All AtlasRelease (athena) compilation and runtime activities should
+
NB: All AtlasRelease (athena) compilation and runtime activities should
be perform only in the bash shell</i>
+
be perform only in the <i>bash shell</i>
  
 
To set up the runtime environment for AMAAthena in conjuction with
 
To set up the runtime environment for AMAAthena in conjuction with
Line 59: Line 68:
 
$ voms-proxy-init
 
$ voms-proxy-init
 
(your pass phrase goes here)
 
(your pass phrase goes here)
$ PYTHONPATH=
+
$ unset PYTHONPATH
 
$ source $HOME/cmthome/setup.sh -tag=14.1.0,32
 
$ source $HOME/cmthome/setup.sh -tag=14.1.0,32
$ export TMPDIR=/tmp  (ONLY when not submitting to stoomboot by batch, which sets TMPDIR automatically)
+
$ export TMPDIR=/tmp   
 
</pre>
 
</pre>
  
Line 76: Line 85:
 
do the following:
 
do the following:
  
<ul>
 
  <li>first compile AMA
 
      <pre>
 
$ cd AtlasModularAnalysis/cmt
 
$ cmt config
 
$ . setup.sh
 
$ gmake
 
</pre>
 
  <li>copy AMAAthena and compile
 
 
<pre>
 
<pre>
 
$ cd /project/atlas/users/<user>/14.1.0/PhysicsAnalysis
 
$ cd /project/atlas/users/<user>/14.1.0/PhysicsAnalysis
$ cp -r /data/atlas/users/gossie/Athena/AMAAthena .
+
$ cvs -d /project/atlas/cvs/Athena co -r V00-00-08 AMAAthena
 
$ cd AMAAthena/cmt
 
$ cd AMAAthena/cmt
 
$ cmt config
 
$ cmt config
Line 93: Line 93:
 
$ gmake
 
$ gmake
 
</pre>
 
</pre>
</ul>
+
 
 +
Or, if you want to update from a previous version, in your PhysicsAnalysis/AMAAthena directory type:
 +
<pre>
 +
$ cvs update -r V00-00-08
 +
</pre>
 +
instead of the checkout command.
  
 
==== How to run AMAAthena code ====
 
==== How to run AMAAthena code ====
  
run AMAAthena with:
+
run AMAAthena as follows in the <tt>run/</tt> directory of the AMAAthena
 
<pre>
 
<pre>
$ athena AMAAthena_jobOptions_FDR.py
+
$ AMAAthenaDriver <sample> <NEvtMax> ConfigFile=<configfile>
 
</pre>
 
</pre>
 +
 +
where <SampleName> is a sample name created with define_grid_sample (part of AMA)
 +
and NumEvents is the number of events you want to process (-1 is all events)
 +
 +
The above jobOptions files is configured to run OK with both local files
 +
(e.g. in /data/atlas) and using the file stager (on the Tier-1). The stager
 +
infrastructure is automatically activated as soon as any file with the prefix
 +
'gridcopy:' is found in the InputCollections definition. Examples for both
 +
modes of operation are included AMAAthena_jobOptions_FDR2.py
 +
 +
To run AMAAthena in batch on stoomboot use the bsub script that is present
 +
in the run directory
 +
 +
$ ./bsub AMAAthenaDriver MySample -1
 +
 +
Note that the bsub included in run/ here was modified to be pure bash to avoid csh problems
 +
related ATLAS releases. <i>Any earlier private copies you may have will not work</i>.
 +
 +
===== Mapping DQ2 datasets to AMA samples =====
 +
A new script <tt>define_dq2_sample</tt> has been added to AMA V00-04-09. The latest version, to work with the most recent dq2-ls, is found from  AMA V00-04-11 onwards.
 +
to create AMA sample definition files of <i>any</i> DQ2 dataset.
 +
 +
<pre>
 +
$ define_dq2_sample fdr08_run2.0052291.physics_Egamma.merge.AOD.o2_f7_m8
 +
$ define_dq2_sample -n R52992EG fdr08_run2.0052291.physics_Egamma.merge.AOD.o2_f7_m8
 +
</pre>
 +
 +
By default the sample name is identical to the dataset name, but can be overruled with the optional -n argument. The <tt>define_dq2_</tt> sample script queries the DQ2 catalogue for the physical file names of the content files on the local SARA/NIKHEF Tier-1 and thus has no (hidden) assumption the storage layout. If a dataset does not exists or does not contain any files in the local replica an error message is returned.
  
 
== Using AMA Standalone ==
 
== Using AMA Standalone ==

Latest revision as of 19:30, 7 July 2008

Setting up ATLAS Release 14.1.0 at NIKHEF

Only once:

  • Create a file: $HOME/cmthome/requirements (and modify <username>)
    set CMTSITE STANDALONE
    set SITEROOT /data/atlas/offline/14.1.0
    macro ATLAS_TEST_AREA /project/atlas/users/<username>
    macro ATLAS_DIST_AREA ${SITEROOT}
    apply_tag projectArea
    macro SITE_PROJECT_AREA ${SITEROOT}
    macro EXTERNAL_PROJECT_AREA ${SITEROOT}
    apply_tag opt
    apply_tag setup
    apply_tag simpleTest
    use AtlasLogin AtlasLogin-* $(ATLAS_DIST_AREA)
    set CMTCONFIG i686-slc4-gcc34-opt
    
  • $ source /data/atlas/offline/14.1.0/CMT/v1r20p20080222/mgr/setup.sh
  • $ cmt config in $HOME/cmthome

Everytime you log on:

$ source $HOME/cmthome/setup.sh -tag=14.1.0,32

AtlasModularAnalysis

To obtain the code, do the following: in /project/atlas/users/<username>/14.1.0/PhysicsAnalysis:

$ cvs -d /project/atlas/cvs/Athena co -r V00-04-13 AtlasModularAnalysis

Or, if you want to update from a previous version, in your PhysicsAnalysis/AtlasModularAnalysis directory type:

$ cvs update -r V00-04-13

Compile the code as follows

$ cd AtlasModularAnalysis/cmt
$ cmt config
$ . setup.sh
$ gmake


There are three ways of running AMA:

  • Standalone (no AODs)
  • with ARA (all formats)
  • in Athena (only AODs) Use this for FDR-II


Using AMA within Athena (AMAAthena)

NB: All AtlasRelease (athena) compilation and runtime activities should be perform only in the bash shell

To set up the runtime environment for AMAAthena in conjuction with the grid file stager do the following from a clean shell.

$ source /global/ices/lcg/current/etc/profile.d/grid_env.sh
$ voms-proxy-init
(your pass phrase goes here)
$ unset PYTHONPATH
$ source $HOME/cmthome/setup.sh -tag=14.1.0,32
$ export TMPDIR=/tmp  

This specific order is required to avoid a python clash between the grid middleware and the ATLAS release environment. The above sequence of command is required before you set up the AMA and AMAAthena code (as explained below) as well as everytime you login and want to run AMAAthena.

How to retrieve AMAAthena code and how to compile it

First install the AMA code itself as explained in the previous section. Then, starting in /project/atlas/users/<user>/14.1.0/PhysicsAnalysis do the following:

$ cd /project/atlas/users/<user>/14.1.0/PhysicsAnalysis
$ cvs -d /project/atlas/cvs/Athena co -r V00-00-08 AMAAthena
$ cd AMAAthena/cmt
$ cmt config
$ . setup.sh
$ gmake

Or, if you want to update from a previous version, in your PhysicsAnalysis/AMAAthena directory type:

$ cvs update -r V00-00-08

instead of the checkout command.

How to run AMAAthena code

run AMAAthena as follows in the run/ directory of the AMAAthena

$ AMAAthenaDriver <sample> <NEvtMax> ConfigFile=<configfile>

where <SampleName> is a sample name created with define_grid_sample (part of AMA) and NumEvents is the number of events you want to process (-1 is all events)

The above jobOptions files is configured to run OK with both local files (e.g. in /data/atlas) and using the file stager (on the Tier-1). The stager infrastructure is automatically activated as soon as any file with the prefix 'gridcopy:' is found in the InputCollections definition. Examples for both modes of operation are included AMAAthena_jobOptions_FDR2.py

To run AMAAthena in batch on stoomboot use the bsub script that is present in the run directory

$ ./bsub AMAAthenaDriver MySample -1

Note that the bsub included in run/ here was modified to be pure bash to avoid csh problems related ATLAS releases. Any earlier private copies you may have will not work.

Mapping DQ2 datasets to AMA samples

A new script define_dq2_sample has been added to AMA V00-04-09. The latest version, to work with the most recent dq2-ls, is found from AMA V00-04-11 onwards. to create AMA sample definition files of any DQ2 dataset.

$ define_dq2_sample fdr08_run2.0052291.physics_Egamma.merge.AOD.o2_f7_m8
$ define_dq2_sample -n R52992EG fdr08_run2.0052291.physics_Egamma.merge.AOD.o2_f7_m8

By default the sample name is identical to the dataset name, but can be overruled with the optional -n argument. The define_dq2_ sample script queries the DQ2 catalogue for the physical file names of the content files on the local SARA/NIKHEF Tier-1 and thus has no (hidden) assumption the storage layout. If a dataset does not exists or does not contain any files in the local replica an error message is returned.

Using AMA Standalone

$ gmake -f Makefile.Standalone amacint
$ gmake -f Makefile.Standalone CSCDriver

Using AMA with ARA

from kit

(no extra tags) in AtlasModularAnalysis/cmt

$ cmt config
$ source setup.sh
$ gmake

with recommended tags

(see also: https://twiki.cern.ch/twiki/bin/view/Atlas/AthenaROOTAccess#14_1_0 ):

  • copy files from /data/atlas/users/gossie/Athena/14.1.0
  • compile (NB: use 'cmt bro gmake' to take care of dependencies ) (in all */cmt directories)
    $ cmt config
    $ source setup.sh
    $ gmake
    

issues

CMT

source setup.sh in AtlasModularAnalysis/cmt directory gives crash when running AMA with ARA.

Solution: do another source setup.sh in the AthenaROOTAccess/cmt directory. Now you can run AMA with ARA without problems again.

PYTHONPATH

Sometimes ARA crashes just after initialisation. This is due to a wrongly set PYTHONPATH. To correct this:

  • Copy /data/atlas/users/gossie/Athena/fix_AMA-ARA_PYTHONPATH.sh to anywhere you like
  • replace <username> with your username
  • $ source fix_AMA-ARA_PYTHONPATH.sh

running i686-slc4-gcc34-opt/CSCDriver.exe should be OK now.

JetCollectionCnv_p2

https://hypernews.cern.ch/HyperNews/Atlas/get/PATDevelopment/270/1.html