Difference between revisions of "AMA in Athena 14 1 0"

From Atlas Wiki
Jump to navigation Jump to search
 
(47 intermediate revisions by 6 users not shown)
Line 1: Line 1:
===Setting up Athena 14.1.0===
+
=Setting up ATLAS Release 14.1.0 at NIKHEF =
 +
 
 +
Only once:
 +
<ul>
 +
  <li>Create a file: <tt>$HOME/cmthome/requirements</tt> (and modify <tt><username></tt>)
 
<pre>
 
<pre>
$ source /data/atlas/offline/14.1.0/cmtsite/setup.sh -tag=14.1.0,32,setup,opt
+
set CMTSITE STANDALONE
$ export CMTPATH=/project/atlas/users/<username>/14.1.0:$CMTPATH
+
set SITEROOT /data/atlas/offline/14.1.0
 +
macro ATLAS_TEST_AREA /project/atlas/users/<username>
 +
macro ATLAS_DIST_AREA ${SITEROOT}
 +
apply_tag projectArea
 +
macro SITE_PROJECT_AREA ${SITEROOT}
 +
macro EXTERNAL_PROJECT_AREA ${SITEROOT}
 +
apply_tag opt
 +
apply_tag setup
 +
apply_tag simpleTest
 +
use AtlasLogin AtlasLogin-* $(ATLAS_DIST_AREA)
 +
set CMTCONFIG i686-slc4-gcc34-opt
 
</pre>
 
</pre>
 +
  <li><tt>$ source /data/atlas/offline/14.1.0/CMT/v1r20p20080222/mgr/setup.sh</tt>
 +
  <li><tt>$ cmt config</tt> in <tt>$HOME/cmthome</tt>
 +
</ul>
  
 +
Everytime you log on:
 +
<pre>
 +
$ source $HOME/cmthome/setup.sh -tag=14.1.0,32
 +
</pre>
 +
 +
=AtlasModularAnalysis=
 +
 +
To obtain the code, do the following:
 
in <tt>/project/atlas/users/<username>/14.1.0/PhysicsAnalysis</tt>:
 
in <tt>/project/atlas/users/<username>/14.1.0/PhysicsAnalysis</tt>:
 
<pre>
 
<pre>
$ cvs -d /project/atlas/cvs/Athena co -r V00-00-04-04 AtlasModularAnalysis
+
$ cvs -d /project/atlas/cvs/Athena co -r V00-04-13 AtlasModularAnalysis
 +
</pre>
 +
Or, if you want to update from a previous version, in your PhysicsAnalysis/AtlasModularAnalysis directory type:
 +
<pre>
 +
$ cvs update -r V00-04-13
 
</pre>
 
</pre>
  
===AtlasModularAnalysis===
+
Compile the code as follows
 +
<pre>
 +
$ cd AtlasModularAnalysis/cmt
 +
$ cmt config
 +
$ . setup.sh
 +
$ gmake
 +
</pre>
 +
 
 +
 
 
There are three ways of running AMA:
 
There are three ways of running AMA:
 
<ul>
 
<ul>
 
  <li>Standalone (no AODs)
 
  <li>Standalone (no AODs)
 
  <li>with ARA (all formats)
 
  <li>with ARA (all formats)
  <li>in Athena (only AODs)
+
  <li>in Athena (only AODs) <b>Use this for FDR-II</b>
 
</ul>
 
</ul>
  
For FDR-II: try running AMA in Athena.
 
  
====Standalone====
+
== Using AMA within Athena (AMAAthena) ==
 +
 
 +
NB: All AtlasRelease (athena) compilation and runtime activities should
 +
be perform only in the <i>bash shell</i>
 +
 
 +
To set up the runtime environment for AMAAthena in conjuction with
 +
the grid file stager do the following from a <i>clean</i> shell.
 +
 
 +
<pre>
 +
$ source /global/ices/lcg/current/etc/profile.d/grid_env.sh
 +
$ voms-proxy-init
 +
(your pass phrase goes here)
 +
$ unset PYTHONPATH
 +
$ source $HOME/cmthome/setup.sh -tag=14.1.0,32
 +
$ export TMPDIR=/tmp 
 +
</pre>
 +
 
 +
This specific order is required to avoid a python clash between the
 +
grid middleware and the ATLAS release environment. The above sequence
 +
of command is required before you set up the AMA and AMAAthena code
 +
(as explained below) as well as everytime you login and want to
 +
run AMAAthena.
 +
 
 +
==== How to retrieve AMAAthena code and how to compile it ====
 +
 
 +
First install the AMA code itself as explained in the previous section.
 +
Then, starting in <tt>/project/atlas/users/<user>/14.1.0/PhysicsAnalysis</tt>
 +
do the following:
 +
 
 +
<pre>
 +
$ cd /project/atlas/users/<user>/14.1.0/PhysicsAnalysis
 +
$ cvs -d /project/atlas/cvs/Athena co -r V00-00-08 AMAAthena
 +
$ cd AMAAthena/cmt
 +
$ cmt config
 +
$ . setup.sh
 +
$ gmake
 +
</pre>
 +
 
 +
Or, if you want to update from a previous version, in your PhysicsAnalysis/AMAAthena directory type:
 +
<pre>
 +
$ cvs update -r V00-00-08
 +
</pre>
 +
instead of the checkout command.
 +
 
 +
==== How to run AMAAthena code ====
 +
 
 +
run AMAAthena as follows in the <tt>run/</tt> directory of the AMAAthena
 +
<pre>
 +
$ AMAAthenaDriver <sample> <NEvtMax> ConfigFile=<configfile>
 +
</pre>
 +
 
 +
where <SampleName> is a sample name created with define_grid_sample (part of AMA)
 +
and NumEvents is the number of events you want to process (-1 is all events)
 +
 
 +
The above jobOptions files is configured to run OK with both local files
 +
(e.g. in /data/atlas) and using the file stager (on the Tier-1). The stager
 +
infrastructure is automatically activated as soon as any file with the prefix
 +
'gridcopy:' is found in the InputCollections definition. Examples for both
 +
modes of operation are included AMAAthena_jobOptions_FDR2.py
 +
 
 +
To run AMAAthena in batch on stoomboot use the bsub script that is present
 +
in the run directory
 +
 
 +
$ ./bsub AMAAthenaDriver MySample -1
 +
 
 +
Note that the bsub included in run/ here was modified to be pure bash to avoid csh problems
 +
related ATLAS releases. <i>Any earlier private copies you may have will not work</i>.
 +
 
 +
===== Mapping DQ2 datasets to AMA samples =====
 +
A new script <tt>define_dq2_sample</tt> has been added to AMA V00-04-09. The latest version, to work with the most recent dq2-ls, is found from  AMA V00-04-11 onwards.
 +
to create AMA sample definition files of <i>any</i> DQ2 dataset.
 +
 
 +
<pre>
 +
$ define_dq2_sample fdr08_run2.0052291.physics_Egamma.merge.AOD.o2_f7_m8
 +
$ define_dq2_sample -n R52992EG fdr08_run2.0052291.physics_Egamma.merge.AOD.o2_f7_m8
 +
</pre>
 +
 
 +
By default the sample name is identical to the dataset name, but can be overruled with the optional -n argument. The <tt>define_dq2_</tt> sample script queries the DQ2 catalogue for the physical file names of the content files on the local SARA/NIKHEF Tier-1 and thus has no (hidden) assumption the storage layout. If a dataset does not exists or does not contain any files in the local replica an error message is returned.
 +
 
 +
== Using AMA Standalone ==
 
<pre>
 
<pre>
 
$ gmake -f Makefile.Standalone amacint
 
$ gmake -f Makefile.Standalone amacint
Line 26: Line 141:
 
</pre>
 
</pre>
  
==== With ARA====
+
== Using AMA with ARA ==
=====from kit=====
+
===from kit===
 
(no extra tags)
 
(no extra tags)
 
in <tt>AtlasModularAnalysis/cmt</tt>
 
in <tt>AtlasModularAnalysis/cmt</tt>
Line 36: Line 151:
 
</pre>
 
</pre>
  
=====with recommended tags=====
+
===with recommended tags===
 
(see also: https://twiki.cern.ch/twiki/bin/view/Atlas/AthenaROOTAccess#14_1_0 ):  
 
(see also: https://twiki.cern.ch/twiki/bin/view/Atlas/AthenaROOTAccess#14_1_0 ):  
 
<ul>
 
<ul>
Line 49: Line 164:
 
</ul>
 
</ul>
  
=====CMT issues=====
+
===issues===
 +
====CMT====
 
<tt>source setup.sh</tt> in <tt>AtlasModularAnalysis/cmt</tt> directory gives crash when running AMA with ARA.  
 
<tt>source setup.sh</tt> in <tt>AtlasModularAnalysis/cmt</tt> directory gives crash when running AMA with ARA.  
  
 
<b>Solution:</b> do another <tt>source setup.sh</tt> in the <tt>AthenaROOTAccess/cmt</tt> directory. Now you can run AMA with ARA without problems again.
 
<b>Solution:</b> do another <tt>source setup.sh</tt> in the <tt>AthenaROOTAccess/cmt</tt> directory. Now you can run AMA with ARA without problems again.
  
=====PYTHONPATH issues=====
+
====PYTHONPATH====
 
Sometimes ARA crashes just after initialisation. This is due to a wrongly set <tt>PYTHONPATH</tt>. To correct this:
 
Sometimes ARA crashes just after initialisation. This is due to a wrongly set <tt>PYTHONPATH</tt>. To correct this:
  
Line 65: Line 181:
 
running <tt>i686-slc4-gcc34-opt/CSCDriver.exe</tt> should be OK now.
 
running <tt>i686-slc4-gcc34-opt/CSCDriver.exe</tt> should be OK now.
  
==== Within Athena====
+
====JetCollectionCnv_p2====
 
+
https://hypernews.cern.ch/HyperNews/Atlas/get/PATDevelopment/270/1.html
Starting in <tt>/project/atlas/users/<user>/14.1.0/PhysicsAnalysis</tt>:
 
<ul>
 
  <li>first compile AMA
 
      <pre>
 
$ cd AtlasModularAnalysis/cmt
 
$ cmt config
 
$ . setup.sh
 
$ gmake
 
</pre>
 
  <li>copy AMAAthena and compile
 
<pre>
 
$ cp /data/atlas/users/gossie/Athena/AMAAthena .
 
$ cd AMAAthena/cmt
 
$ cmt config
 
$ . setup.sh
 
$ gmake
 
</pre>
 
</ul>
 
run AMAAthena with:
 
<pre>
 
$ athena AMAAthena_jobOptions_FDR.py
 
</pre>
 

Latest revision as of 18:30, 7 July 2008

Setting up ATLAS Release 14.1.0 at NIKHEF

Only once:

  • Create a file: $HOME/cmthome/requirements (and modify <username>)
    set CMTSITE STANDALONE
    set SITEROOT /data/atlas/offline/14.1.0
    macro ATLAS_TEST_AREA /project/atlas/users/<username>
    macro ATLAS_DIST_AREA ${SITEROOT}
    apply_tag projectArea
    macro SITE_PROJECT_AREA ${SITEROOT}
    macro EXTERNAL_PROJECT_AREA ${SITEROOT}
    apply_tag opt
    apply_tag setup
    apply_tag simpleTest
    use AtlasLogin AtlasLogin-* $(ATLAS_DIST_AREA)
    set CMTCONFIG i686-slc4-gcc34-opt
    
  • $ source /data/atlas/offline/14.1.0/CMT/v1r20p20080222/mgr/setup.sh
  • $ cmt config in $HOME/cmthome

Everytime you log on:

$ source $HOME/cmthome/setup.sh -tag=14.1.0,32

AtlasModularAnalysis

To obtain the code, do the following: in /project/atlas/users/<username>/14.1.0/PhysicsAnalysis:

$ cvs -d /project/atlas/cvs/Athena co -r V00-04-13 AtlasModularAnalysis

Or, if you want to update from a previous version, in your PhysicsAnalysis/AtlasModularAnalysis directory type:

$ cvs update -r V00-04-13

Compile the code as follows

$ cd AtlasModularAnalysis/cmt
$ cmt config
$ . setup.sh
$ gmake


There are three ways of running AMA:

  • Standalone (no AODs)
  • with ARA (all formats)
  • in Athena (only AODs) Use this for FDR-II


Using AMA within Athena (AMAAthena)

NB: All AtlasRelease (athena) compilation and runtime activities should be perform only in the bash shell

To set up the runtime environment for AMAAthena in conjuction with the grid file stager do the following from a clean shell.

$ source /global/ices/lcg/current/etc/profile.d/grid_env.sh
$ voms-proxy-init
(your pass phrase goes here)
$ unset PYTHONPATH
$ source $HOME/cmthome/setup.sh -tag=14.1.0,32
$ export TMPDIR=/tmp  

This specific order is required to avoid a python clash between the grid middleware and the ATLAS release environment. The above sequence of command is required before you set up the AMA and AMAAthena code (as explained below) as well as everytime you login and want to run AMAAthena.

How to retrieve AMAAthena code and how to compile it

First install the AMA code itself as explained in the previous section. Then, starting in /project/atlas/users/<user>/14.1.0/PhysicsAnalysis do the following:

$ cd /project/atlas/users/<user>/14.1.0/PhysicsAnalysis
$ cvs -d /project/atlas/cvs/Athena co -r V00-00-08 AMAAthena
$ cd AMAAthena/cmt
$ cmt config
$ . setup.sh
$ gmake

Or, if you want to update from a previous version, in your PhysicsAnalysis/AMAAthena directory type:

$ cvs update -r V00-00-08

instead of the checkout command.

How to run AMAAthena code

run AMAAthena as follows in the run/ directory of the AMAAthena

$ AMAAthenaDriver <sample> <NEvtMax> ConfigFile=<configfile>

where <SampleName> is a sample name created with define_grid_sample (part of AMA) and NumEvents is the number of events you want to process (-1 is all events)

The above jobOptions files is configured to run OK with both local files (e.g. in /data/atlas) and using the file stager (on the Tier-1). The stager infrastructure is automatically activated as soon as any file with the prefix 'gridcopy:' is found in the InputCollections definition. Examples for both modes of operation are included AMAAthena_jobOptions_FDR2.py

To run AMAAthena in batch on stoomboot use the bsub script that is present in the run directory

$ ./bsub AMAAthenaDriver MySample -1

Note that the bsub included in run/ here was modified to be pure bash to avoid csh problems related ATLAS releases. Any earlier private copies you may have will not work.

Mapping DQ2 datasets to AMA samples

A new script define_dq2_sample has been added to AMA V00-04-09. The latest version, to work with the most recent dq2-ls, is found from AMA V00-04-11 onwards. to create AMA sample definition files of any DQ2 dataset.

$ define_dq2_sample fdr08_run2.0052291.physics_Egamma.merge.AOD.o2_f7_m8
$ define_dq2_sample -n R52992EG fdr08_run2.0052291.physics_Egamma.merge.AOD.o2_f7_m8

By default the sample name is identical to the dataset name, but can be overruled with the optional -n argument. The define_dq2_ sample script queries the DQ2 catalogue for the physical file names of the content files on the local SARA/NIKHEF Tier-1 and thus has no (hidden) assumption the storage layout. If a dataset does not exists or does not contain any files in the local replica an error message is returned.

Using AMA Standalone

$ gmake -f Makefile.Standalone amacint
$ gmake -f Makefile.Standalone CSCDriver

Using AMA with ARA

from kit

(no extra tags) in AtlasModularAnalysis/cmt

$ cmt config
$ source setup.sh
$ gmake

with recommended tags

(see also: https://twiki.cern.ch/twiki/bin/view/Atlas/AthenaROOTAccess#14_1_0 ):

  • copy files from /data/atlas/users/gossie/Athena/14.1.0
  • compile (NB: use 'cmt bro gmake' to take care of dependencies ) (in all */cmt directories)
    $ cmt config
    $ source setup.sh
    $ gmake
    

issues

CMT

source setup.sh in AtlasModularAnalysis/cmt directory gives crash when running AMA with ARA.

Solution: do another source setup.sh in the AthenaROOTAccess/cmt directory. Now you can run AMA with ARA without problems again.

PYTHONPATH

Sometimes ARA crashes just after initialisation. This is due to a wrongly set PYTHONPATH. To correct this:

  • Copy /data/atlas/users/gossie/Athena/fix_AMA-ARA_PYTHONPATH.sh to anywhere you like
  • replace <username> with your username
  • $ source fix_AMA-ARA_PYTHONPATH.sh

running i686-slc4-gcc34-opt/CSCDriver.exe should be OK now.

JetCollectionCnv_p2

https://hypernews.cern.ch/HyperNews/Atlas/get/PATDevelopment/270/1.html