Difference between revisions of "NIKHEF PAT workshop Oct2006"

From Atlas Wiki
Jump to navigation Jump to search
 
(126 intermediate revisions by one other user not shown)
Line 1: Line 1:
 +
==Pre-Tutorial==
  
−
==Set up CMT==
+
*This pre-Tutorial is copy of the one given by Ketevi A. Assamagan, adapted to work in 12.0.2
  
−
* Login to lxplus at CERN and create a working directory called "Tutorial". We will use the release 12.0.2 this tutorial:
+
*here is a link to other Tutorials by Ketevi, and other usful PAT links:
 +
[https://twiki.cern.ch/twiki/bin/view/Atlas/PhysicsAnalysisTools, PAT]
 +
 
 +
*Please follow the following pre-Tutorial before the date of the Workshop. It is a very straight forward, tutorial and will help to set up the athena enviroment for the workshop, so we can focus our time in more interested things
 +
 
 +
[http://www.nikhef.nl/pub/experiments/atlaswiki/index.php/NIKHEF_PAT_workshop_Oct2006_pre-Tutorial Pre-Tutorial link]
 +
 
 +
During the workshop we'll browse over the code that we've used in the pre-Tutorial to try and uderstand it. Later we'll focused on the TAG and AAN selection.
 +
 
 +
==Event Selection==
 +
First we're going to generate a new AAN that we'll use later for selection.  
 
<pre>
 
<pre>
−
ssh lxplus.cern.ch
+
cd ~scratch0/Workshop/
−
cd scratch0
+
source setup.sh -tag=12.0.2,opt
−
mkdir Tutorial
+
cd 12.0.2/PhysicsAnalysis/AnalysisCommon/UserAnalysis
−
cd Tutorial
+
cp ~ordonez/public/Workshop_files/src/AnalysisSkeleton.cxx src/.
−
mkdir 12.0.2
+
cp ~ordonez/public/Workshop_files/UserAnalysis/AnalysisSkeleton.h UserAnalysis/.
 +
cd cmt
 +
gmake
 +
cd ..
 +
mkdir run2
 +
cd run2
 +
cp ~ordonez/public/Workshop_files/run2/AnalysisSkeleton_jobOptions.py .
 +
cp ~ordonez/public/Workshop_files/data2/PoolFileCatalog.xml .
 +
cp ~ordonez/public/Workshop_files/data2/mc11.004202.ZmumuJimmy.recon.AOD.v11000401.py .
 +
athena AnalysisSkeleton_jobOptions.py | tee athenaOut_AnalysisSkeleton.log
 
</pre>
 
</pre>
  
−
* Create a file called "requirements" and setup CMT. My requirements file looks like this where my working area for 12.0.2 is ${HOME}/scratch0/Workshop/12.0.2:
+
This will generate the file AnalysisSkeleton.aan.root.
 +
 
 +
==Event Selection using TAG==
 +
*First we generate our own TAG.
 +
 
 +
*Still in the new run2 directory:
 +
 
 
<pre>
 
<pre>
−
set  CMTSITE  CERN
+
cp ~ordonez/public/Workshop_files/run2/Main_topOptions.py .
−
set  SITEROOT /afs/cern.ch
+
</pre>
−
macro ATLAS_DIST_AREA ${SITEROOT}/atlas/software/dist
+
 
−
                     
+
*Edit Main_topOptions.py and uncomment the line
−
macro ATLAS_GROUP_AREA "/afs/cern.ch/atlas/groups/PAT/Tutorial"
+
<pre>
 +
### Create TAG file
 +
doWriteTAG=True
 +
</pre>
 
   
 
   
−
macro ATLAS_TEST_AREA "" \
+
<pre>
−
        12.0.2 "${HOME}/scratch0/Workshop/12.0.2"
+
athena Main_topOptions.py | tee athenaOut.log
−
+
</pre>
−
+
 
−
apply_tag oneTest
+
*This will generate the file called ZmumuOnAOD_TAG.root
−
+
 
−
use AtlasLogin AtlasLogin-* $(ATLAS_DIST_AREA)
+
*Now we are ready to run our algorithms with a TAG selection on the fly. Edit Main_topOptions.py again comment out:
 +
<pre>
 +
#doWriteTAG=True
 +
</pre>
 +
 
 +
*and uncomment
 +
<pre>
 +
doRunAODwithSelection=True
 +
 
 +
and
 +
 
 +
useTag=True
 +
</pre>
 +
 
 +
*Copy:
 +
<pre>
 +
cp ~ordonez/public/Workshop_files/run2/ZmumuOnAOD_new_jobOptions.py .
 +
</pre>
 +
*And run again. Note that only will run the events that satisfy the query:
 +
EventSelector.Query = "NLooseMuon>0 && NLooseMuon<3 && abs(LooseMuonEta1)<2.5 && abs(LooseMuonEta2)<2.5 && LooseMuonPt1>10000 && LooseMuonPt2>10000"
 +
 
 +
 
 +
<pre>
 +
athena Main_topOptions.py | tee athenaOut.log
 +
</pre>
 +
 
 +
*As an output you have ZmumuOnAOD_withTAGSelection.aan.root. You can have a look at it in root to make sure that the query has been satisfied
 +
 
 +
*Now we'll rewrite our own AOD containeing only the events selected. We'll do it for only 50 events to gain some time
 +
 
 +
*Edit again Main_topOptions.py and change:
 +
 
 +
<pre>
 +
TheApp.EvtMax = 500
 +
 
 +
to
 +
 
 +
TheApp.EvtMax = 50
 +
</pre>
 +
 
 +
*Comment out
 +
<pre>
 +
#doRunAODwithSelection=True
 +
</pre>
 +
 
 +
*uncomment:
 +
<pre>
 +
doRewriteAOD=True
 +
</pre>
 +
 
 +
and run once more
 +
 
 +
<pre>
 +
athena Main_topOptions.py | tee athenaOut.log
 +
</pre>
 +
 
 +
You'll see a new Zmumu_new_AOD_withTAGSelection.root file has been created
 +
 
 +
==Event Selection using AAN==
 +
*Follow as in the previous section but useANN=True instead of useTAG=True. In the end you should have the following two files:
 +
ZmumuOnAOD_withAANSelection.aan.root
 +
Zmumu_new_AOD_withAANSelection.root
 +
 
 +
==Interactive Analysis in ATHENA==
 +
*In this exercise, we will see how to do interactive analysis in ATHENA, how to browse the Raw Data in the AOD. Proceed as follows:
 +
<pre>
 +
get_files Interactive_topO.py
 +
</pre>
 +
 
 +
*Edit Interactive_topO.py and replace this line
 +
<pre>
 +
EventSelector.InputCollections = [ "AOD.pool.root" ]
 +
</pre>
 +
*with this line:
 +
<pre>
 +
include ( "mc11.004202.ZmumuJimmy.recon.AOD.v11000401.py" )
 +
</pre>
 +
 
 +
*run
 +
<pre>
 +
athena -i Interactive_topO.py
 +
</pre>
 +
*Initialize application manager
 +
<pre>
 +
athena>theApp.initialize()
 +
</pre>
 +
*Run 1 event
 +
<pre>
 +
athena>theApp.nextEvent()
 +
</pre>
 +
 
 +
*Retrieve Muons from AOD
 +
<pre>
 +
athena>mcon = PyParticleTools.getMuons("MuidMuonCollection")
 +
</pre>
 +
 
 +
*Other methods are defined in [http://tmaeno.home.cern.ch/tmaeno/PyParticleTools/index.html PyParticleTools]
 +
 
 +
*Get the first muon
 +
<pre>
 +
athena>m = mcon[0]
 +
</pre>
 +
 
 +
*Get a list of methods
 +
<pre>
 +
athena>dir(m)
 +
</pre>
 +
 
 +
*See pT
 +
<pre>
 +
athena>m.pt()
 
</pre>
 
</pre>
  
−
*Then do the following:
+
*Get TrackParticle via ElementLink
 
<pre>
 
<pre>
−
source /afs/cern.ch/sw/contrib/CMT/v1r18p20060301/mgr/setup.sh
+
athena>tp = m.track()
−
cmt config
+
athena>dir(tp)
 +
athena>tp.pt()
 
</pre>
 
</pre>
  
−
==Setup for the release 12.0.2==
+
*Retrieve the MC Truth
−
*Do the following:
 
 
<pre>
 
<pre>
−
source setup.sh -tag=12.0.2
+
athena>mcc = PyTruthTools.getMcEvents("TruthEvent")
 +
athena>mc = mcc[0]
 +
athena>mc.alphaQCD()
 
</pre>
 
</pre>
  
−
*Check that the CMT path is correct by doing this
+
*Dump Data Store
 
<pre>
 
<pre>
−
echo $CMTPATH
+
athena>dumpSG()
 
</pre>
 
</pre>
  
−
*Stuff like this should be printed to your screen:
+
*Run next event
 
<pre>
 
<pre>
−
/afs/cern.ch/user/o/ordonez/scratch0/Workshop/12.0.2:
+
athena>theApp.nextEvent()
−
/afs/cern.ch/atlas/software/builds/AtlasOffline/12.0.2
 
 
</pre>
 
</pre>
  
−
*You see that the path to your working directory, the path to the release directory and the external are set correctly - do not proceed unless this is done correctly.
+
Rewind event counter
−
==Run Time Setup==
 
−
*Go to your working area and check out the follow package:
 
 
<pre>
 
<pre>
−
cd 12.0.2
+
athena>theApp.seek(0)
−
cmt co -r UserAnalysis-00-08-05 PhysicsAnalysis/AnalysisCommon/UserAnalysis
 
 
</pre>
 
</pre>
  
−
*Now compile everything as follows:
+
*See pT distribution of muons
 
<pre>
 
<pre>
−
cd PhysicsAnalysis/AnalysisCommon/UserAnalysis/cmt
+
athena> plot("MuonContainer#MuidMuonCollection","$x.pt()",nEvent=5)
−
cmt config
+
</pre>
 +
 
 +
 
 +
 
 +
==Analysis with the EventView==
 +
*Due to the time I won't cover this section during the Workshop. But now you should have the basic knowledge requiered to understand it. You can learn about the [https://twiki.cern.ch/twiki/bin/view/Atlas/EventView Event View] here
 +
 
 +
== Analysis on the GRID ==
 +
* In the following examples we'll run on Zmumu datasets.
 +
*This will be a brief introduction to GRID tools for those of you who already went through the trouble of getting a GRID certificate.
 +
 
 +
*First we should set up DQ2 tools:
 +
 
 +
<pre>
 +
source /afs/usatlas.bnl.gov/Grid/Don-Quijote/dq2_user_client/setup.csh.CERN
 +
</pre>
 +
 
 +
==PANDA/PATHENA==
 +
*For further details visit [https://twiki.cern.ch/twiki/bin/view/Atlas/DAonPanda panda wiki]
 +
*Setup PATHENA :
 +
<pre>
 +
cd ~/scratch0/Workshop/12.0.2/PhysicsAnalysis/AnalysisCommon/UserAnalysis
 +
cvs update -r UserAnalysis-00-08-13 -A python
 +
cvs update -r UserAnalysis-00-08-13 -A share/pathena
 +
cvs update -r UserAnalysis-00-08-13 -A share/pathena_util
 +
cvs update -r UserAnalysis-00-08-13 -A share/ConfigExtractor.py
 +
cd cmt
 
source setup.sh
 
source setup.sh
−
cmt broadcast gmake
+
gmake
 
cd ../run
 
cd ../run
 
</pre>
 
</pre>
  
−
*To test that everything has been done correctly, do:
+
*run pathena
 +
Note: '''change user.GustavoOrdonezSanz.00008.AAN.root with an appropiate name!!'''
 +
<pre>
 +
pathena ZmumuOnAOD_jobOptions.py --inDS 'csc11.005145.PythiaZmumu.recon.AOD.v11004103' --outDS \
 +
'user.GustavoOrdonezSanz.00008.AAN.root'
 +
</pre>
 +
 
 +
*For job status monitoring visit: [http://gridui02.usatlas.bnl.gov:25880/server/pandamon/query Monitoring]
 +
 
 +
 
 +
 
 +
==DQ2 Tools==
 +
*For furhter details visit [https://twiki.cern.ch/twiki/bin/view/Atlas/UsingDQ2 DQ2 wiki]
 +
*This tools make your life easier when working on the GRID
 
<pre>
 
<pre>
−
get_files HelloWorldOptions.py
+
dq2_ls *Zmumu*AOD*
−
athena.py -b HelloWorldOptions.py
 
 
</pre>
 
</pre>
  
−
*Stuff like this should be printed to the screen:
+
*Have a look at the files contained in one of the sets
 
<pre>
 
<pre>
−
HelloWorld          INFO execute()
+
dq2_ls -f <datasetname>
−
HelloWorld          INFO An INFO message
 
−
HelloWorld          WARNING A WARNING message
 
−
HelloWorld          ERROR An ERROR message
 
−
HelloWorld          FATAL A FATAL error message
 
−
AthenaEventLoopMgr  INFO  ===>>> end of event 9    <<<===
 
−
HistorySvc          INFO Service finalised successfully
 
−
ChronoStatSvc.f...  INFO  Service finalized succesfully
 
−
ToolSvc              INFO Removing all tools created by ToolSvc
 
−
ApplicationMgr      INFO Application Manager Finalized successfully
 
−
ApplicationMgr      INFO Application Manager Terminated successfully
 
 
</pre>
 
</pre>
  
−
==The Next Time You Login==
+
*change user.GustavoOrdonezSanz.00008.AAN.root for your own
−
*The next time you login, just do the following:
 
 
<pre>
 
<pre>
−
cd ${HOME}/scratch0/Workshop
+
dq2_get -rv -d . user.GustavoOrdonezSanz.00008.AAN.root
−
source setup.sh -tag=12.0.2
 
−
cd 12.0.2/PhysicsAnalysis/AnalysisCommon/UserAnalysis/cmt
 
−
source setup.sh
 
−
cd ../run
 
 
</pre>
 
</pre>
 +
 +
==Ganga==
 +
We're not going to cover Ganga in this tutorial. But for reference look at:[https://twiki.cern.ch/twiki/bin/view/Atlas/DistributedAnalysisUsingGanga Ganga link]
 +
 +
==Final Exercice==
 +
To conclude the workshop we're going to try to make our own Higgs to four muon analysis. Taking as an example the ZmumuOnAOD algorithm, make a H4mu one. The purpose of this, is to dig into the Phisics Analysis Tools used in ZmumuOnAOD, and try to have a realistic approach of how is like to make Analyis in Athena.
 +
 +
Good luck!

Latest revision as of 09:08, 25 October 2006

Pre-Tutorial

  • This pre-Tutorial is copy of the one given by Ketevi A. Assamagan, adapted to work in 12.0.2
  • here is a link to other Tutorials by Ketevi, and other usful PAT links:

PAT

  • Please follow the following pre-Tutorial before the date of the Workshop. It is a very straight forward, tutorial and will help to set up the athena enviroment for the workshop, so we can focus our time in more interested things

Pre-Tutorial link

During the workshop we'll browse over the code that we've used in the pre-Tutorial to try and uderstand it. Later we'll focused on the TAG and AAN selection.

Event Selection

First we're going to generate a new AAN that we'll use later for selection. 

cd ~scratch0/Workshop/
source setup.sh -tag=12.0.2,opt
cd 12.0.2/PhysicsAnalysis/AnalysisCommon/UserAnalysis
cp ~ordonez/public/Workshop_files/src/AnalysisSkeleton.cxx src/.
cp ~ordonez/public/Workshop_files/UserAnalysis/AnalysisSkeleton.h UserAnalysis/.
cd cmt
gmake
cd ..
mkdir run2
cd run2
cp ~ordonez/public/Workshop_files/run2/AnalysisSkeleton_jobOptions.py .
cp ~ordonez/public/Workshop_files/data2/PoolFileCatalog.xml .
cp ~ordonez/public/Workshop_files/data2/mc11.004202.ZmumuJimmy.recon.AOD.v11000401.py .
athena AnalysisSkeleton_jobOptions.py | tee athenaOut_AnalysisSkeleton.log

This will generate the file AnalysisSkeleton.aan.root.

Event Selection using TAG

  • First we generate our own TAG.
  • Still in the new run2 directory:
cp ~ordonez/public/Workshop_files/run2/Main_topOptions.py .
  • Edit Main_topOptions.py and uncomment the line
### Create TAG file
doWriteTAG=True

athena Main_topOptions.py | tee athenaOut.log
  • This will generate the file called ZmumuOnAOD_TAG.root
  • Now we are ready to run our algorithms with a TAG selection on the fly. Edit Main_topOptions.py again comment out:
#doWriteTAG=True
  • and uncomment
doRunAODwithSelection=True

and

useTag=True
  • Copy:
cp ~ordonez/public/Workshop_files/run2/ZmumuOnAOD_new_jobOptions.py .
  • And run again. Note that only will run the events that satisfy the query:

EventSelector.Query = "NLooseMuon>0 && NLooseMuon<3 && abs(LooseMuonEta1)<2.5 && abs(LooseMuonEta2)<2.5 && LooseMuonPt1>10000 && LooseMuonPt2>10000" 


athena Main_topOptions.py | tee athenaOut.log
  • As an output you have ZmumuOnAOD_withTAGSelection.aan.root. You can have a look at it in root to make sure that the query has been satisfied
  • Now we'll rewrite our own AOD containeing only the events selected. We'll do it for only 50 events to gain some time
  • Edit again Main_topOptions.py and change:
TheApp.EvtMax = 500

to

TheApp.EvtMax = 50
  • Comment out
#doRunAODwithSelection=True
  • uncomment:
doRewriteAOD=True

and run once more 

athena Main_topOptions.py | tee athenaOut.log

You'll see a new Zmumu_new_AOD_withTAGSelection.root file has been created

Event Selection using AAN

  • Follow as in the previous section but useANN=True instead of useTAG=True. In the end you should have the following two files:

ZmumuOnAOD_withAANSelection.aan.root Zmumu_new_AOD_withAANSelection.root

Interactive Analysis in ATHENA

  • In this exercise, we will see how to do interactive analysis in ATHENA, how to browse the Raw Data in the AOD. Proceed as follows:
get_files Interactive_topO.py
  • Edit Interactive_topO.py and replace this line
EventSelector.InputCollections = [ "AOD.pool.root" ]
  • with this line:
include ( "mc11.004202.ZmumuJimmy.recon.AOD.v11000401.py" )
  • run
athena -i Interactive_topO.py
  • Initialize application manager
athena>theApp.initialize()
  • Run 1 event
athena>theApp.nextEvent()
  • Retrieve Muons from AOD
athena>mcon = PyParticleTools.getMuons("MuidMuonCollection")
  • Get the first muon
athena>m = mcon[0]
  • Get a list of methods
athena>dir(m)
  • See pT
athena>m.pt()
  • Get TrackParticle via ElementLink
athena>tp = m.track()
athena>dir(tp)
athena>tp.pt()
  • Retrieve the MC Truth
athena>mcc = PyTruthTools.getMcEvents("TruthEvent")
athena>mc = mcc[0]
athena>mc.alphaQCD()
  • Dump Data Store
athena>dumpSG()
  • Run next event
athena>theApp.nextEvent()

Rewind event counter 

athena>theApp.seek(0)
  • See pT distribution of muons
athena> plot("MuonContainer#MuidMuonCollection","$x.pt()",nEvent=5)


Analysis with the EventView

  • Due to the time I won't cover this section during the Workshop. But now you should have the basic knowledge requiered to understand it. You can learn about the Event View here

Analysis on the GRID

  • In the following examples we'll run on Zmumu datasets.
  • This will be a brief introduction to GRID tools for those of you who already went through the trouble of getting a GRID certificate.
  • First we should set up DQ2 tools:
source /afs/usatlas.bnl.gov/Grid/Don-Quijote/dq2_user_client/setup.csh.CERN

PANDA/PATHENA

  • For further details visit panda wiki
  • Setup PATHENA :
cd ~/scratch0/Workshop/12.0.2/PhysicsAnalysis/AnalysisCommon/UserAnalysis
cvs update -r UserAnalysis-00-08-13 -A python
cvs update -r UserAnalysis-00-08-13 -A share/pathena
cvs update -r UserAnalysis-00-08-13 -A share/pathena_util
cvs update -r UserAnalysis-00-08-13 -A share/ConfigExtractor.py
cd cmt
source setup.sh
gmake
cd ../run
  • run pathena

Note: change user.GustavoOrdonezSanz.00008.AAN.root with an appropiate name!! 

pathena ZmumuOnAOD_jobOptions.py --inDS 'csc11.005145.PythiaZmumu.recon.AOD.v11004103' --outDS \
'user.GustavoOrdonezSanz.00008.AAN.root'


DQ2 Tools

  • For furhter details visit DQ2 wiki
  • This tools make your life easier when working on the GRID
dq2_ls *Zmumu*AOD*
  • Have a look at the files contained in one of the sets
dq2_ls -f <datasetname>
  • change user.GustavoOrdonezSanz.00008.AAN.root for your own
dq2_get -rv -d . user.GustavoOrdonezSanz.00008.AAN.root

Ganga

We're not going to cover Ganga in this tutorial. But for reference look at:Ganga link

Final Exercice

To conclude the workshop we're going to try to make our own Higgs to four muon analysis. Taking as an example the ZmumuOnAOD algorithm, make a H4mu one. The purpose of this, is to dig into the Phisics Analysis Tools used in ZmumuOnAOD, and try to have a realistic approach of how is like to make Analyis in Athena.

Good luck!

Retrieved from "https://wiki.nikhef.nl/atlas/index.php?title=NIKHEF_PAT_workshop_Oct2006&oldid=4762"