Difference between revisions of "NIKHEF PAT workshop Oct2006"

From Atlas Wiki
Jump to navigation Jump to search
 
(77 intermediate revisions by one other user not shown)
Line 1: Line 1:
 
==Pre-Tutorial==
 
==Pre-Tutorial==
  
*This pre-Tutorial is copy of the one gave by Ketevi A. Assamagan, adapted to work in 12.0.2  
+
*This pre-Tutorial is copy of the one given by Ketevi A. Assamagan, adapted to work in 12.0.2  
  
 
*here is a link to other Tutorials by Ketevi, and other usful PAT links:
 
*here is a link to other Tutorials by Ketevi, and other usful PAT links:
Line 10: Line 10:
 
[http://www.nikhef.nl/pub/experiments/atlaswiki/index.php/NIKHEF_PAT_workshop_Oct2006_pre-Tutorial Pre-Tutorial link]
 
[http://www.nikhef.nl/pub/experiments/atlaswiki/index.php/NIKHEF_PAT_workshop_Oct2006_pre-Tutorial Pre-Tutorial link]
  
==Event Selection with the TAG==
+
During the workshop we'll browse over the code that we've used in the pre-Tutorial to try and uderstand it. Later we'll focused on the TAG and AAN selection.
*In this exercise, we will use the TAG to select the final states of interest. The data that we are contains Z to mumu so there is not must selection final state we can do with the TAG on this data but this exercise will give a flavor of what you can to with the TAG and how to it. Consider the following job options.
+
 
 +
==Event Selection==
 +
First we're going to generate a new AAN that we'll use later for selection.  
 
<pre>
 
<pre>
####################################################
+
cd ~scratch0/Workshop/
#Author - Ketevi A. Assamagan
+
source setup.sh -tag=12.0.2,opt
AllAlgs = False
+
cd 12.0.2/PhysicsAnalysis/AnalysisCommon/UserAnalysis
doHist = False
+
cp ~ordonez/public/Workshop_files/src/AnalysisSkeleton.cxx src/.
doCBNT = False
+
cp ~ordonez/public/Workshop_files/UserAnalysis/AnalysisSkeleton.h UserAnalysis/.
doWriteTAG = False
+
cd cmt
doWriteESD = False
+
gmake
readAOD = True
+
cd ..
doAOD = False
+
mkdir run2
doWriteAOD = True
+
cd run2
# read the TAG as input to the job
+
cp ~ordonez/public/Workshop_files/run2/AnalysisSkeleton_jobOptions.py .
readColl = True
+
cp ~ordonez/public/Workshop_files/data2/PoolFileCatalog.xml .
 +
cp ~ordonez/public/Workshop_files/data2/mc11.004202.ZmumuJimmy.recon.AOD.v11000401.py .
 +
athena AnalysisSkeleton_jobOptions.py | tee athenaOut_AnalysisSkeleton.log
 +
</pre>
  
# Number of Events
+
This will generate the file AnalysisSkeleton.aan.root.
EvtMax = 20
 
  
# Use the TAG that you just produce to select the events
+
==Event Selection using TAG==
# define also the selection criteria
+
*First we generate our own TAG.
include ( "mc11.004202.ZmumuJimmy.recon.TAG.v11000401.py" )
 
CollInputQuery="NLooseMuon>0 && NLooseMuon<3 && abs(LooseMuonEta1)<2.5 && abs(LooseMuonEta2)<2.5 && LooseMuonPt2>10000 && LooseMuonPt2>10000"
 
  
#Create a new RDO, ESD and AOD that contain only selected events
+
*Still in the new run2 directory:
PoolAODOutput = "mc11.004202.ZmumuJimmy.recon.AOD.TagSel.v11000401.root"
 
  
# main reconstruction job jobOptions
+
<pre>
include ("RecExCommon/RecExCommon_topOptions.py")
+
cp ~ordonez/public/Workshop_files/run2/Main_topOptions.py .
#######################################################
 
 
</pre>
 
</pre>
*Create a file called TagBasedEventSelection_topOptions.py and copy the above options into it and save it. We are using the TAG file as input and selecting the events at the AOD level. Our selection criteria are defined in the CollInputQuery. We are querying for the events having:
+
 
 +
*Edit Main_topOptions.py and uncomment the line
 
<pre>
 
<pre>
Two one or 2 loose muons
+
### Create TAG file
Eta of each loose muons within 2.5
+
doWriteTAG=True
Pt of 2 loose muons above 10 GeV - unit in MeV
+
</pre>
 +
 +
<pre>
 +
athena Main_topOptions.py | tee athenaOut.log
 
</pre>
 
</pre>
  
*For the all the events that pass the selection, we create an AOD file that contains ONLY the selected events: for details on how to use the TAG and which TAG content or attributes you can make selection on, follow this link: TagForEventSelection. The new AOD file name should be mc11.004202.ZmumuJimmy.recon.AOD.TagSel.v11000401.root as specified in the above job options. To select events at the ESD level, one would simply add this lines:
+
*This will generate the file called ZmumuOnAOD_TAG.root
  
 +
*Now we are ready to run our algorithms with a TAG selection on the fly. Edit Main_topOptions.py again comment out:
 
<pre>
 
<pre>
readESD = True
+
#doWriteTAG=True  
readAOD = False
 
 
</pre>
 
</pre>
  
*Now run this job options and see that you have a new AOD file that contains only the selected events. An easy and a quick way to see the TAG attributes is to just open the TAG file in ROOT and browse. Copy a sample TAG file from my area and open it in ROOT:
+
*and uncomment
 
<pre>
 
<pre>
cp ~ordonez/scratch0/Workshop/12.0.2/PhysicsAnalysis/AnalysisCommon/UserAnalysis/run/TAG.fromAOD.pool.root .
+
doRunAODwithSelection=True
root TAG.fromAOD.pool.root
+
 
new TBrowser
+
and
 +
 
 +
useTag=True
 
</pre>
 
</pre>
*Now you can do analysis on the new AOD that contains ONLY the selected events.
 
  
==Event Selection with the AthenaAwareNTuple==
+
*Copy:
*In this exercise, we will use the AthenaAwareNTuple that you produced in the introdcution: you should have an NTuple called AnalysisSkeleton.aan.root. Proceed as follows:
 
 
<pre>
 
<pre>
cp AnalysisSkeleton_jobOptions.py AanSelectionAnalysis_topOptions.py
+
cp ~ordonez/public/Workshop_files/run2/ZmumuOnAOD_new_jobOptions.py .
 
</pre>
 
</pre>
*Edit AanSelectionAnalysis_topOptions.py and replace this line
+
*And run again. Note that only will run the events that satisfy the query:
 +
EventSelector.Query = "NLooseMuon>0 && NLooseMuon<3 && abs(LooseMuonEta1)<2.5 && abs(LooseMuonEta2)<2.5 && LooseMuonPt1>10000 && LooseMuonPt2>10000"
 +
 
 +
 
 
<pre>
 
<pre>
include( "mc11.004201.ZeeJimmy.recon.AOD.v11000401.py" )
+
athena Main_topOptions.py | tee athenaOut.log
 
</pre>
 
</pre>
  
*with these lines
+
*As an output you have ZmumuOnAOD_withTAGSelection.aan.root. You can have a look at it in root to make sure that the query has been satisfied
 +
 
 +
*Now we'll rewrite our own AOD containeing only the events selected. We'll do it for only 50 events to gain some time
 +
 
 +
*Edit again Main_topOptions.py and change:
 +
 
 
<pre>
 
<pre>
EventSelector.InputCollections = [ "AnalysisSkeleton.aan" ]
+
TheApp.EvtMax = 500
EventSelector.Query="NElectrons>0 && NElectrons<3 && abs(ElectronEta[0])<2.5 && abs(ElectronEta[1])<2.5 && ElectronPt[0]>10000 && ElectronPt[1]>10000"
+
 
EventSelector.CollectionType = "ExplicitROOT"
+
to
 +
 
 +
TheApp.EvtMax = 50
 
</pre>
 
</pre>
  
*Here, we are using the AthenaAwareNTuple called AnalysisSkeleton.aan.root to selected the event at the AOD. Then, we are run the Analysis code, AnalysisSkeleton.cxx on the selected events. Note that when you use the TAG or the AthenaAwareNTuple to event selection, the file extension .root must not be specified: it is assumed and will be appended. In these exercises, we are using the ROOT version of the TAG --- also, our AthenaAwareNTuple is a ROOT file. This is why we specify the CollectionType to be ExplicitROOT. There is also a database version of the TAG: for details see TagForEventSelection. In AanSelectionAnalysis_topOptions.py, replace these lines
+
*Comment out
 
<pre>
 
<pre>
THistSvc.Output = ["AANT DATAFILE='AnalysisSkeleton.aan.root' OPT='RECREATE'"]
+
#doRunAODwithSelection=True
AANTupleStream.OutputName = 'AnalysisSkeleton.aan.root'
 
 
</pre>
 
</pre>
  
*with these lines:
+
*uncomment:
 
<pre>
 
<pre>
THistSvc.Output = ["AANT DATAFILE='AnalysisSkeleton.sel.aan.root' OPT='RECREATE'"]
+
doRewriteAOD=True
AANTupleStream.OutputName = 'AnalysisSkeleton.sel.aan.root'
 
 
</pre>
 
</pre>
*Now run:
+
 
 +
and run once more
 +
 
 
<pre>
 
<pre>
athena.py AanSelectionAnalysis_topOptions.py
+
athena Main_topOptions.py | tee athenaOut.log
 
</pre>
 
</pre>
*You should have a new AthenaAwareNTuple file called AnalysisSkeleton.sel.aan.root. That we've just done is the following: we use a previously available AthenaAwareNTuple or TAG to select events and we did analysis on the selected events creating a new AthenaAwareNTuple. We also saw that it is possible to create a new AOD that contains ONLY the selected events. Now check that the selection criteria that we specified above were indeed apply: open AnalysisSkeleton.sel.aan.root and plot the EventNumber and the NElectrons (this should be one or two) according to our selection criteria.
+
 
 +
You'll see a new Zmumu_new_AOD_withTAGSelection.root file has been created
 +
 
 +
==Event Selection using AAN==
 +
*Follow as in the previous section but useANN=True instead of useTAG=True. In the end you should have the following two files:
 +
ZmumuOnAOD_withAANSelection.aan.root
 +
Zmumu_new_AOD_withAANSelection.root
  
 
==Interactive Analysis in ATHENA==
 
==Interactive Analysis in ATHENA==
Line 109: Line 129:
 
*with this line:
 
*with this line:
 
<pre>
 
<pre>
include( "mc11.004100.T1_McAtNLO_top.recon.AOD.v11000401.py" )
+
include ( "mc11.004202.ZmumuJimmy.recon.AOD.v11000401.py" )
 +
</pre>
 +
 
 +
*run
 +
<pre>
 +
athena -i Interactive_topO.py
 +
</pre>
 +
*Initialize application manager
 +
<pre>
 +
athena>theApp.initialize()
 +
</pre>
 +
*Run 1 event
 +
<pre>
 +
athena>theApp.nextEvent()
 +
</pre>
 +
 
 +
*Retrieve Muons from AOD
 +
<pre>
 +
athena>mcon = PyParticleTools.getMuons("MuidMuonCollection")
 +
</pre>
 +
 
 +
*Other methods are defined in [http://tmaeno.home.cern.ch/tmaeno/PyParticleTools/index.html PyParticleTools]
 +
 
 +
*Get the first muon
 +
<pre>
 +
athena>m = mcon[0]
 +
</pre>
 +
 
 +
*Get a list of methods
 +
<pre>
 +
athena>dir(m)
 
</pre>
 
</pre>
*Then start the interactive analysis session from this link.
 
  
 +
*See pT
 +
<pre>
 +
athena>m.pt()
 +
</pre>
  
 +
*Get TrackParticle via ElementLink
 +
<pre>
 +
athena>tp = m.track()
 +
athena>dir(tp)
 +
athena>tp.pt()
 +
</pre>
 +
 +
*Retrieve the MC Truth
 +
<pre>
 +
athena>mcc = PyTruthTools.getMcEvents("TruthEvent")
 +
athena>mc = mcc[0]
 +
athena>mc.alphaQCD()
 +
</pre>
 +
 +
*Dump Data Store
 +
<pre>
 +
athena>dumpSG()
 +
</pre>
  
==Monte Carlo Truth Tools on AOD==
+
*Run next event
*In this exercise, we will look MC Truth Tools on AOD. Follow the link below, but skip the configuration part since it pertains to the release 11.0.5 while we are using the release 12.0.1 here and we have already done the configuration for 12.0.1 in the Introduction above. You will also need to make the following changes for 12.0.1: everywhere where application, replace the following
 
 
<pre>
 
<pre>
#include "TruthParticleAlgs/McVtxFilterTool.h"
+
athena>theApp.nextEvent()
#include "TruthParticleAlgs/TruthParticleCnvTool.h"
 
#include "AnalysisUtils/McVtxFilter.h"
 
#include "ParticleEvent/TruthParticle.h"
 
#include "ParticleEvent/TruthParticleContainer.h"
 
 
</pre>
 
</pre>
*with the following:
+
 
 +
Rewind event counter
 
<pre>
 
<pre>
#include "McParticleTools/McVtxFilterTool.h"
+
athena>theApp.seek(0)
#include "McParticleTools/TruthParticleCnvTool.h"
 
#include "McParticleUtils/McVtxFilter.h"
 
#include "McParticleEvent/TruthParticle.h"
 
#include "McParticleEvent/TruthParticleContainer.h"
 
 
</pre>
 
</pre>
*Also in your requirements file, you may need to add the following lines and redo cmt config and gmake:
+
 
 +
*See pT distribution of muons
 
<pre>
 
<pre>
cmt co -r McParticleTools    McParticleTools-00-*      PhysicsAnalysis/TruthParticleID
+
athena> plot("MuonContainer#MuidMuonCollection","$x.pt()",nEvent=5)
cmt co -r McParticleUtils      McParticleUtils-00-*      PhysicsAnalysis/TruthParticleID
 
cmt co -r McParticleEvent    McParticleEvent-00-*    PhysicsAnalysis/TruthParticleID
 
 
</pre>
 
</pre>
  
*Now follow this link, McTrtuh on AOD, make sure that you make the above modifications before running.
+
 
  
 
==Analysis with the EventView==
 
==Analysis with the EventView==
*Follow this link: EventView. However, I would suggest that you do the specific exercises above on ESD/AOD/TAG first: that provides the background for understanding the ESD/AOD/TAG before proceeding to the EventView.
+
*Due to the time I won't cover this section during the Workshop. But now you should have the basic knowledge requiered to understand it. You can learn about the [https://twiki.cern.ch/twiki/bin/view/Atlas/EventView Event View] here
 +
 
 +
== Analysis on the GRID ==
 +
* In the following examples we'll run on Zmumu datasets.
 +
*This will be a brief introduction to GRID tools for those of you who already went through the trouble of getting a GRID certificate.
 +
 
 +
*First we should set up DQ2 tools:
 +
 
 +
<pre>
 +
source /afs/usatlas.bnl.gov/Grid/Don-Quijote/dq2_user_client/setup.csh.CERN
 +
</pre>
 +
 
 +
==PANDA/PATHENA==
 +
*For further details visit [https://twiki.cern.ch/twiki/bin/view/Atlas/DAonPanda panda wiki]
 +
*Setup PATHENA :
 +
<pre>
 +
cd ~/scratch0/Workshop/12.0.2/PhysicsAnalysis/AnalysisCommon/UserAnalysis
 +
cvs update -r UserAnalysis-00-08-13 -A python
 +
cvs update -r UserAnalysis-00-08-13 -A share/pathena
 +
cvs update -r UserAnalysis-00-08-13 -A share/pathena_util
 +
cvs update -r UserAnalysis-00-08-13 -A share/ConfigExtractor.py
 +
cd cmt
 +
source setup.sh
 +
gmake
 +
cd ../run
 +
</pre>
 +
 
 +
*run pathena
 +
Note: '''change user.GustavoOrdonezSanz.00008.AAN.root with an appropiate name!!'''
 +
<pre>
 +
pathena ZmumuOnAOD_jobOptions.py --inDS 'csc11.005145.PythiaZmumu.recon.AOD.v11004103' --outDS \
 +
'user.GustavoOrdonezSanz.00008.AAN.root'
 +
</pre>
 +
 
 +
*For job status monitoring visit: [http://gridui02.usatlas.bnl.gov:25880/server/pandamon/query Monitoring]
 +
 
 +
 
 +
 
 +
==DQ2 Tools==
 +
*For furhter details visit [https://twiki.cern.ch/twiki/bin/view/Atlas/UsingDQ2 DQ2 wiki]
 +
*This tools make your life easier when working on the GRID
 +
<pre>
 +
dq2_ls *Zmumu*AOD*
 +
</pre>
 +
 
 +
*Have a look at the files contained in one of the sets
 +
<pre>
 +
dq2_ls -f <datasetname>
 +
</pre>
 +
 
 +
*change user.GustavoOrdonezSanz.00008.AAN.root for your own
 +
<pre>
 +
dq2_get -rv -d . user.GustavoOrdonezSanz.00008.AAN.root
 +
</pre>
  
==Analysis on the Grid==
+
==Ganga==
*This will be a brief introduction to GRID tools for those of you who already went through the trouble of getting a GRID certificate
+
We're not going to cover Ganga in this tutorial. But for reference look at:[https://twiki.cern.ch/twiki/bin/view/Atlas/DistributedAnalysisUsingGanga Ganga link]
  
*PANDA/PATHENA
+
==Final Exercice==
 +
To conclude the workshop we're going to try to make our own Higgs to four muon analysis. Taking as an example the ZmumuOnAOD algorithm, make a H4mu one. The purpose of this, is to dig into the Phisics Analysis Tools used in ZmumuOnAOD, and try to have a realistic approach of how is like to make Analyis in Athena.
  
*GANGA
+
Good luck!

Latest revision as of 09:08, 25 October 2006

Pre-Tutorial

  • This pre-Tutorial is copy of the one given by Ketevi A. Assamagan, adapted to work in 12.0.2
  • here is a link to other Tutorials by Ketevi, and other usful PAT links:

PAT

  • Please follow the following pre-Tutorial before the date of the Workshop. It is a very straight forward, tutorial and will help to set up the athena enviroment for the workshop, so we can focus our time in more interested things

Pre-Tutorial link

During the workshop we'll browse over the code that we've used in the pre-Tutorial to try and uderstand it. Later we'll focused on the TAG and AAN selection.

Event Selection

First we're going to generate a new AAN that we'll use later for selection.

cd ~scratch0/Workshop/
source setup.sh -tag=12.0.2,opt
cd 12.0.2/PhysicsAnalysis/AnalysisCommon/UserAnalysis
cp ~ordonez/public/Workshop_files/src/AnalysisSkeleton.cxx src/.
cp ~ordonez/public/Workshop_files/UserAnalysis/AnalysisSkeleton.h UserAnalysis/.
cd cmt
gmake
cd ..
mkdir run2
cd run2
cp ~ordonez/public/Workshop_files/run2/AnalysisSkeleton_jobOptions.py .
cp ~ordonez/public/Workshop_files/data2/PoolFileCatalog.xml .
cp ~ordonez/public/Workshop_files/data2/mc11.004202.ZmumuJimmy.recon.AOD.v11000401.py .
athena AnalysisSkeleton_jobOptions.py | tee athenaOut_AnalysisSkeleton.log

This will generate the file AnalysisSkeleton.aan.root.

Event Selection using TAG

  • First we generate our own TAG.
  • Still in the new run2 directory:
cp ~ordonez/public/Workshop_files/run2/Main_topOptions.py .
  • Edit Main_topOptions.py and uncomment the line
### Create TAG file
doWriteTAG=True
athena Main_topOptions.py | tee athenaOut.log
  • This will generate the file called ZmumuOnAOD_TAG.root
  • Now we are ready to run our algorithms with a TAG selection on the fly. Edit Main_topOptions.py again comment out:
#doWriteTAG=True 
  • and uncomment
doRunAODwithSelection=True

and

useTag=True
  • Copy:
cp ~ordonez/public/Workshop_files/run2/ZmumuOnAOD_new_jobOptions.py .
  • And run again. Note that only will run the events that satisfy the query:

EventSelector.Query = "NLooseMuon>0 && NLooseMuon<3 && abs(LooseMuonEta1)<2.5 && abs(LooseMuonEta2)<2.5 && LooseMuonPt1>10000 && LooseMuonPt2>10000"


athena Main_topOptions.py | tee athenaOut.log
  • As an output you have ZmumuOnAOD_withTAGSelection.aan.root. You can have a look at it in root to make sure that the query has been satisfied
  • Now we'll rewrite our own AOD containeing only the events selected. We'll do it for only 50 events to gain some time
  • Edit again Main_topOptions.py and change:
TheApp.EvtMax = 500

to

TheApp.EvtMax = 50
  • Comment out
#doRunAODwithSelection=True
  • uncomment:
doRewriteAOD=True

and run once more

athena Main_topOptions.py | tee athenaOut.log

You'll see a new Zmumu_new_AOD_withTAGSelection.root file has been created

Event Selection using AAN

  • Follow as in the previous section but useANN=True instead of useTAG=True. In the end you should have the following two files:

ZmumuOnAOD_withAANSelection.aan.root Zmumu_new_AOD_withAANSelection.root

Interactive Analysis in ATHENA

  • In this exercise, we will see how to do interactive analysis in ATHENA, how to browse the Raw Data in the AOD. Proceed as follows:
get_files Interactive_topO.py
  • Edit Interactive_topO.py and replace this line
EventSelector.InputCollections = [ "AOD.pool.root" ]
  • with this line:
include ( "mc11.004202.ZmumuJimmy.recon.AOD.v11000401.py" )
  • run
athena -i Interactive_topO.py
  • Initialize application manager
athena>theApp.initialize()
  • Run 1 event
athena>theApp.nextEvent()
  • Retrieve Muons from AOD
athena>mcon = PyParticleTools.getMuons("MuidMuonCollection")
  • Get the first muon
athena>m = mcon[0]
  • Get a list of methods
athena>dir(m)
  • See pT
athena>m.pt()
  • Get TrackParticle via ElementLink
athena>tp = m.track()
athena>dir(tp)
athena>tp.pt()
  • Retrieve the MC Truth
athena>mcc = PyTruthTools.getMcEvents("TruthEvent")
athena>mc = mcc[0]
athena>mc.alphaQCD()
  • Dump Data Store
athena>dumpSG()
  • Run next event
athena>theApp.nextEvent()

Rewind event counter

athena>theApp.seek(0)
  • See pT distribution of muons
athena> plot("MuonContainer#MuidMuonCollection","$x.pt()",nEvent=5)


Analysis with the EventView

  • Due to the time I won't cover this section during the Workshop. But now you should have the basic knowledge requiered to understand it. You can learn about the Event View here

Analysis on the GRID

  • In the following examples we'll run on Zmumu datasets.
  • This will be a brief introduction to GRID tools for those of you who already went through the trouble of getting a GRID certificate.
  • First we should set up DQ2 tools:
source /afs/usatlas.bnl.gov/Grid/Don-Quijote/dq2_user_client/setup.csh.CERN

PANDA/PATHENA

  • For further details visit panda wiki
  • Setup PATHENA :
cd ~/scratch0/Workshop/12.0.2/PhysicsAnalysis/AnalysisCommon/UserAnalysis
cvs update -r UserAnalysis-00-08-13 -A python
cvs update -r UserAnalysis-00-08-13 -A share/pathena
cvs update -r UserAnalysis-00-08-13 -A share/pathena_util
cvs update -r UserAnalysis-00-08-13 -A share/ConfigExtractor.py
cd cmt
source setup.sh
gmake
cd ../run
  • run pathena

Note: change user.GustavoOrdonezSanz.00008.AAN.root with an appropiate name!!

pathena ZmumuOnAOD_jobOptions.py --inDS 'csc11.005145.PythiaZmumu.recon.AOD.v11004103' --outDS \
'user.GustavoOrdonezSanz.00008.AAN.root'


DQ2 Tools

  • For furhter details visit DQ2 wiki
  • This tools make your life easier when working on the GRID
dq2_ls *Zmumu*AOD*
  • Have a look at the files contained in one of the sets
dq2_ls -f <datasetname>
  • change user.GustavoOrdonezSanz.00008.AAN.root for your own
dq2_get -rv -d . user.GustavoOrdonezSanz.00008.AAN.root

Ganga

We're not going to cover Ganga in this tutorial. But for reference look at:Ganga link

Final Exercice

To conclude the workshop we're going to try to make our own Higgs to four muon analysis. Taking as an example the ZmumuOnAOD algorithm, make a H4mu one. The purpose of this, is to dig into the Phisics Analysis Tools used in ZmumuOnAOD, and try to have a realistic approach of how is like to make Analyis in Athena.

Good luck!