Difference between revisions of "NIKHEF PAT workshop Oct2006"

From Atlas Wiki
Jump to navigation Jump to search
 
(123 intermediate revisions by one other user not shown)
Line 1: Line 1:
 +
==Pre-Tutorial==
  
−
==Set up CMT==
+
*This pre-Tutorial is copy of the one given by Ketevi A. Assamagan, adapted to work in 12.0.2
  
−
* Login to lxplus at CERN and create a working directory called "Tutorial". We will use the release 12.0.2 this tutorial:
+
*here is a link to other Tutorials by Ketevi, and other usful PAT links:
 +
[https://twiki.cern.ch/twiki/bin/view/Atlas/PhysicsAnalysisTools, PAT]
 +
 
 +
*Please follow the following pre-Tutorial before the date of the Workshop. It is a very straight forward, tutorial and will help to set up the athena enviroment for the workshop, so we can focus our time in more interested things
 +
 
 +
[http://www.nikhef.nl/pub/experiments/atlaswiki/index.php/NIKHEF_PAT_workshop_Oct2006_pre-Tutorial Pre-Tutorial link]
 +
 
 +
During the workshop we'll browse over the code that we've used in the pre-Tutorial to try and uderstand it. Later we'll focused on the TAG and AAN selection.
 +
 
 +
==Event Selection==
 +
First we're going to generate a new AAN that we'll use later for selection.  
 
<pre>
 
<pre>
−
ssh lxplus.cern.ch
+
cd ~scratch0/Workshop/
−
cd scratch0
+
source setup.sh -tag=12.0.2,opt
−
mkdir Tutorial
+
cd 12.0.2/PhysicsAnalysis/AnalysisCommon/UserAnalysis
−
cd Tutorial
+
cp ~ordonez/public/Workshop_files/src/AnalysisSkeleton.cxx src/.
−
mkdir 12.0.2
+
cp ~ordonez/public/Workshop_files/UserAnalysis/AnalysisSkeleton.h UserAnalysis/.
 +
cd cmt
 +
gmake
 +
cd ..
 +
mkdir run2
 +
cd run2
 +
cp ~ordonez/public/Workshop_files/run2/AnalysisSkeleton_jobOptions.py .
 +
cp ~ordonez/public/Workshop_files/data2/PoolFileCatalog.xml .
 +
cp ~ordonez/public/Workshop_files/data2/mc11.004202.ZmumuJimmy.recon.AOD.v11000401.py .
 +
athena AnalysisSkeleton_jobOptions.py | tee athenaOut_AnalysisSkeleton.log
 
</pre>
 
</pre>
  
−
* Create a file called "requirements" and setup CMT. My requirements file looks like this where my working area for 12.0.2 is ${HOME}/scratch0/Workshop/12.0.2:
+
This will generate the file AnalysisSkeleton.aan.root.
 +
 
 +
==Event Selection using TAG==
 +
*First we generate our own TAG.
 +
 
 +
*Still in the new run2 directory:
 +
 
 
<pre>
 
<pre>
−
set  CMTSITE  CERN
+
cp ~ordonez/public/Workshop_files/run2/Main_topOptions.py .
−
set  SITEROOT /afs/cern.ch
+
</pre>
−
macro ATLAS_DIST_AREA ${SITEROOT}/atlas/software/dist
+
 
−
                     
+
*Edit Main_topOptions.py and uncomment the line
−
macro ATLAS_GROUP_AREA "/afs/cern.ch/atlas/groups/PAT/Tutorial"
+
<pre>
 +
### Create TAG file
 +
doWriteTAG=True
 +
</pre>
 
   
 
   
−
macro ATLAS_TEST_AREA "" \
+
<pre>
−
        12.0.2 "${HOME}/scratch0/Workshop/12.0.2"
+
athena Main_topOptions.py | tee athenaOut.log
−
+
</pre>
−
+
 
−
apply_tag oneTest
+
*This will generate the file called ZmumuOnAOD_TAG.root
−
+
 
−
use AtlasLogin AtlasLogin-* $(ATLAS_DIST_AREA)
+
*Now we are ready to run our algorithms with a TAG selection on the fly. Edit Main_topOptions.py again comment out:
 +
<pre>
 +
#doWriteTAG=True
 +
</pre>
 +
 
 +
*and uncomment
 +
<pre>
 +
doRunAODwithSelection=True
 +
 
 +
and
 +
 
 +
useTag=True
 +
</pre>
 +
 
 +
*Copy:
 +
<pre>
 +
cp ~ordonez/public/Workshop_files/run2/ZmumuOnAOD_new_jobOptions.py .
 +
</pre>
 +
*And run again. Note that only will run the events that satisfy the query:
 +
EventSelector.Query = "NLooseMuon>0 && NLooseMuon<3 && abs(LooseMuonEta1)<2.5 && abs(LooseMuonEta2)<2.5 && LooseMuonPt1>10000 && LooseMuonPt2>10000"
 +
 
 +
 
 +
<pre>
 +
athena Main_topOptions.py | tee athenaOut.log
 +
</pre>
 +
 
 +
*As an output you have ZmumuOnAOD_withTAGSelection.aan.root. You can have a look at it in root to make sure that the query has been satisfied
 +
 
 +
*Now we'll rewrite our own AOD containeing only the events selected. We'll do it for only 50 events to gain some time
 +
 
 +
*Edit again Main_topOptions.py and change:
 +
 
 +
<pre>
 +
TheApp.EvtMax = 500
 +
 
 +
to
 +
 
 +
TheApp.EvtMax = 50
 +
</pre>
 +
 
 +
*Comment out
 +
<pre>
 +
#doRunAODwithSelection=True
 +
</pre>
 +
 
 +
*uncomment:
 +
<pre>
 +
doRewriteAOD=True
 +
</pre>
 +
 
 +
and run once more
 +
 
 +
<pre>
 +
athena Main_topOptions.py | tee athenaOut.log
 +
</pre>
 +
 
 +
You'll see a new Zmumu_new_AOD_withTAGSelection.root file has been created
 +
 
 +
==Event Selection using AAN==
 +
*Follow as in the previous section but useANN=True instead of useTAG=True. In the end you should have the following two files:
 +
ZmumuOnAOD_withAANSelection.aan.root
 +
Zmumu_new_AOD_withAANSelection.root
 +
 
 +
==Interactive Analysis in ATHENA==
 +
*In this exercise, we will see how to do interactive analysis in ATHENA, how to browse the Raw Data in the AOD. Proceed as follows:
 +
<pre>
 +
get_files Interactive_topO.py
 
</pre>
 
</pre>
  
−
*Then do the following:
+
*Edit Interactive_topO.py and replace this line
 
<pre>
 
<pre>
−
source /afs/cern.ch/sw/contrib/CMT/v1r18p20060301/mgr/setup.sh
+
EventSelector.InputCollections = [ "AOD.pool.root" ]
−
cmt config
+
</pre>
 +
*with this line:
 +
<pre>
 +
include ( "mc11.004202.ZmumuJimmy.recon.AOD.v11000401.py" )
 
</pre>
 
</pre>
  
−
==Setup for the release 12.0.2==
+
*run
−
*Do the following:
+
<pre>
 +
athena -i Interactive_topO.py
 +
</pre>
 +
*Initialize application manager
 +
<pre>
 +
athena>theApp.initialize()
 +
</pre>
 +
*Run 1 event
 
<pre>
 
<pre>
−
source setup.sh -tag=12.0.2
+
athena>theApp.nextEvent()
 
</pre>
 
</pre>
  
−
*Check that the CMT path is correct by doing this
+
*Retrieve Muons from AOD
 
<pre>
 
<pre>
−
echo $CMTPATH
+
athena>mcon = PyParticleTools.getMuons("MuidMuonCollection")
 
</pre>
 
</pre>
  
−
*Stuff like this should be printed to your screen:
+
*Other methods are defined in [http://tmaeno.home.cern.ch/tmaeno/PyParticleTools/index.html PyParticleTools]
 +
 
 +
*Get the first muon
 
<pre>
 
<pre>
−
/afs/cern.ch/user/o/ordonez/scratch0/Workshop/12.0.2:
+
athena>m = mcon[0]
−
/afs/cern.ch/atlas/software/builds/AtlasOffline/12.0.2
 
 
</pre>
 
</pre>
  
−
*You see that the path to your working directory, the path to the release directory and the external are set correctly - do not proceed unless this is done correctly.
+
*Get a list of methods
−
==Run Time Setup==
 
−
*Go to your working area and check out the follow package:
 
 
<pre>
 
<pre>
−
cd 12.0.2
+
athena>dir(m)
−
cmt co -r UserAnalysis-00-08-05 PhysicsAnalysis/AnalysisCommon/UserAnalysis
 
 
</pre>
 
</pre>
  
−
*Now compile everything as follows:
+
*See pT
 
<pre>
 
<pre>
−
cd PhysicsAnalysis/AnalysisCommon/UserAnalysis/cmt
+
athena>m.pt()
−
cmt config
 
−
source setup.sh
 
−
cmt broadcast gmake
 
−
cd ../run
 
 
</pre>
 
</pre>
  
−
*To test that everything has been done correctly, do:
+
*Get TrackParticle via ElementLink
 
<pre>
 
<pre>
−
get_files HelloWorldOptions.py
+
athena>tp = m.track()
−
athena.py -b HelloWorldOptions.py
+
athena>dir(tp)
 +
athena>tp.pt()
 
</pre>
 
</pre>
  
−
*Stuff like this should be printed to the screen:
+
*Retrieve the MC Truth
 
<pre>
 
<pre>
−
HelloWorld          INFO execute()
+
athena>mcc = PyTruthTools.getMcEvents("TruthEvent")
−
HelloWorld          INFO An INFO message
+
athena>mc = mcc[0]
−
HelloWorld          WARNING A WARNING message
+
athena>mc.alphaQCD()
−
HelloWorld          ERROR An ERROR message
 
−
HelloWorld          FATAL A FATAL error message
 
−
AthenaEventLoopMgr  INFO  ===>>> end of event 9    <<<===
 
−
HistorySvc          INFO Service finalised successfully
 
−
ChronoStatSvc.f...  INFO  Service finalized succesfully
 
−
ToolSvc              INFO Removing all tools created by ToolSvc
 
−
ApplicationMgr      INFO Application Manager Finalized successfully
 
−
ApplicationMgr      INFO Application Manager Terminated successfully
 
 
</pre>
 
</pre>
  
−
==The Next Time You Login==
+
*Dump Data Store
−
*The next time you login, just do the following:
 
 
<pre>
 
<pre>
−
cd ${HOME}/scratch0/Workshop
+
athena>dumpSG()
−
source setup.sh -tag=12.0.2
 
−
cd 12.0.2/PhysicsAnalysis/AnalysisCommon/UserAnalysis/cmt
 
−
source setup.sh
 
−
cd ../run
 
 
</pre>
 
</pre>
  
−
==Get The Necessary Files==
+
*Run next event
−
*We will use inclusive Z to mumu events (sample 4202), digitized in 11.0.41 but the ESD and AOD produced in the release 12.0.1. Copy the PoolFileCatalog.xml for the AOD, ESD and Raw Data (RDO) into your UserAnalysis "run" directory:
 
 
<pre>
 
<pre>
−
cp ~ordonez/scratch0/Workshop/12.0.2/data/PoolFileCatalog.xml .
+
athena>theApp.nextEvent()
−
cp ~ordonez/scratch0/Workshop/12.0.2/data/*.py .
 
 
</pre>
 
</pre>
  
−
==Run the AnalysisSkeleton for 500 Events==
+
Rewind event counter
−
*get the necessary job options file from the release:
 
 
<pre>
 
<pre>
−
get_files AnalysisSkeleton_jobOptions.py
+
athena>theApp.seek(0)
 
</pre>
 
</pre>
  
−
*Edit AnalysisSkeleton_jobOptions.py and specify the input data to be the AOD files that you just copied in the previous step: to do this, replace these lines
+
*See pT distribution of muons
 
<pre>
 
<pre>
−
EventSelector.InputCollections = [
+
athena> plot("MuonContainer#MuidMuonCollection","$x.pt()",nEvent=5)
−
  "AOD.pool.root"
+
</pre>
−
                                ]
+
 
  
 +
 +
==Analysis with the EventView==
 +
*Due to the time I won't cover this section during the Workshop. But now you should have the basic knowledge requiered to understand it. You can learn about the [https://twiki.cern.ch/twiki/bin/view/Atlas/EventView Event View] here
 +
 +
== Analysis on the GRID ==
 +
* In the following examples we'll run on Zmumu datasets.
 +
*This will be a brief introduction to GRID tools for those of you who already went through the trouble of getting a GRID certificate.
 +
 +
*First we should set up DQ2 tools:
 +
 +
<pre>
 +
source /afs/usatlas.bnl.gov/Grid/Don-Quijote/dq2_user_client/setup.csh.CERN
 
</pre>
 
</pre>
−
*with this line
+
 
 +
==PANDA/PATHENA==
 +
*For further details visit [https://twiki.cern.ch/twiki/bin/view/Atlas/DAonPanda panda wiki]
 +
*Setup PATHENA :
 
<pre>
 
<pre>
−
include( "mc11.004201.ZeeJimmy.recon.AOD.v11000401.py" )
+
cd ~/scratch0/Workshop/12.0.2/PhysicsAnalysis/AnalysisCommon/UserAnalysis
 +
cvs update -r UserAnalysis-00-08-13 -A python
 +
cvs update -r UserAnalysis-00-08-13 -A share/pathena
 +
cvs update -r UserAnalysis-00-08-13 -A share/pathena_util
 +
cvs update -r UserAnalysis-00-08-13 -A share/ConfigExtractor.py
 +
cd cmt
 +
source setup.sh
 +
gmake
 +
cd ../run
 
</pre>
 
</pre>
−
*Check that in AnalysisSkeleton_jobOptions.py, the number of events to run is set to 500:
+
 
 +
*run pathena
 +
Note: '''change user.GustavoOrdonezSanz.00008.AAN.root with an appropiate name!!'''
 
<pre>
 
<pre>
−
# Number of Events to process
+
pathena ZmumuOnAOD_jobOptions.py --inDS 'csc11.005145.PythiaZmumu.recon.AOD.v11004103' --outDS \
−
theApp.EvtMax = 500
+
'user.GustavoOrdonezSanz.00008.AAN.root'
 
</pre>
 
</pre>
−
*Now run the AnalysisSkeleton_jobOptions.py. It should produce a file called AnalysisSkeleton.aan.root which contains some ROOT histograms and an AthenaAwareNTuple:
+
 
 +
*For job status monitoring visit: [http://gridui02.usatlas.bnl.gov:25880/server/pandamon/query Monitoring]
 +
 
 +
 
 +
 
 +
==DQ2 Tools==
 +
*For furhter details visit [https://twiki.cern.ch/twiki/bin/view/Atlas/UsingDQ2 DQ2 wiki]
 +
*This tools make your life easier when working on the GRID
 
<pre>
 
<pre>
−
athena.py AnalysisSkeleton_jobOptions.py | tee athenaout.log
+
dq2_ls *Zmumu*AOD*
 
</pre>
 
</pre>
−
*You've just produced the AthenaAwareNTuple in a standalone way, directly on the AOD, or the ESD or even on the RDO. For Further details on how to produce the AthenaAwareNTuple in this way, follow this link: AthenaAwareNTuple.
 
  
−
*Now, you may look at the the AnalysisSkeleton.h, the AnalysisSkeleton.cxx and AnalysisSkeleton_jobOptions.py to see how the ROOT histograms and the AthenaAwareNTuple of the output file AnalysisSkeleton.aan.root are defined, filled and saved: Follow this link to the AnalysisSkeleton. For further details on how to produces ROOT histograms and Tuples in ATHENA, look at the THistSvc.
+
*Have a look at the files contained in one of the sets
 +
<pre>
 +
dq2_ls -f <datasetname>
 +
</pre>
  
−
*You should open the output file AnalysisSkeleton.aan.root in ROOT and browse the histograms and the AthenaAwareNTuple:
+
*change user.GustavoOrdonezSanz.00008.AAN.root for your own
 
<pre>
 
<pre>
−
root AnalysisSkeleton.aan.root
+
dq2_get -rv -d . user.GustavoOrdonezSanz.00008.AAN.root
−
new TBrowser
 
 
</pre>
 
</pre>
−
*You will notice that beside the user-defined NTuple, the AthenaAwareNTuple tools add references to the AOD, ESD and Raw Data, in addition the RunNumber and the EventNumber.
 
  
−
==More Information==
+
==Ganga==
−
*For further details on the analysis tools, look at the PhysicsAnalysisTools. Details on the content of the ESD, the AOD and the TAG are on StoregateKeysForESD, StoregateKeysForAOD and TagForEventSelection.
+
We're not going to cover Ganga in this tutorial. But for reference look at:[https://twiki.cern.ch/twiki/bin/view/Atlas/DistributedAnalysisUsingGanga Ganga link]
 +
 
 +
==Final Exercice==
 +
To conclude the workshop we're going to try to make our own Higgs to four muon analysis. Taking as an example the ZmumuOnAOD algorithm, make a H4mu one. The purpose of this, is to dig into the Phisics Analysis Tools used in ZmumuOnAOD, and try to have a realistic approach of how is like to make Analyis in Athena.
 +
 
 +
Good luck!

Latest revision as of 11:08, 25 October 2006

Pre-Tutorial

  • This pre-Tutorial is copy of the one given by Ketevi A. Assamagan, adapted to work in 12.0.2
  • here is a link to other Tutorials by Ketevi, and other usful PAT links:

PAT

  • Please follow the following pre-Tutorial before the date of the Workshop. It is a very straight forward, tutorial and will help to set up the athena enviroment for the workshop, so we can focus our time in more interested things

Pre-Tutorial link

During the workshop we'll browse over the code that we've used in the pre-Tutorial to try and uderstand it. Later we'll focused on the TAG and AAN selection.

Event Selection

First we're going to generate a new AAN that we'll use later for selection. 

cd ~scratch0/Workshop/
source setup.sh -tag=12.0.2,opt
cd 12.0.2/PhysicsAnalysis/AnalysisCommon/UserAnalysis
cp ~ordonez/public/Workshop_files/src/AnalysisSkeleton.cxx src/.
cp ~ordonez/public/Workshop_files/UserAnalysis/AnalysisSkeleton.h UserAnalysis/.
cd cmt
gmake
cd ..
mkdir run2
cd run2
cp ~ordonez/public/Workshop_files/run2/AnalysisSkeleton_jobOptions.py .
cp ~ordonez/public/Workshop_files/data2/PoolFileCatalog.xml .
cp ~ordonez/public/Workshop_files/data2/mc11.004202.ZmumuJimmy.recon.AOD.v11000401.py .
athena AnalysisSkeleton_jobOptions.py | tee athenaOut_AnalysisSkeleton.log

This will generate the file AnalysisSkeleton.aan.root.

Event Selection using TAG

  • First we generate our own TAG.
  • Still in the new run2 directory:
cp ~ordonez/public/Workshop_files/run2/Main_topOptions.py .
  • Edit Main_topOptions.py and uncomment the line
### Create TAG file
doWriteTAG=True

athena Main_topOptions.py | tee athenaOut.log
  • This will generate the file called ZmumuOnAOD_TAG.root
  • Now we are ready to run our algorithms with a TAG selection on the fly. Edit Main_topOptions.py again comment out:
#doWriteTAG=True
  • and uncomment
doRunAODwithSelection=True

and

useTag=True
  • Copy:
cp ~ordonez/public/Workshop_files/run2/ZmumuOnAOD_new_jobOptions.py .
  • And run again. Note that only will run the events that satisfy the query:

EventSelector.Query = "NLooseMuon>0 && NLooseMuon<3 && abs(LooseMuonEta1)<2.5 && abs(LooseMuonEta2)<2.5 && LooseMuonPt1>10000 && LooseMuonPt2>10000" 


athena Main_topOptions.py | tee athenaOut.log
  • As an output you have ZmumuOnAOD_withTAGSelection.aan.root. You can have a look at it in root to make sure that the query has been satisfied
  • Now we'll rewrite our own AOD containeing only the events selected. We'll do it for only 50 events to gain some time
  • Edit again Main_topOptions.py and change:
TheApp.EvtMax = 500

to

TheApp.EvtMax = 50
  • Comment out
#doRunAODwithSelection=True
  • uncomment:
doRewriteAOD=True

and run once more 

athena Main_topOptions.py | tee athenaOut.log

You'll see a new Zmumu_new_AOD_withTAGSelection.root file has been created

Event Selection using AAN

  • Follow as in the previous section but useANN=True instead of useTAG=True. In the end you should have the following two files:

ZmumuOnAOD_withAANSelection.aan.root Zmumu_new_AOD_withAANSelection.root

Interactive Analysis in ATHENA

  • In this exercise, we will see how to do interactive analysis in ATHENA, how to browse the Raw Data in the AOD. Proceed as follows:
get_files Interactive_topO.py
  • Edit Interactive_topO.py and replace this line
EventSelector.InputCollections = [ "AOD.pool.root" ]
  • with this line:
include ( "mc11.004202.ZmumuJimmy.recon.AOD.v11000401.py" )
  • run
athena -i Interactive_topO.py
  • Initialize application manager
athena>theApp.initialize()
  • Run 1 event
athena>theApp.nextEvent()
  • Retrieve Muons from AOD
athena>mcon = PyParticleTools.getMuons("MuidMuonCollection")
  • Get the first muon
athena>m = mcon[0]
  • Get a list of methods
athena>dir(m)
  • See pT
athena>m.pt()
  • Get TrackParticle via ElementLink
athena>tp = m.track()
athena>dir(tp)
athena>tp.pt()
  • Retrieve the MC Truth
athena>mcc = PyTruthTools.getMcEvents("TruthEvent")
athena>mc = mcc[0]
athena>mc.alphaQCD()
  • Dump Data Store
athena>dumpSG()
  • Run next event
athena>theApp.nextEvent()

Rewind event counter 

athena>theApp.seek(0)
  • See pT distribution of muons
athena> plot("MuonContainer#MuidMuonCollection","$x.pt()",nEvent=5)


Analysis with the EventView

  • Due to the time I won't cover this section during the Workshop. But now you should have the basic knowledge requiered to understand it. You can learn about the Event View here

Analysis on the GRID

  • In the following examples we'll run on Zmumu datasets.
  • This will be a brief introduction to GRID tools for those of you who already went through the trouble of getting a GRID certificate.
  • First we should set up DQ2 tools:
source /afs/usatlas.bnl.gov/Grid/Don-Quijote/dq2_user_client/setup.csh.CERN

PANDA/PATHENA

  • For further details visit panda wiki
  • Setup PATHENA :
cd ~/scratch0/Workshop/12.0.2/PhysicsAnalysis/AnalysisCommon/UserAnalysis
cvs update -r UserAnalysis-00-08-13 -A python
cvs update -r UserAnalysis-00-08-13 -A share/pathena
cvs update -r UserAnalysis-00-08-13 -A share/pathena_util
cvs update -r UserAnalysis-00-08-13 -A share/ConfigExtractor.py
cd cmt
source setup.sh
gmake
cd ../run
  • run pathena

Note: change user.GustavoOrdonezSanz.00008.AAN.root with an appropiate name!! 

pathena ZmumuOnAOD_jobOptions.py --inDS 'csc11.005145.PythiaZmumu.recon.AOD.v11004103' --outDS \
'user.GustavoOrdonezSanz.00008.AAN.root'


DQ2 Tools

  • For furhter details visit DQ2 wiki
  • This tools make your life easier when working on the GRID
dq2_ls *Zmumu*AOD*
  • Have a look at the files contained in one of the sets
dq2_ls -f <datasetname>
  • change user.GustavoOrdonezSanz.00008.AAN.root for your own
dq2_get -rv -d . user.GustavoOrdonezSanz.00008.AAN.root

Ganga

We're not going to cover Ganga in this tutorial. But for reference look at:Ganga link

Final Exercice

To conclude the workshop we're going to try to make our own Higgs to four muon analysis. Taking as an example the ZmumuOnAOD algorithm, make a H4mu one. The purpose of this, is to dig into the Phisics Analysis Tools used in ZmumuOnAOD, and try to have a realistic approach of how is like to make Analyis in Athena.

Good luck!

Retrieved from "https://wiki.nikhef.nl/atlas/index.php?title=NIKHEF_PAT_workshop_Oct2006&oldid=4762"