Difference between revisions of "LHCb Software At Nikhef: Prepare for Workshop 22-24 June 2009"

From LHCb Wiki
Jump to navigation Jump to search
 
(140 intermediate revisions by 3 users not shown)
Line 1: Line 1:
== Your first time to run over a MicroDST at Nikhef ==
+
== Introduction ==
  
''' Prepare ssh on your Nikhef account '''
+
The Bfys workshop will not take place at a high energy physics lab.
 +
Wireless connection is available, but for the hands-on work, we will not rely on that.
 +
Instead we provide two solutions:
  
Follow these instructions:  
+
1) Connect to a computer/server.
[http://cvs.web.cern.ch/cvs/howto.php#accessing-sshlinux Configuring SSH access from Linux/Unix]
+
(See: [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php?title=LHCb_Software_At_Nikhef:_Prepare_for_Workshop_22-24_June_2009#Running_on_Workshop_Server Running on Workshop Server] )
  
 +
2) Install and configure virtual machine on your laptop, "CernVM".
 +
[http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/How_to_install_and_configure_Cern_Virtual_Machine_(CernVM)_on_your_laptop (How to install)]
  
''' Get software packages '''
+
(This will give the possibility to run LHCb software locally,
 +
very much as if you logon to a computer inside your laptop...)
  
<pre>
 
mkdir cmtuser
 
cd cmtuser
 
source /project/bfys/lhcb/sw/setup.sh  # for sh, bash, ksh, zsh
 
source /project/bfys/lhcb/sw/setup.csh  # for csh, tcsh
 
SetupProject --build-env DaVinci v23r0p1
 
mkdir Phys
 
cd Phys
 
cp -r --no-dereference $DAVINCIROOT .
 
  
getpack Ex/MicroDSTExample v2r3
 
cd ~/cmtuser/DaVinci_v23r0p1/Phys/DaVinci/cmt/
 
cmt br cmt make
 
</pre>
 
(NB: zsh-users should add 'setopt SH_WORD_SPLIT' to their .zshrc )
 
  
''' Loop over events '''
+
We ask you to try to run an example job on the server '''using your laptop''', before the workshop starts on Monday.
 +
(See [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php?title=LHCb_Software_At_Nikhef:_Prepare_for_Workshop_22-24_June_2009#Running_on_Workshop_Server Running on Workshop Server])
 +
 
 +
 
 +
 
 +
In addition - to familiarize with the LHCb software jargon - we ask you to read through the
 +
following pages too:
 +
 
 +
1) Python [http://www.astro.ufl.edu/~warner/prog/python.html Basics]
 +
 
 +
2) Bender [https://twiki.cern.ch/twiki/bin/view/LHCb/BenderTutorial Tutorial (twiki)]
 +
[http://www.nikhef.nl/pub/experiments/bfys/lhcb/computing/20080522_NIKHEF_Bender-Tutorial.pdf Tutorial (pdf)]
 +
 
 +
3) Gaudi [https://twiki.cern.ch/twiki/bin/view/LHCb/LHCbSoftwareTrainingBasics Basics (twiki)]
 +
 
 +
== Running on Workshop Server ==
 +
 
 +
=== Login ===
  
 +
Linux or mac:
 
<pre>
 
<pre>
SetupProject DaVinci v23r0p1
+
ssh -Y u01@192.16.186.110    (# Available usernames: u01 - u15)
cd ~/cmtuser/DaVinci_v23r0p1/Ex/MicroDSTExample/python
+
Password: (LHCb psswrd)
python -i SomeTestFileToBeDefinedByJuan.py --input /data/bfys/h71/Workshop/SeqDC06selBs2JpsiPhiWithMC_50.0_Kevt_NoPVReFit.dst
 
 
</pre>
 
</pre>
  
''' Plot '''
+
Windows:
 
 
 
<pre>
 
<pre>
>>> massPlots.plots[443].Draw()
+
Make sure you have Exceed running
>>> massPlots.plots[531].Draw()
+
ssh to 192.16.186.110      (# Available usernames: u01 - u15)
>>> massResPlots.plots[531].Draw()
+
Password: (LHCb psswrd)
>>> propTimePlots.plots[531].Draw()
 
 
</pre>
 
</pre>
  
== Install a Virtual Machine on laptop (windows): CernVM ==
+
'''NB: In Egmond use: 192.168.1.1 '''
  
See talk by Pere Mato, slide 14:
+
=== Environment ===
[http://indico.cern.ch/conferenceDisplay.py?confId=52399 17 March 2009]
 
  
CernVM beginner's guide:
+
At startup, the script /opt/LHCb/LbLogin.sh is executed for all users. It sets up the LHCb environment. After this, follow the steps described in [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/LHCb_Software_At_Nikhef:_Prepare_for_Workshop_22-24_June_2009#Setting_the_Environment]
[https://twiki.cern.ch/twiki/bin/view/Atlas/CernVMGuideDetail CernVM beginner's guide]
 
  
 +
=== Run ===
  
''' 1a) Download VMware player under Windows '''
+
Run over events as described [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/LHCb_Software_At_Nikhef:_Prepare_for_Workshop_22-24_June_2009#Running_over_some_events_on_a_DST_or_MicroDST here].
  
<pre>
+
== Running on a Cern Virtual Machine (CernVM) ==
Download from [http://www.vmware.com/download/player/ vmware player]
 
Unzip and put CernVM in fixed place. For example C:\Program Files\CernVM
 
Start vmplayer from desktop
 
Open CernVM from C:\Program Files\CernVM
 
Open in browser: http://192.168.204.128:8004/
 
                (your number might be different!)
 
Create account. For example: use lxplus username and password
 
Virtual organization: lhcb
 
Change memory allocation in vmware player
 
  -> troubleshoot
 
    -> change memory allocation
 
        -> 1024 GB
 
</pre>
 
  
''' 1b) Download VMware player under MAC '''
+
This assumes you have successfully installed a CernVM on your computer. The installation varies between Windows, Mac OS and Linux, so see section further down for instructions on how to set it up. It also assumes you have set up basic login scripts, and installed AFS, X11R6, CVS and SVN if necessary, as indicated [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/How_to_install_and_configure_Cern_Virtual_Machine_(CernVM)_on_your_laptop#Configuration_of_CernVM here]. Also note that you '''have to run an application like DaVinci, Bender or Panoramix at least once while connected to the network in order to cache the necessary software'''.
  
''' 1c) Download VMware player under Linux '''
+
===Login===
  
 +
This could mean starting the virtual machine and CernVM, starting it and logging in "remotely" from a terminal (Linux, Mac) or Exceed (Windows).
  
''' 2) Start CernVM '''
+
==== Linux/Mac ====
  
Follow CernVM beginner's guide step by step:
+
<pre>
[https://twiki.cern.ch/twiki/bin/view/Atlas/CernVMGuideDetail CernVM beginner's guide]
+
ssh -Y -p 2222 CernVMUserName@localhost
 +
</pre>
  
''' 3) Start Exceed + SSH '''
+
====Windows====
  
 
<pre>
 
<pre>
 
Start exceed
 
Start exceed
start ssh
+
Start ssh
connect to vm with ssh to 192.168.204.128
+
Connect to vm with ssh to 192.168.204.128
 
     (Your number might be different!)
 
     (Your number might be different!)
 
     (fink tunnel x11 connections in ssh!)
 
     (fink tunnel x11 connections in ssh!)
 
</pre>
 
</pre>
  
''' 4) Configure CernVM '''
+
=== Set the environment ===
  
 +
If you have followed the instructions for [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/How_to_install_and_configure_Cern_Virtual_Machine_(CernVM)_on_your_laptop#Configuration_of_CernVM setting up CernVM] and added the necessary group login scripts to your own login scripts, the LHCb environment should be set. If this is not the case,
 
<pre>
 
<pre>
(once for configuring cernvm:)
+
# for sh, bash, ksh, zsh
back to instructions afs
+
. /opt/lhcb/group_login.sh
sudo conary update openafs=1.4.8-1-1
+
# for csh, tcsh
sudo conary update openafs-client=1.4.8-1-1
+
source /lhcb/group_login.csh
sudo conary update openafs-kernel=1.4.8-1-1
+
</pre>
sudo /etc/init.d/afs start
+
Then follow the steps outlined in [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/LHCb_Software_At_Nikhef:_Prepare_for_Workshop_22-24_June_2009#Setting_the_Environment Setting the Environment]. If you can't find the required version of DaVinci or Bender, try refreshing the CernVM cache. '''You need an internet connection for this!'''
 +
<pre>
 +
sudo /etc/init.d/cernvm restart
 
</pre>
 
</pre>
  
''' 5) CernVM session on ssh '''
+
The main difference in setting the application environments here is that if you do not have the necessary software installed, CernVM will pull it and cache it for you. So the first time you do this you should have an interned connection.
  
 +
== Setting the Environment ==
 +
 +
We use the LHCb script [https://twiki.cern.ch/twiki/bin/view/LHCb/SetupProject SetupProject]. We don't need to care about details, other that when invoked with '''--build-env''' it doesn't set any environment variables in the shell, and it creates a folder
 
<pre>
 
<pre>
Never do: ls /opt/ !! (It could start caching all atlas software...)
+
~/cmtuser/ProjectName_vXrY
 +
</pre>
 +
where all the software should be built. We will be using the Bender v12r1 and DaVinci v23r1p1 environments, but since Bender used DaVinci, the environment necessary for the latter is provided by the former, so we will set the Bender environment.
 +
 
 +
<pre>
 +
# this line in case ~/cmtuser/Bender_v12r1 doesn't exist yet
 +
SetupProject Bender v12r1 --build-env
 +
# this exports environment variables to the shell. Some people might not like doing this
 +
SetupProject Bender v12r1
 +
</pre>
 +
 
 +
This procedure is common to working at CERN, Nikhef, CernVM and the workshop server, provided the LHCb scripts have been set up correctly, be it automatically in the login scripts, or explicitly by sourcing some script. Details are in the subsections for each platform.
 +
 
 +
== Running over some events on a DST or MicroDST ==
 +
 
 +
After the environment has been set (either by remote login to Nikhef or CERN, or in the local CernVM on your laptop,) we can start some simple analysis:
  
kinit user@CERN.CH (capitals!)
+
=== Get Tutorial packages Tutorial/BenderTutor, Tutorial/GaudiPythonTutor if not already there ===
cd /opt/lhcb
 
source /opt/lhcb/LbLogin.cshhcb/lhcb/GAUDI/GAUDI_v21r0
 
python $GAUDIEXAMPLESROOT/python/HistoEx.py
 
SetupProject Bender v12r0
 
cd $BENDEREXAMPLEROOT/python/BenderExample/
 
./Phi.py
 
  
''' 6) Create shared data folder on your laptop '''
+
If you haven't already got the tutorial packages in your build area, get them, as you will need to perform modifications to the exercise scripts. This has to be done when you have access to the internet.
  
 
<pre>
 
<pre>
Make folder C:\Data
+
cd ~/cmtuser/Bender_v12r1
Change properties of C:\Data
+
 
  -> sharing
+
# if the package is not there, do:
    -> network sharing without wizard
+
# cvs co -r Tutorial/GaudiPythonTutor
      V share
+
# cvs co -r v12r1 Tutorial/BenderTutor
          name: Data
+
 
      V allow
+
# re-sets the $GAUDIPYTHONTUTORROOT and $BENDERTUTOR environment variables
 +
SetupProject Bender v12r1
 +
cd $GAUDIPYTHONTUTOR/cmt/
 +
cmt config
 +
make
 +
cd $BENDERTUTORROOT/cmt
 +
cmt config
 +
make
 
</pre>
 
</pre>
 +
 +
=== Run over some events and make plots using a script from the release area ===
 +
Don't worry, we'll go through exercises that will gradually allow us to write such a script. For now, just run it and stare, awe-struck, at the amazing plots.
 +
 +
'''Run over events'''
  
 
<pre>
 
<pre>
Edit configuration file (in C:\Program Files\CernVM )
+
python -i $MICRODSTEXAMPLEROOT/scripts/MicroDSTReadingExample.py --input <filename>
  
sharedFolder.maxNum = "1"
+
# <filename> can take one of the following values depending on context:
  
sharedFolder0.present = "TRUE"
+
# Workshop Server: /data/bfys/lhcb/workshop/Bs2JpsiPhiWithMC_1.mdst
  sharedFolder0.enabled = "TRUE"
+
# Windows CernVM: /mnt/hgfs/data/Bs2JpsiPhiWithMC_1.mdst
sharedFolder0.readAccess = "TRUE"
+
# Mac CernVM:     /data/lhcb/Bs2JpsiPhiWithMC_1.mdst
sharedFolder0.writeAccess = "TRUE"
 
sharedFolder0.hostPath = "C:\Data"
 
sharedFolder0.guestName = "data"
 
sharedFolder0.expiration = "session"
 
  
usb.autoConnect.device1 = ""
 
 
</pre>
 
</pre>
 +
 +
''' Plot some of the histograms made by the script '''
  
 
<pre>
 
<pre>
In VMware:
+
>>> massPlots[443].plot()
  -> Share folders
+
>>> massPlots[531].plot()
    -> o always enabled
+
>>> massResPlots[531].plot()
 +
>>> propTimePlot.plot()
 +
>>> propTimeResPlot.plot()
 +
>>> refitPropTimeResPlot.plot()
 +
>>> Ctrl-D
 
</pre>
 
</pre>
 +
 +
Some more examples, using Gaudi or Bender environments and example scripts. Note we run them directly without getpacking them.
  
 
<pre>
 
<pre>
sudo ln -s /usr/sbin/mount.vmhgfs /sbin
+
SetupProject Gaudi v21r0
 +
python $GAUDIEXAMPLESROOT/python/HistoEx.py
 +
SetupProject Bender v12r1
 +
cd $BENDEREXAMPLEROOT/python/BenderExample/
 +
./Phi.py
 +
</pre>
  
I edited Phi.py and put input data location: /mnt/hgfs/data/Bs2DsK_1.dst )
+
== Introduction to Python ==
Data can be found here:
+
 
/afs/cern.ch/lhcb/group/calo/ecal/vol10/DATA
+
It is important that attendees make themselves familiar with the python programming language, at least at the level of writing simple scripts.
</pre>
+
 
 +
Python [http://www.astro.ufl.edu/~warner/prog/python.html Basics]
 +
 
 +
Extensive tutorial: [http://doc.python.org/tutorial Tutorial]
 +
 
 +
Cheat Sheet: [http://www.yukoncollege.yk.ca/~ttopper/COMP118/rCheatSheet.html Cheat Sheet]
  
== Once everything is installed... Start CernVM ==
+
Example: [http://www.nikhef.nl/~h71/Workshop/Instructions/myfirst.py myfirst.py ]
  
 
<pre>
 
<pre>
Don't select 'Open', but 'Recent'.
+
Execute:
 +
python myfirst.py
  
source /opt/lhcb/LbLogin.csh
 
SetupProject Bender v12r0
 
./Phi.py
 
 
</pre>
 
</pre>

Latest revision as of 12:03, 25 June 2009

Introduction

The Bfys workshop will not take place at a high energy physics lab. Wireless connection is available, but for the hands-on work, we will not rely on that. Instead we provide two solutions:

1) Connect to a computer/server. (See: Running on Workshop Server )

2) Install and configure virtual machine on your laptop, "CernVM". (How to install)

(This will give the possibility to run LHCb software locally, very much as if you logon to a computer inside your laptop...)


We ask you to try to run an example job on the server using your laptop, before the workshop starts on Monday. (See Running on Workshop Server)


In addition - to familiarize with the LHCb software jargon - we ask you to read through the following pages too:

1) Python Basics

2) Bender Tutorial (twiki) Tutorial (pdf)

3) Gaudi Basics (twiki)

Running on Workshop Server

Login

Linux or mac:

ssh -Y u01@192.16.186.110    (# Available usernames: u01 - u15)
Password: (LHCb psswrd)

Windows:

Make sure you have Exceed running
ssh to 192.16.186.110       (# Available usernames: u01 - u15)
Password: (LHCb psswrd)

NB: In Egmond use: 192.168.1.1

Environment

At startup, the script /opt/LHCb/LbLogin.sh is executed for all users. It sets up the LHCb environment. After this, follow the steps described in [1]

Run

Run over events as described here.

Running on a Cern Virtual Machine (CernVM)

This assumes you have successfully installed a CernVM on your computer. The installation varies between Windows, Mac OS and Linux, so see section further down for instructions on how to set it up. It also assumes you have set up basic login scripts, and installed AFS, X11R6, CVS and SVN if necessary, as indicated here. Also note that you have to run an application like DaVinci, Bender or Panoramix at least once while connected to the network in order to cache the necessary software.

Login

This could mean starting the virtual machine and CernVM, starting it and logging in "remotely" from a terminal (Linux, Mac) or Exceed (Windows).

Linux/Mac

ssh -Y -p 2222 CernVMUserName@localhost

Windows

Start exceed
Start ssh
Connect to vm with ssh to 192.168.204.128
     (Your number might be different!)
     (fink tunnel x11 connections in ssh!)

Set the environment

If you have followed the instructions for setting up CernVM and added the necessary group login scripts to your own login scripts, the LHCb environment should be set. If this is not the case,

# for sh, bash, ksh, zsh
. /opt/lhcb/group_login.sh
# for csh, tcsh
source /lhcb/group_login.csh

Then follow the steps outlined in Setting the Environment. If you can't find the required version of DaVinci or Bender, try refreshing the CernVM cache. You need an internet connection for this!

sudo /etc/init.d/cernvm restart

The main difference in setting the application environments here is that if you do not have the necessary software installed, CernVM will pull it and cache it for you. So the first time you do this you should have an interned connection.

Setting the Environment

We use the LHCb script SetupProject. We don't need to care about details, other that when invoked with --build-env it doesn't set any environment variables in the shell, and it creates a folder

~/cmtuser/ProjectName_vXrY

where all the software should be built. We will be using the Bender v12r1 and DaVinci v23r1p1 environments, but since Bender used DaVinci, the environment necessary for the latter is provided by the former, so we will set the Bender environment.

# this line in case ~/cmtuser/Bender_v12r1 doesn't exist yet
SetupProject Bender v12r1 --build-env
# this exports environment variables to the shell. Some people might not like doing this
SetupProject Bender v12r1

This procedure is common to working at CERN, Nikhef, CernVM and the workshop server, provided the LHCb scripts have been set up correctly, be it automatically in the login scripts, or explicitly by sourcing some script. Details are in the subsections for each platform.

Running over some events on a DST or MicroDST

After the environment has been set (either by remote login to Nikhef or CERN, or in the local CernVM on your laptop,) we can start some simple analysis:

Get Tutorial packages Tutorial/BenderTutor, Tutorial/GaudiPythonTutor if not already there

If you haven't already got the tutorial packages in your build area, get them, as you will need to perform modifications to the exercise scripts. This has to be done when you have access to the internet.

cd ~/cmtuser/Bender_v12r1

# if the package is not there, do:
# cvs co -r Tutorial/GaudiPythonTutor
# cvs co -r v12r1 Tutorial/BenderTutor

# re-sets the $GAUDIPYTHONTUTORROOT and $BENDERTUTOR environment variables
SetupProject Bender v12r1
cd $GAUDIPYTHONTUTOR/cmt/
cmt config
make
cd $BENDERTUTORROOT/cmt
cmt config
make

Run over some events and make plots using a script from the release area

Don't worry, we'll go through exercises that will gradually allow us to write such a script. For now, just run it and stare, awe-struck, at the amazing plots.

Run over events

python -i $MICRODSTEXAMPLEROOT/scripts/MicroDSTReadingExample.py --input <filename>

# <filename> can take one of the following values depending on context:

# Workshop Server: /data/bfys/lhcb/workshop/Bs2JpsiPhiWithMC_1.mdst
# Windows CernVM:  /mnt/hgfs/data/Bs2JpsiPhiWithMC_1.mdst
# Mac CernVM:      /data/lhcb/Bs2JpsiPhiWithMC_1.mdst

Plot some of the histograms made by the script

>>> massPlots[443].plot()
>>> massPlots[531].plot()
>>> massResPlots[531].plot()
>>> propTimePlot.plot()
>>> propTimeResPlot.plot()
>>> refitPropTimeResPlot.plot()
>>> Ctrl-D

Some more examples, using Gaudi or Bender environments and example scripts. Note we run them directly without getpacking them.

SetupProject Gaudi v21r0
python $GAUDIEXAMPLESROOT/python/HistoEx.py
SetupProject Bender v12r1
cd $BENDEREXAMPLEROOT/python/BenderExample/
./Phi.py

Introduction to Python

It is important that attendees make themselves familiar with the python programming language, at least at the level of writing simple scripts.

Python Basics

Extensive tutorial: Tutorial

Cheat Sheet: Cheat Sheet

Example: myfirst.py

Execute:
python myfirst.py