Difference between revisions of "LHCb Software At Nikhef: Prepare for Workshop 22-24 June 2009"

From LHCb Wiki
Jump to navigation Jump to search
 
(35 intermediate revisions by 2 users not shown)
Line 5: Line 5:
 
Instead we provide two solutions:
 
Instead we provide two solutions:
  
1) Connect with ethernet cable to a computer/server.
+
1) Connect to a computer/server.
 
(See: [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php?title=LHCb_Software_At_Nikhef:_Prepare_for_Workshop_22-24_June_2009#Running_on_Workshop_Server Running on Workshop Server] )
 
(See: [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php?title=LHCb_Software_At_Nikhef:_Prepare_for_Workshop_22-24_June_2009#Running_on_Workshop_Server Running on Workshop Server] )
  
Line 30: Line 30:
  
 
3) Gaudi [https://twiki.cern.ch/twiki/bin/view/LHCb/LHCbSoftwareTrainingBasics Basics (twiki)]
 
3) Gaudi [https://twiki.cern.ch/twiki/bin/view/LHCb/LHCbSoftwareTrainingBasics Basics (twiki)]
 
== Running at Nikhef ==
 
 
=== Logon at Nikhef ===
 
 
<pre>
 
ssh user@ribble.nikhef.nl
 
ssh stbc-16  (on stoomboot)
 
</pre>
 
 
=== Set the environment ===
 
 
Here we set the environment for DaVinci, but it could also another LHCb application, like Bender or Panoramix.
 
<pre>
 
mkdir cmtuser
 
cd cmtuser
 
source /project/bfys/lhcb/sw/setup.sh  # for sh, bash, ksh, zsh
 
source /project/bfys/lhcb/sw/setup.csh  # for csh, tcsh
 
SetupProject --build-env DaVinci v23r1 # makes a DaVinci_v23r1 directory if one doesn't already exist
 
SetupProject DaVinci v23r1 # exports environment variables to the shell
 
</pre>
 
 
(NB: zsh-users should add 'setopt SH_WORD_SPLIT' to their .zshrc )
 
  
 
== Running on Workshop Server ==
 
== Running on Workshop Server ==
  
=== Logon to server ===
+
=== Login ===
  
Linux:  
+
Linux or mac:  
 
<pre>
 
<pre>
ssh -Y u01@192.16.186.107   (# Available usernames: u01 - u15)
+
ssh -Y u01@192.16.186.110   (# Available usernames: u01 - u15)
 
Password: (LHCb psswrd)
 
Password: (LHCb psswrd)
 
</pre>
 
</pre>
Line 67: Line 44:
 
<pre>
 
<pre>
 
Make sure you have Exceed running
 
Make sure you have Exceed running
ssh to 192.16.186.107       (# Available usernames: u01 - u15)
+
ssh to 192.16.186.110       (# Available usernames: u01 - u15)
 
Password: (LHCb psswrd)
 
Password: (LHCb psswrd)
 
</pre>
 
</pre>
  
=== Setup Environment ===
+
'''NB: In Egmond use: 192.168.1.1 '''
  
(At startup, the script /opt/LHCb/LbLogin.sh is executed for all users.
+
=== Environment ===
It sets up the necessary environment.)
 
  
Each of the user accounts has the MicroDSTExample installed.
+
At startup, the script /opt/LHCb/LbLogin.sh is executed for all users. It sets up the LHCb environment. After this, follow the steps described in [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/LHCb_Software_At_Nikhef:_Prepare_for_Workshop_22-24_June_2009#Setting_the_Environment]
To run it:
 
  
<pre>
+
=== Run ===
SetupProject DaVinci v23r1
 
cd $MICRODSTEXAMPLEROOT/python
 
</pre>
 
  
Run over events as described in section [].
+
Run over events as described [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/LHCb_Software_At_Nikhef:_Prepare_for_Workshop_22-24_June_2009#Running_over_some_events_on_a_DST_or_MicroDST here].
  
== Running on a virtual machine on your very own laptop ==
+
== Running on a Cern Virtual Machine (CernVM) ==
  
This assumes you have successfully installed a CernVM on your computer. The installation varies between Windows, Mac OS and Linux, so see section further down for instructions on how to set it up. It also assumes you have set up basic login scripts, and installed AFS, X11R6, CVS and SVN if necessary. Also note that you '''have to run an application like DaVinci, Bender or Panoramix at least once while connected to the network in order to cache the necessary software'''.
+
This assumes you have successfully installed a CernVM on your computer. The installation varies between Windows, Mac OS and Linux, so see section further down for instructions on how to set it up. It also assumes you have set up basic login scripts, and installed AFS, X11R6, CVS and SVN if necessary, as indicated [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/How_to_install_and_configure_Cern_Virtual_Machine_(CernVM)_on_your_laptop#Configuration_of_CernVM here]. Also note that you '''have to run an application like DaVinci, Bender or Panoramix at least once while connected to the network in order to cache the necessary software'''.
  
===Log onto your CernVM===
+
===Login===
  
 
This could mean starting the virtual machine and CernVM, starting it and logging in "remotely" from a terminal (Linux, Mac) or Exceed (Windows).
 
This could mean starting the virtual machine and CernVM, starting it and logging in "remotely" from a terminal (Linux, Mac) or Exceed (Windows).
''' Linux/Mac login'''
+
 
 +
==== Linux/Mac ====
  
 
<pre>
 
<pre>
Line 99: Line 72:
 
</pre>
 
</pre>
  
'''Windows login'''
+
====Windows====
  
 
<pre>
 
<pre>
Line 109: Line 82:
 
</pre>
 
</pre>
  
''' AFS access '''
+
=== Set the environment ===
  
If you need access to CERN AFS areas (not necessary for the workshop),
+
If you have followed the instructions for [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/How_to_install_and_configure_Cern_Virtual_Machine_(CernVM)_on_your_laptop#Configuration_of_CernVM setting up CernVM] and added the necessary group login scripts to your own login scripts, the LHCb environment should be set. If this is not the case,
 +
<pre>
 +
# for sh, bash, ksh, zsh
 +
. /opt/lhcb/group_login.sh
 +
# for csh, tcsh
 +
source /lhcb/group_login.csh
 +
</pre>
 +
Then follow the steps outlined in [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/LHCb_Software_At_Nikhef:_Prepare_for_Workshop_22-24_June_2009#Setting_the_Environment Setting the Environment]. If you can't find the required version of DaVinci or Bender, try refreshing the CernVM cache. '''You need an internet connection for this!'''
 
<pre>
 
<pre>
klog user@CERN.CH
+
sudo /etc/init.d/cernvm restart
 
</pre>
 
</pre>
Where user is your CERN AFS username. Note the '''UPPER CASE''' for the domain name.
 
Be careful: '''Never do: ls /opt/ !! (It could start caching all atlas software...)'''.
 
  
=== Set the environment ===
+
The main difference in setting the application environments here is that if you do not have the necessary software installed, CernVM will pull it and cache it for you. So the first time you do this you should have an interned connection.
 +
 
 +
== Setting the Environment ==
 +
 
 +
We use the LHCb script [https://twiki.cern.ch/twiki/bin/view/LHCb/SetupProject SetupProject]. We don't need to care about details, other that when invoked with '''--build-env''' it doesn't set any environment variables in the shell, and it creates a folder
 +
<pre>
 +
~/cmtuser/ProjectName_vXrY
 +
</pre>
 +
where all the software should be built. We will be using the Bender v12r1 and DaVinci v23r1p1 environments, but since Bender used DaVinci, the environment necessary for the latter is provided by the former, so we will set the Bender environment.
  
This is the same as in the remote login to Nikhef, except that the LHCb environment should be set when you log in if you have set up your CernVM correctly.
 
 
<pre>
 
<pre>
sudo /etc/init.d/cernvm restart
+
# this line in case ~/cmtuser/Bender_v12r1 doesn't exist yet
SetupProject --build-env DaVinci v23r1 # makes a DaVinci_v23r1 directory if one doesn't already exist
+
SetupProject Bender v12r1 --build-env
SetupProject DaVinci v23r1 # exports environment variables to the shell
+
# this exports environment variables to the shell. Some people might not like doing this
 +
SetupProject Bender v12r1
 
</pre>
 
</pre>
The main difference here is that if you do not have the necessary software installed, CernVM will pull it and cache it for you. So the first time you do this you should have an interned connection.
+
 
 +
This procedure is common to working at CERN, Nikhef, CernVM and the workshop server, provided the LHCb scripts have been set up correctly, be it automatically in the login scripts, or explicitly by sourcing some script. Details are in the subsections for each platform.
  
 
== Running over some events on a DST or MicroDST ==
 
== Running over some events on a DST or MicroDST ==
  
After the environment has been set (either by remote login to Nikhef or CERN, or in the local CernVM on your laptop, we can start some simple analysis:
+
After the environment has been set (either by remote login to Nikhef or CERN, or in the local CernVM on your laptop,) we can start some simple analysis:
  
''' Get a nice package full of example scripts (new Bender Examples package exists too...)
+
=== Get Tutorial packages Tutorial/BenderTutor, Tutorial/GaudiPythonTutor if not already there ===
  
If you haven't already got an examples package, get it, as you might need to perform modifications to some of the scripts. This has to be done when you have access to the internet.
+
If you haven't already got the tutorial packages in your build area, get them, as you will need to perform modifications to the exercise scripts. This has to be done when you have access to the internet.
  
 
<pre>
 
<pre>
getpack -p anonymous Ex/MicroDSTExample v2r5
+
cd ~/cmtuser/Bender_v12r1
 +
 
 +
# if the package is not there, do:
 +
# cvs co -r Tutorial/GaudiPythonTutor
 +
# cvs co -r v12r1 Tutorial/BenderTutor
 +
 
 +
# re-sets the $GAUDIPYTHONTUTORROOT and $BENDERTUTOR environment variables
 +
SetupProject Bender v12r1
 +
cd $GAUDIPYTHONTUTOR/cmt/
 +
cmt config
 +
make
 +
cd $BENDERTUTORROOT/cmt
 +
cmt config
 +
make
 
</pre>
 
</pre>
 +
 +
=== Run over some events and make plots using a script from the release area ===
 +
Don't worry, we'll go through exercises that will gradually allow us to write such a script. For now, just run it and stare, awe-struck, at the amazing plots.
 +
 +
'''Run over events'''
  
 
<pre>
 
<pre>
# Run over events:
+
python -i $MICRODSTEXAMPLEROOT/scripts/MicroDSTReadingExample.py --input <filename>
python -i $MICRODSTEXAMPLEROOT/MicroDSTReadingExample.py --input datafile.mdst
 
 
 
# The data on the server is stored here:
 
# /data/bfys/lhcb/workshop/Bs2JpsiPhiWithMC_1.mdst
 
  
# The data for CernVM (windows) is here:
+
# <filename> can take one of the following values depending on context:
# /mnt/hgfs/data/Bs2JpsiPhiWithMC_1.mdst
 
  
# The data for CernVM (mac) is here:
+
# Workshop Server: /data/bfys/lhcb/workshop/Bs2JpsiPhiWithMC_1.mdst
# /data/lhcb/Bs2JpsiPhiWithMC_1.mdst
+
# Windows CernVM: /mnt/hgfs/data/Bs2JpsiPhiWithMC_1.mdst
 +
# Mac CernVM:      /data/lhcb/Bs2JpsiPhiWithMC_1.mdst
  
 
</pre>
 
</pre>
Line 172: Line 173:
 
SetupProject Gaudi v21r0
 
SetupProject Gaudi v21r0
 
python $GAUDIEXAMPLESROOT/python/HistoEx.py
 
python $GAUDIEXAMPLESROOT/python/HistoEx.py
SetupProject Bender v12r0
+
SetupProject Bender v12r1
 
cd $BENDEREXAMPLEROOT/python/BenderExample/
 
cd $BENDEREXAMPLEROOT/python/BenderExample/
 
./Phi.py
 
./Phi.py

Latest revision as of 12:03, 25 June 2009

Introduction

The Bfys workshop will not take place at a high energy physics lab. Wireless connection is available, but for the hands-on work, we will not rely on that. Instead we provide two solutions:

1) Connect to a computer/server. (See: Running on Workshop Server )

2) Install and configure virtual machine on your laptop, "CernVM". (How to install)

(This will give the possibility to run LHCb software locally, very much as if you logon to a computer inside your laptop...)


We ask you to try to run an example job on the server using your laptop, before the workshop starts on Monday. (See Running on Workshop Server)


In addition - to familiarize with the LHCb software jargon - we ask you to read through the following pages too:

1) Python Basics

2) Bender Tutorial (twiki) Tutorial (pdf)

3) Gaudi Basics (twiki)

Running on Workshop Server

Login

Linux or mac:

ssh -Y u01@192.16.186.110    (# Available usernames: u01 - u15)
Password: (LHCb psswrd)

Windows:

Make sure you have Exceed running
ssh to 192.16.186.110       (# Available usernames: u01 - u15)
Password: (LHCb psswrd)

NB: In Egmond use: 192.168.1.1

Environment

At startup, the script /opt/LHCb/LbLogin.sh is executed for all users. It sets up the LHCb environment. After this, follow the steps described in [1]

Run

Run over events as described here.

Running on a Cern Virtual Machine (CernVM)

This assumes you have successfully installed a CernVM on your computer. The installation varies between Windows, Mac OS and Linux, so see section further down for instructions on how to set it up. It also assumes you have set up basic login scripts, and installed AFS, X11R6, CVS and SVN if necessary, as indicated here. Also note that you have to run an application like DaVinci, Bender or Panoramix at least once while connected to the network in order to cache the necessary software.

Login

This could mean starting the virtual machine and CernVM, starting it and logging in "remotely" from a terminal (Linux, Mac) or Exceed (Windows).

Linux/Mac

ssh -Y -p 2222 CernVMUserName@localhost

Windows

Start exceed
Start ssh
Connect to vm with ssh to 192.168.204.128
     (Your number might be different!)
     (fink tunnel x11 connections in ssh!)

Set the environment

If you have followed the instructions for setting up CernVM and added the necessary group login scripts to your own login scripts, the LHCb environment should be set. If this is not the case,

# for sh, bash, ksh, zsh
. /opt/lhcb/group_login.sh
# for csh, tcsh
source /lhcb/group_login.csh

Then follow the steps outlined in Setting the Environment. If you can't find the required version of DaVinci or Bender, try refreshing the CernVM cache. You need an internet connection for this!

sudo /etc/init.d/cernvm restart

The main difference in setting the application environments here is that if you do not have the necessary software installed, CernVM will pull it and cache it for you. So the first time you do this you should have an interned connection.

Setting the Environment

We use the LHCb script SetupProject. We don't need to care about details, other that when invoked with --build-env it doesn't set any environment variables in the shell, and it creates a folder

~/cmtuser/ProjectName_vXrY

where all the software should be built. We will be using the Bender v12r1 and DaVinci v23r1p1 environments, but since Bender used DaVinci, the environment necessary for the latter is provided by the former, so we will set the Bender environment.

# this line in case ~/cmtuser/Bender_v12r1 doesn't exist yet
SetupProject Bender v12r1 --build-env
# this exports environment variables to the shell. Some people might not like doing this
SetupProject Bender v12r1

This procedure is common to working at CERN, Nikhef, CernVM and the workshop server, provided the LHCb scripts have been set up correctly, be it automatically in the login scripts, or explicitly by sourcing some script. Details are in the subsections for each platform.

Running over some events on a DST or MicroDST

After the environment has been set (either by remote login to Nikhef or CERN, or in the local CernVM on your laptop,) we can start some simple analysis:

Get Tutorial packages Tutorial/BenderTutor, Tutorial/GaudiPythonTutor if not already there

If you haven't already got the tutorial packages in your build area, get them, as you will need to perform modifications to the exercise scripts. This has to be done when you have access to the internet.

cd ~/cmtuser/Bender_v12r1

# if the package is not there, do:
# cvs co -r Tutorial/GaudiPythonTutor
# cvs co -r v12r1 Tutorial/BenderTutor

# re-sets the $GAUDIPYTHONTUTORROOT and $BENDERTUTOR environment variables
SetupProject Bender v12r1
cd $GAUDIPYTHONTUTOR/cmt/
cmt config
make
cd $BENDERTUTORROOT/cmt
cmt config
make

Run over some events and make plots using a script from the release area

Don't worry, we'll go through exercises that will gradually allow us to write such a script. For now, just run it and stare, awe-struck, at the amazing plots.

Run over events

python -i $MICRODSTEXAMPLEROOT/scripts/MicroDSTReadingExample.py --input <filename>

# <filename> can take one of the following values depending on context:

# Workshop Server: /data/bfys/lhcb/workshop/Bs2JpsiPhiWithMC_1.mdst
# Windows CernVM:  /mnt/hgfs/data/Bs2JpsiPhiWithMC_1.mdst
# Mac CernVM:      /data/lhcb/Bs2JpsiPhiWithMC_1.mdst

Plot some of the histograms made by the script

>>> massPlots[443].plot()
>>> massPlots[531].plot()
>>> massResPlots[531].plot()
>>> propTimePlot.plot()
>>> propTimeResPlot.plot()
>>> refitPropTimeResPlot.plot()
>>> Ctrl-D

Some more examples, using Gaudi or Bender environments and example scripts. Note we run them directly without getpacking them.

SetupProject Gaudi v21r0
python $GAUDIEXAMPLESROOT/python/HistoEx.py
SetupProject Bender v12r1
cd $BENDEREXAMPLEROOT/python/BenderExample/
./Phi.py

Introduction to Python

It is important that attendees make themselves familiar with the python programming language, at least at the level of writing simple scripts.

Python Basics

Extensive tutorial: Tutorial

Cheat Sheet: Cheat Sheet

Example: myfirst.py

Execute:
python myfirst.py