Difference between revisions of "LHCb Software At Nikhef: Prepare for Workshop 22-24 June 2009"

From LHCb Wiki
Jump to navigation Jump to search
 
(116 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Your first time to run over a MicroDST at Nikhef ==
+
== Introduction ==
  
''' Prepare ssh on your Nikhef account '''
+
The Bfys workshop will not take place at a high energy physics lab.
 +
Wireless connection is available, but for the hands-on work, we will not rely on that.
 +
Instead we provide two solutions:
  
Follow these instructions:  
+
1) Connect to a computer/server.
[http://cvs.web.cern.ch/cvs/howto.php#accessing-sshlinux Configuring SSH access from Linux/Unix]
+
(See: [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php?title=LHCb_Software_At_Nikhef:_Prepare_for_Workshop_22-24_June_2009#Running_on_Workshop_Server Running on Workshop Server] )
 +
 
 +
2) Install and configure virtual machine on your laptop, "CernVM".
 +
[http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/How_to_install_and_configure_Cern_Virtual_Machine_(CernVM)_on_your_laptop (How to install)]
 +
 
 +
(This will give the possibility to run LHCb software locally,
 +
very much as if you logon to a computer inside your laptop...)
  
''' Logon at Nikhef '''
 
  
<pre>
 
ssh user@ribble.nikhef.nl
 
ssh stbc-16  (on stoomboot)
 
</pre>
 
  
''' Get software packages '''
+
We ask you to try to run an example job on the server '''using your laptop''', before the workshop starts on Monday.
 +
(See [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php?title=LHCb_Software_At_Nikhef:_Prepare_for_Workshop_22-24_June_2009#Running_on_Workshop_Server Running on Workshop Server])
  
<pre>
 
mkdir cmtuser
 
cd cmtuser
 
source /project/bfys/lhcb/sw/setup.sh  # for sh, bash, ksh, zsh
 
source /project/bfys/lhcb/sw/setup.csh  # for csh, tcsh
 
SetupProject --build-env DaVinci v23r1
 
getpack --user=CERNUSERNAME Ex/MicroDSTExample jpalac_20090610_1
 
</pre>
 
(NB: zsh-users should add 'setopt SH_WORD_SPLIT' to their .zshrc )
 
  
''' Loop over events '''
 
  
<pre>
+
In addition - to familiarize with the LHCb software jargon - we ask you to read through the
SetupProject DaVinci v23r1
+
following pages too:
cd ~/cmtuser/DaVinci_v23r1/Ex/MicroDSTExample/python
 
python -i MicroDSTReadingExample.py --input /data/bfys/h71/Workshop/SeqDC06selBs2JpsiPhiWithMC_50.0_Kevt_NoPVReFit.dst
 
</pre>
 
  
''' Plot '''
+
1) Python [http://www.astro.ufl.edu/~warner/prog/python.html Basics]
  
<pre>
+
2) Bender [https://twiki.cern.ch/twiki/bin/view/LHCb/BenderTutorial Tutorial (twiki)]
>>> massPlots[443].Draw()
+
[http://www.nikhef.nl/pub/experiments/bfys/lhcb/computing/20080522_NIKHEF_Bender-Tutorial.pdf Tutorial (pdf)]
>>> massPlots[531].Draw()
 
>>> massResPlots[531].Draw()
 
>>> propTimePlots[531].Draw()
 
>>> Ctrl-D
 
</pre>
 
  
== Introduction to Python ==
+
3) Gaudi [https://twiki.cern.ch/twiki/bin/view/LHCb/LHCbSoftwareTrainingBasics Basics (twiki)]
  
<pre>
+
== Running on Workshop Server ==
It is important that attendees make themselves familiar with the python programming language, at least at the level of writing simple scripts.
 
  
Extensive tutorial: [http://doc.python.org/tutorial Tutorial]
+
=== Login ===
  
 +
Linux or mac:
 +
<pre>
 +
ssh -Y u01@192.16.186.110    (# Available usernames: u01 - u15)
 +
Password: (LHCb psswrd)
 +
</pre>
  
 +
Windows:
 
<pre>
 
<pre>
 +
Make sure you have Exceed running
 +
ssh to 192.16.186.110      (# Available usernames: u01 - u15)
 +
Password: (LHCb psswrd)
 +
</pre>
  
== Install a Virtual Machine on laptop (windows): CernVM ==
+
'''NB: In Egmond use: 192.168.1.1 '''
  
See talk by Pere Mato, slide 14:
+
=== Environment ===
[http://indico.cern.ch/conferenceDisplay.py?confId=52399 17 March 2009]
 
  
CernVM beginner's guide:
+
At startup, the script /opt/LHCb/LbLogin.sh is executed for all users. It sets up the LHCb environment. After this, follow the steps described in [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/LHCb_Software_At_Nikhef:_Prepare_for_Workshop_22-24_June_2009#Setting_the_Environment]
[https://twiki.cern.ch/twiki/bin/view/Atlas/CernVMGuideDetail CernVM beginner's guide]
 
  
 +
=== Run ===
  
''' 1a) Download VMware player under Windows '''
+
Run over events as described [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/LHCb_Software_At_Nikhef:_Prepare_for_Workshop_22-24_June_2009#Running_over_some_events_on_a_DST_or_MicroDST here].
  
Download vmware player [http://www.vmware.com/download/player/ here]
+
== Running on a Cern Virtual Machine (CernVM) ==
<pre>
 
Unzip and put CernVM in fixed place. For example C:\Program Files\CernVM
 
Start vmplayer from desktop
 
Open CernVM from C:\Program Files\CernVM
 
Open in browser: http://192.168.204.128:8004/
 
                (your number might be different!)
 
Create account. For example: use lxplus username and password
 
Virtual organization: lhcb
 
Change memory allocation in vmware player
 
  -> troubleshoot
 
    -> change memory allocation
 
        -> 1024 MB
 
</pre>
 
  
''' 1b) Download VMware player under MAC '''
+
This assumes you have successfully installed a CernVM on your computer. The installation varies between Windows, Mac OS and Linux, so see section further down for instructions on how to set it up. It also assumes you have set up basic login scripts, and installed AFS, X11R6, CVS and SVN if necessary, as indicated [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/How_to_install_and_configure_Cern_Virtual_Machine_(CernVM)_on_your_laptop#Configuration_of_CernVM here]. Also note that you '''have to run an application like DaVinci, Bender or Panoramix at least once while connected to the network in order to cache the necessary software'''.
  
''' 1c) Download VMware player under Linux '''
+
===Login===
  
 +
This could mean starting the virtual machine and CernVM, starting it and logging in "remotely" from a terminal (Linux, Mac) or Exceed (Windows).
  
''' 2) Start CernVM '''
+
==== Linux/Mac ====
  
Follow CernVM beginner's guide step by step:
+
<pre>
[https://twiki.cern.ch/twiki/bin/view/Atlas/CernVMGuideDetail CernVM beginner's guide]
+
ssh -Y -p 2222 CernVMUserName@localhost
 +
</pre>
  
''' 3) Start Exceed + SSH '''
+
====Windows====
  
 
<pre>
 
<pre>
Line 99: Line 82:
 
</pre>
 
</pre>
  
''' 4) Configure CernVM '''
+
=== Set the environment ===
  
 +
If you have followed the instructions for [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/How_to_install_and_configure_Cern_Virtual_Machine_(CernVM)_on_your_laptop#Configuration_of_CernVM setting up CernVM] and added the necessary group login scripts to your own login scripts, the LHCb environment should be set. If this is not the case,
 +
<pre>
 +
# for sh, bash, ksh, zsh
 +
. /opt/lhcb/group_login.sh
 +
# for csh, tcsh
 +
source /lhcb/group_login.csh
 +
</pre>
 +
Then follow the steps outlined in [http://www.nikhef.nl/pub/experiments/bfyswiki/index.php/LHCb_Software_At_Nikhef:_Prepare_for_Workshop_22-24_June_2009#Setting_the_Environment Setting the Environment]. If you can't find the required version of DaVinci or Bender, try refreshing the CernVM cache. '''You need an internet connection for this!'''
 
<pre>
 
<pre>
(once for configuring cernvm:)
+
sudo /etc/init.d/cernvm restart
back to instructions afs
 
sudo conary update openafs=1.4.8-1-1
 
sudo conary update openafs-client=1.4.8-1-1
 
sudo conary update openafs-kernel=1.4.8-1-1
 
sudo /etc/init.d/afs start
 
 
</pre>
 
</pre>
  
''' 5) CernVM session on ssh '''
+
The main difference in setting the application environments here is that if you do not have the necessary software installed, CernVM will pull it and cache it for you. So the first time you do this you should have an interned connection.
  
 +
== Setting the Environment ==
 +
 +
We use the LHCb script [https://twiki.cern.ch/twiki/bin/view/LHCb/SetupProject SetupProject]. We don't need to care about details, other that when invoked with '''--build-env''' it doesn't set any environment variables in the shell, and it creates a folder
 
<pre>
 
<pre>
Never do: ls /opt/ !! (It could start caching all atlas software...)
+
~/cmtuser/ProjectName_vXrY
 +
</pre>
 +
where all the software should be built. We will be using the Bender v12r1 and DaVinci v23r1p1 environments, but since Bender used DaVinci, the environment necessary for the latter is provided by the former, so we will set the Bender environment.
  
kinit user@CERN.CH (capitals!)
+
<pre>
cd /opt/lhcb
+
# this line in case ~/cmtuser/Bender_v12r1 doesn't exist yet
source /opt/lhcb/LbLogin.csh
+
SetupProject Bender v12r1 --build-env
SetupProject Gaudi v21r0
+
# this exports environment variables to the shell. Some people might not like doing this
python $GAUDIEXAMPLESROOT/python/HistoEx.py
+
SetupProject Bender v12r1
SetupProject Bender v12r0
 
cd $BENDEREXAMPLEROOT/python/BenderExample/
 
./Phi.py
 
 
</pre>
 
</pre>
  
''' 6) Create shared data folder on your laptop '''
+
This procedure is common to working at CERN, Nikhef, CernVM and the workshop server, provided the LHCb scripts have been set up correctly, be it automatically in the login scripts, or explicitly by sourcing some script. Details are in the subsections for each platform.
 +
 
 +
== Running over some events on a DST or MicroDST ==
 +
 
 +
After the environment has been set (either by remote login to Nikhef or CERN, or in the local CernVM on your laptop,) we can start some simple analysis:
 +
 
 +
=== Get Tutorial packages Tutorial/BenderTutor, Tutorial/GaudiPythonTutor if not already there ===
 +
 
 +
If you haven't already got the tutorial packages in your build area, get them, as you will need to perform modifications to the exercise scripts. This has to be done when you have access to the internet.
  
 
<pre>
 
<pre>
Make folder C:\Data
+
cd ~/cmtuser/Bender_v12r1
Change properties of C:\Data
+
 
  -> sharing
+
# if the package is not there, do:
    -> network sharing without wizard
+
# cvs co -r Tutorial/GaudiPythonTutor
      V share
+
# cvs co -r v12r1 Tutorial/BenderTutor
          name: Data
+
 
      V allow
+
# re-sets the $GAUDIPYTHONTUTORROOT and $BENDERTUTOR environment variables
 +
SetupProject Bender v12r1
 +
cd $GAUDIPYTHONTUTOR/cmt/
 +
cmt config
 +
make
 +
cd $BENDERTUTORROOT/cmt
 +
cmt config
 +
make
 
</pre>
 
</pre>
 +
 +
=== Run over some events and make plots using a script from the release area ===
 +
Don't worry, we'll go through exercises that will gradually allow us to write such a script. For now, just run it and stare, awe-struck, at the amazing plots.
 +
 +
'''Run over events'''
  
 
<pre>
 
<pre>
Edit the 2kB VMware configuration file (in C:\Program Files\CernVM\ ) and add:
+
python -i $MICRODSTEXAMPLEROOT/scripts/MicroDSTReadingExample.py --input <filename>
  
sharedFolder.maxNum = "1"
+
# <filename> can take one of the following values depending on context:
  
sharedFolder0.present = "TRUE"
+
# Workshop Server: /data/bfys/lhcb/workshop/Bs2JpsiPhiWithMC_1.mdst
sharedFolder0.enabled = "TRUE"
+
# Windows CernVM:  /mnt/hgfs/data/Bs2JpsiPhiWithMC_1.mdst
sharedFolder0.readAccess = "TRUE"
+
# Mac CernVM:     /data/lhcb/Bs2JpsiPhiWithMC_1.mdst
sharedFolder0.writeAccess = "TRUE"
 
sharedFolder0.hostPath = "C:\Data"
 
sharedFolder0.guestName = "data"
 
sharedFolder0.expiration = "session"
 
  
usb.autoConnect.device1 = ""
+
</pre>
  
NB: after editing the configuration file, restart VMware Player
+
''' Plot some of the histograms made by the script '''
 +
 
 +
<pre>
 +
>>> massPlots[443].plot()
 +
>>> massPlots[531].plot()
 +
>>> massResPlots[531].plot()
 +
>>> propTimePlot.plot()
 +
>>> propTimeResPlot.plot()
 +
>>> refitPropTimeResPlot.plot()
 +
>>> Ctrl-D
 
</pre>
 
</pre>
 +
 +
Some more examples, using Gaudi or Bender environments and example scripts. Note we run them directly without getpacking them.
  
 
<pre>
 
<pre>
In VMware:
+
SetupProject Gaudi v21r0
  -> Share folders
+
python $GAUDIEXAMPLESROOT/python/HistoEx.py
    -> o always enabled
+
SetupProject Bender v12r1
 +
cd $BENDEREXAMPLEROOT/python/BenderExample/
 +
./Phi.py
 
</pre>
 
</pre>
  
<pre>
+
== Introduction to Python ==
Add the following line to /etc/fstab:
 
sudo emacs -nw /etc/fstab
 
    .host:/ /mnt/hgfs vmhgfs defaults 0 0
 
  
sudo ln -s /usr/sbin/mount.vmhgfs /sbin
+
It is important that attendees make themselves familiar with the python programming language, at least at the level of writing simple scripts.
  
I edited Phi.py and put input data location: /mnt/hgfs/data/Bs2DsK_1.dst )
+
Python [http://www.astro.ufl.edu/~warner/prog/python.html Basics]
Data can be found here:
 
/afs/cern.ch/lhcb/group/calo/ecal/vol10/DATA
 
(or ask Niels or Vanya for a copy...)
 
</pre>
 
  
== Once everything is installed... Start CernVM ==
+
Extensive tutorial: [http://doc.python.org/tutorial Tutorial]
  
<pre>
+
Cheat Sheet: [http://www.yukoncollege.yk.ca/~ttopper/COMP118/rCheatSheet.html Cheat Sheet]
Start VMware Player (don't select 'Open', but 'Recent')
 
  
Login to your CernVM with your SSH.
+
Example: [http://www.nikhef.nl/~h71/Workshop/Instructions/myfirst.py myfirst.py ]
  
source /opt/lhcb/LbLogin.csh
+
<pre>
SetupProject Bender v12r0
+
Execute:
./Phi.py
+
python myfirst.py
  
root
 
TBrowser f
 
.q
 
 
</pre>
 
</pre>

Latest revision as of 12:03, 25 June 2009

Introduction

The Bfys workshop will not take place at a high energy physics lab. Wireless connection is available, but for the hands-on work, we will not rely on that. Instead we provide two solutions:

1) Connect to a computer/server. (See: Running on Workshop Server )

2) Install and configure virtual machine on your laptop, "CernVM". (How to install)

(This will give the possibility to run LHCb software locally, very much as if you logon to a computer inside your laptop...)


We ask you to try to run an example job on the server using your laptop, before the workshop starts on Monday. (See Running on Workshop Server)


In addition - to familiarize with the LHCb software jargon - we ask you to read through the following pages too:

1) Python Basics

2) Bender Tutorial (twiki) Tutorial (pdf)

3) Gaudi Basics (twiki)

Running on Workshop Server

Login

Linux or mac:

ssh -Y u01@192.16.186.110    (# Available usernames: u01 - u15)
Password: (LHCb psswrd)

Windows:

Make sure you have Exceed running
ssh to 192.16.186.110       (# Available usernames: u01 - u15)
Password: (LHCb psswrd)

NB: In Egmond use: 192.168.1.1

Environment

At startup, the script /opt/LHCb/LbLogin.sh is executed for all users. It sets up the LHCb environment. After this, follow the steps described in [1]

Run

Run over events as described here.

Running on a Cern Virtual Machine (CernVM)

This assumes you have successfully installed a CernVM on your computer. The installation varies between Windows, Mac OS and Linux, so see section further down for instructions on how to set it up. It also assumes you have set up basic login scripts, and installed AFS, X11R6, CVS and SVN if necessary, as indicated here. Also note that you have to run an application like DaVinci, Bender or Panoramix at least once while connected to the network in order to cache the necessary software.

Login

This could mean starting the virtual machine and CernVM, starting it and logging in "remotely" from a terminal (Linux, Mac) or Exceed (Windows).

Linux/Mac

ssh -Y -p 2222 CernVMUserName@localhost

Windows

Start exceed
Start ssh
Connect to vm with ssh to 192.168.204.128
     (Your number might be different!)
     (fink tunnel x11 connections in ssh!)

Set the environment

If you have followed the instructions for setting up CernVM and added the necessary group login scripts to your own login scripts, the LHCb environment should be set. If this is not the case,

# for sh, bash, ksh, zsh
. /opt/lhcb/group_login.sh
# for csh, tcsh
source /lhcb/group_login.csh

Then follow the steps outlined in Setting the Environment. If you can't find the required version of DaVinci or Bender, try refreshing the CernVM cache. You need an internet connection for this!

sudo /etc/init.d/cernvm restart

The main difference in setting the application environments here is that if you do not have the necessary software installed, CernVM will pull it and cache it for you. So the first time you do this you should have an interned connection.

Setting the Environment

We use the LHCb script SetupProject. We don't need to care about details, other that when invoked with --build-env it doesn't set any environment variables in the shell, and it creates a folder

~/cmtuser/ProjectName_vXrY

where all the software should be built. We will be using the Bender v12r1 and DaVinci v23r1p1 environments, but since Bender used DaVinci, the environment necessary for the latter is provided by the former, so we will set the Bender environment.

# this line in case ~/cmtuser/Bender_v12r1 doesn't exist yet
SetupProject Bender v12r1 --build-env
# this exports environment variables to the shell. Some people might not like doing this
SetupProject Bender v12r1

This procedure is common to working at CERN, Nikhef, CernVM and the workshop server, provided the LHCb scripts have been set up correctly, be it automatically in the login scripts, or explicitly by sourcing some script. Details are in the subsections for each platform.

Running over some events on a DST or MicroDST

After the environment has been set (either by remote login to Nikhef or CERN, or in the local CernVM on your laptop,) we can start some simple analysis:

Get Tutorial packages Tutorial/BenderTutor, Tutorial/GaudiPythonTutor if not already there

If you haven't already got the tutorial packages in your build area, get them, as you will need to perform modifications to the exercise scripts. This has to be done when you have access to the internet.

cd ~/cmtuser/Bender_v12r1

# if the package is not there, do:
# cvs co -r Tutorial/GaudiPythonTutor
# cvs co -r v12r1 Tutorial/BenderTutor

# re-sets the $GAUDIPYTHONTUTORROOT and $BENDERTUTOR environment variables
SetupProject Bender v12r1
cd $GAUDIPYTHONTUTOR/cmt/
cmt config
make
cd $BENDERTUTORROOT/cmt
cmt config
make

Run over some events and make plots using a script from the release area

Don't worry, we'll go through exercises that will gradually allow us to write such a script. For now, just run it and stare, awe-struck, at the amazing plots.

Run over events

python -i $MICRODSTEXAMPLEROOT/scripts/MicroDSTReadingExample.py --input <filename>

# <filename> can take one of the following values depending on context:

# Workshop Server: /data/bfys/lhcb/workshop/Bs2JpsiPhiWithMC_1.mdst
# Windows CernVM:  /mnt/hgfs/data/Bs2JpsiPhiWithMC_1.mdst
# Mac CernVM:      /data/lhcb/Bs2JpsiPhiWithMC_1.mdst

Plot some of the histograms made by the script

>>> massPlots[443].plot()
>>> massPlots[531].plot()
>>> massResPlots[531].plot()
>>> propTimePlot.plot()
>>> propTimeResPlot.plot()
>>> refitPropTimeResPlot.plot()
>>> Ctrl-D

Some more examples, using Gaudi or Bender environments and example scripts. Note we run them directly without getpacking them.

SetupProject Gaudi v21r0
python $GAUDIEXAMPLESROOT/python/HistoEx.py
SetupProject Bender v12r1
cd $BENDEREXAMPLEROOT/python/BenderExample/
./Phi.py

Introduction to Python

It is important that attendees make themselves familiar with the python programming language, at least at the level of writing simple scripts.

Python Basics

Extensive tutorial: Tutorial

Cheat Sheet: Cheat Sheet

Example: myfirst.py

Execute:
python myfirst.py