Difference between revisions of "Modern C Compiler, ROOT, Geant"

From PDP/Grid Wiki
Jump to navigationJump to search
(Created page with "=== Introduction === So you want a modern C compiler and/or some other tool. As of this writing (2 November 2016) the normal desktop machines have gcc version 4.4.7 201203...")
 
(→‎Figure out your platform: Add different options (e.g. needed for Ganymede/Visar))
 
(18 intermediate revisions by one other user not shown)
Line 1: Line 1:
 
=== Introduction ===
 
=== Introduction ===
  
−
So you want a modern C compiler and/or some other tool.  As of this writing (2 November 2016) the normal desktop machines have  
+
So you want a modern C compiler and/or some other tool.  As of this writing (28 November 2017) the stoomboot interactive nodes have  
−
  gcc version 4.4.7 20120313 (Red Hat 4.4.7-17) (GCC)
+
  gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)
 
as the default compiler.  This is not recent enough for much of the HEP code ecosystem.
 
as the default compiler.  This is not recent enough for much of the HEP code ecosystem.
  
Line 22: Line 22:
  
 
Today on one of the stoomboot worker nodes, it says
 
Today on one of the stoomboot worker nodes, it says
−
  Scientific Linux CERN SLC release 6.8 (Carbon)
+
  Scientific Linux CERN SLC release 6.9 (Carbon)
 
  Kernel \r on an \m
 
  Kernel \r on an \m
  
 
Remember that first line, that tells you the platform, you'll need it shortly.
 
Remember that first line, that tells you the platform, you'll need it shortly.
  
−
=== Poke around in SFT and find the relevant release and compiler ===
 
  
−
stbc-022:lcg> ls /cvmfs/sft.cern.ch/lcg
+
Other ways to find which platform you're on include
−
app dev  external LCG85b   lcgjenkins  mapfile.txt     mapfile.txt.05042016  mapfile.txt.16092016 nightlies  releases-javier   updatemapfile.py
+
* cat /proc/version
−
contrib  etc  hepsoft lcgcmake  mapfile.ts  mapfile.txt.01052016  mapfile.txt.13042016  mapfile.txt.18082016 releases  root-training-setup.sh  views
+
* lsb_release -a
−
Note the LCG85b ... this is generally a listing of the latest full release; when there is a new release, the tag gets updated, so you might see LCG91, LCG93c, etc instead.  Whatever it is, this is the latest release (there are many older releases available btw).
+
* uname -a
 +
* ls /etc/*release* /etc/*version*
  
−
You can search for gcc in that LCG85b file, but it's easier to look at the file system.  Note the "LCG_85b" at the end of the "ls" command ... change this to whatever you find the most recent release to be!
+
=== Poke around in SFT and find the relevant release and compiler ===
  
−
  stbc-022:LCG_85b> cd /cvmfs/sft.cern.ch/lcg/releases/LCG_85b
+
  stbc-i2:lcg> . /cvmfs/sft.cern.ch/lcg/views/setupViews.sh
−
  stbc-022:LCG_85b> ls
+
  Usage:
−
  4suite       curl       GSL   jupyter_console       m4   oracle   pylint   root_numpy   urllib3
+
  setupViews <LCG_directory> <arch-os-complier>
−
AIDA       cx_oracle        gtest   jupyter_core       MarkupSafe   pacparser   pyminuit   rootpy   uuid
+
  where <LCG_directory> can be any of these
−
astroid      cycler       hepdata_converter  lapack       matplotlib   pandas   pyparsing   rpy2   valgrind
+
   
−
autoconf      Davix       hepdata_validator  LCGCMT       MCGenerators   pathlib2   pyqt   scikitlearn   vdt
+
  LCG_71
−
automake      decorator        HepMC   lcgenv       messaging   pathos   pyqt5   scipy   vectorclass
+
  LCG_71root6
−
backports    distribute      hepmc3   LCG_externals_x86_64-centos7-gcc49-dbg.txt   metakernel   pcre   pytest   setuptools   wcwidth
+
  LCG_72a
−
blas       doxygen       HepPDT   LCG_externals_x86_64-centos7-gcc49-opt.txt   mistune   pexpect   Python   simplegeneric   wheel
+
  LCG_79
−
Boost       eigen       igprof   LCG_externals_x86_64-slc6-gcc49-dbg.txt      mock   pickleshare   python_dateutil  singledispatch  widgetsnbextension
+
  LCG_81c
−
castor       elasticsearch    ipykernel   LCG_externals_x86_64-slc6-gcc49-opt.txt      mpich2   pip   pytimber   sip   xapian
+
   LCG_81d
−
ccache       entrypoints      ipython   LCG_generators_x86_64-centos7-gcc49-dbg.txt  multiprocessing   pkg_config   pytools   six   XercesC
+
  LCG_81e
−
certifi      expat       ipython_genutils   LCG_generators_x86_64-centos7-gcc49-opt.txt  mysql   prettytable   pytz   soqt   xqilla
+
  LCG_81f
−
clhep       fastjet       ipywidgets   LCG_generators_x86_64-slc6-gcc49-dbg.txt    mysql_python   processing   pyxml   spark   xrootd
+
  LCG_82rootaas2
−
CMake       fftw3       java   LCG_generators_x86_64-slc6-gcc49-opt.txt    nbconvert   prompt_toolkit  pyyaml   sqlalchemy   xrootd_python
+
  LCG_82rootaas3
−
cmaketools    fjcontrib        jemalloc   lcov       nbformat   ptyprocess   pyzmq   sqlite   yamlcpp
+
  LCG_82rootaas4
−
cmmnbuild    freetype       Jinja2   lhapdfsets       networkx   py   QMtest   stomppy   zeromq
+
  LCG_82rootaas5
−
cmt       frontier_client  joblib   libaio       neurobayes   py2neo   qt   storm   zlib
+
  LCG_82rootaas6
−
coin3d       gcc       jpype   libsvm       neurobayes_expert  py4j   qt5   subprocess32
+
  LCG_83
−
configparser  Geant4       json   libtool       ninja   pyanalysis   qtconsole   swig
+
  LCG_84
−
COOL       genshi       jsonc   libunwind       nose   pyapigitlab   qwt   sympy
+
   LCG_84swan1
−
CORAL       gmp       jsoncpp   libxml2       notebook   pydot   R   tbb
+
   LCG_84swan2
−
CouchDB      gperftools      jsonschema   libxslt       numexpr   pygments   RELAX   terminado
+
   LCG_84swan3
−
coverage      graphviz       jupyter   logilabcommon       numpy   pygraphics   requests   tornado
+
   LCG_84swan4
−
CppUnit      Grid       jupyter_client   lxml       openssl   pygsi   ROOT   traitlets
+
   LCG_85
 +
   LCG_85a
 +
   LCG_85b
 +
   LCG_85swan1
 +
   LCG_85swan2
 +
   LCG_85swan3
 +
   LCG_86
 +
   LCG_87
 +
   LCG_88
 +
   LCG_88Py3
 +
   LCG_89
 +
   LCG_89python3
 +
   LCG_90
 +
   LCG_90a
 +
   LCG_90python3
 +
   LCG_91
 +
   LCG_91python3
 +
   LCG_gaudiext20170622
 +
   LCG_latest
 +
   LCG_rootext20161010
 +
  LCG_rootext20161129
 +
   LCG_rootext20170508
 +
   LCG_rootext20170929
 +
  LCG_rootext20171122
 +
   dev3
 +
   dev3arch
 +
  dev3python3
 +
   dev4
 +
   devgeantv
 +
   geantv
 +
   ml/1.0
 
   
 
   
−
There is a lot of stuff in here that might be interesting to you (eg ROOT). See that gcc is also listed, list that directory to see the options:
+
  and <arch-os-complier> depends on what is available in <LCG_directory>
−
  stbc-022:LCG_85b> ls gcc
+
−
  4.9.3
+
  stbc-i2:lcg>
−
stbc-022:LCG_85b> ls gcc/4.9.3
+
Each of these LCG_* tags is a release of a bunch of software such as Tensorflow, Python, ROOT, scikitlearn, etc. To find out what is available inside each release (including versions) you can take a look at
−
x86_64-centos7 x86_64-slc6
+
[http://lcginfo.cern.ch/ this page].
  
−
So gcc 4.9.3 is there, and there are two versions.  Now the /etc/issue business above becomes relevant: it said "Scientific Linux CERN SLC release 6.8" which corresponds to the "slc6" version here, so that's the one you want to use in this example. 
+
The release exists in several versions, each one a specific combination of unix platform (red hat, ubuntu, etc) and C compiler version, as you can only be certain that software will work together if it's all been compiled on the same platform with the same C compiler. The views script will tell you which platform is available, if you select a LCG release:
−
  stbc-022:LCG_85b> ls gcc/4.9.3/x86_64-slc6
+
  stbc-i2:lcg> . /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_91
−
bin  include  lib  lib64  libexec  setup.csh  setup.csh~  setup.sh  setup.sh~  share
+
Available <arch-os-complier> for LCG_91 :
 +
  x86_64-centos7-gcc62-dbg
 +
  x86_64-centos7-gcc62-opt
 +
  x86_64-centos7-gcc7-dbg
 +
  x86_64-centos7-gcc7-opt
 +
  x86_64-slc6-gcc62-dbg
 +
  x86_64-slc6-gcc62-opt
 +
  x86_64-slc6-gcc7-dbg
 +
  x86_64-slc6-gcc7-opt
 +
  x86_64-ubuntu1604-gcc54-dbg
 +
  x86_64-ubuntu1604-gcc54-opt
 +
Going back to the previous section, we had CERN SLC 6.9 so this is the "slc6" platform ... C compilers available are gcc 6.2 or 7 ... and you can either have the software compiled with debugging enabled (dbg) or with optimisation turned on (opt).
  
 
=== Run the setup script ===
 
=== Run the setup script ===
 +
The views script will set things up for you.  Note that as of this writing, it doesn't work correctly with zsh but does with bash.
  
−
You want that setup.sh file ..
+
$ . /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_91 x86_64-slc6-gcc7-opt
−
 
+
After running that command, you can check that things are set up:
−
source /cvmfs/sft.cern.ch/lcg/releases/LCG_put_release_here/gcc/4.9.3/x86_64-slc6/setup.sh
+
  $ gcc --version
−
 
+
  gcc (GCC) 7.1.0
−
Note where it says LCG_put_release_here you'll need to put the release version you found (LCG_93c or whatever).
+
  Copyright (C) 2017 Free Software Foundation, Inc.
−
 
 
−
=== Compile your code ===
 
−
 
 
−
Now you'll have the right stuff in your path, etc to use this version of gcc:
 
−
  stbc-022:x86_64-slc6> gcc --version
 
−
  gcc (GCC) 4.9.3
 
−
  Copyright (C) 2015 Free Software Foundation, Inc.
 
 
  This is free software; see the source for copying conditions.  There is NO
 
  This is free software; see the source for copying conditions.  There is NO
 
  warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 
  warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 +
$ root -b
 +
  ------------------------------------------------------------
 +
  | Welcome to ROOT 6.10/06                http://root.cern.ch |
 +
  |                              (c) 1995-2017, The ROOT Team |
 +
  | Built for linuxx8664gcc                                    |
 +
  | From tag v6-10-06, 19 September 2017                      |
 +
  | Try '.help', '.demo', '.license', '.credits', '.quit'/'.q' |
 +
  ------------------------------------------------------------
 
   
 
   
−
  stbc-022:x86_64-slc6> g++ --version
+
  root [0] .q
−
  g++ (GCC) 4.9.3
+
$ python
−
  Copyright (C) 2015 Free Software Foundation, Inc.
+
Python 2.7.13 (default, May  9 2017, 16:41:38)
−
  This is free software; see the source for copying conditions. There is NO
+
  [GCC 7.1.0] on linux2
−
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+
  Type "help", "copyright", "credits" or "license" for more information.
 +
  >>>
 +
  $
 +
 
 +
=== Compile your code ===
  
 +
Now you'll have the right stuff in your path, etc to use that version of gcc.
 
Enjoy that prize.  Don't forget to mention the PDP group in your speech.
 
Enjoy that prize.  Don't forget to mention the PDP group in your speech.

Latest revision as of 09:47, 25 March 2022

Introduction

So you want a modern C compiler and/or some other tool. As of this writing (28 November 2017) the stoomboot interactive nodes have 

gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)

as the default compiler. This is not recent enough for much of the HEP code ecosystem.

Modern C compilers can be found in a repository called "SFT" which is maintained by CERN, and distributed via something called CVMFS which is installed on stoomboot machines and most of the desktop machines (older ones may not have it). You can check whether you have it with: 

ls /cvmfs/sft.cern.ch

if you don't get an error (and it says "lcg") then you're good. What you'll need to do to set up the compiler is:

  1. figure out what kind of platform you'll be running on
  2. poke around in SFT to find out what the current release is
  3. run the setup script
  4. compile your code and win the Nobel Prize.

Figure out your platform

You'll generally need to compile for a specific platform; go (one of) the machine(s) on which you'll be running your code and type 

cat /etc/issue

Today on one of the stoomboot worker nodes, it says 

Scientific Linux CERN SLC release 6.9 (Carbon)
Kernel \r on an \m

Remember that first line, that tells you the platform, you'll need it shortly.


Other ways to find which platform you're on include

  • cat /proc/version
  • lsb_release -a
  • uname -a
  • ls /etc/*release* /etc/*version*

Poke around in SFT and find the relevant release and compiler

stbc-i2:lcg> . /cvmfs/sft.cern.ch/lcg/views/setupViews.sh
Usage:
 setupViews <LCG_directory> <arch-os-complier>
  where <LCG_directory> can be any of these

 LCG_71
 LCG_71root6
 LCG_72a
 LCG_79
 LCG_81c
 LCG_81d
 LCG_81e
 LCG_81f
 LCG_82rootaas2
 LCG_82rootaas3
 LCG_82rootaas4
 LCG_82rootaas5
 LCG_82rootaas6
 LCG_83
 LCG_84
 LCG_84swan1
 LCG_84swan2
 LCG_84swan3
 LCG_84swan4
 LCG_85
 LCG_85a
 LCG_85b
 LCG_85swan1
 LCG_85swan2
 LCG_85swan3
 LCG_86
 LCG_87
 LCG_88
 LCG_88Py3
 LCG_89
 LCG_89python3
 LCG_90
 LCG_90a
 LCG_90python3
 LCG_91
 LCG_91python3
 LCG_gaudiext20170622
 LCG_latest
 LCG_rootext20161010
 LCG_rootext20161129
 LCG_rootext20170508
 LCG_rootext20170929
 LCG_rootext20171122
 dev3
 dev3arch
 dev3python3
 dev4
 devgeantv
 geantv
 ml/1.0

 and <arch-os-complier> depends on what is available in <LCG_directory>

stbc-i2:lcg>

Each of these LCG_* tags is a release of a bunch of software such as Tensorflow, Python, ROOT, scikitlearn, etc. To find out what is available inside each release (including versions) you can take a look at this page.

The release exists in several versions, each one a specific combination of unix platform (red hat, ubuntu, etc) and C compiler version, as you can only be certain that software will work together if it's all been compiled on the same platform with the same C compiler. The views script will tell you which platform is available, if you select a LCG release: 

stbc-i2:lcg> . /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_91
Available <arch-os-complier> for LCG_91 :
 x86_64-centos7-gcc62-dbg
 x86_64-centos7-gcc62-opt
 x86_64-centos7-gcc7-dbg
 x86_64-centos7-gcc7-opt
 x86_64-slc6-gcc62-dbg
 x86_64-slc6-gcc62-opt
 x86_64-slc6-gcc7-dbg
 x86_64-slc6-gcc7-opt
 x86_64-ubuntu1604-gcc54-dbg
 x86_64-ubuntu1604-gcc54-opt

Going back to the previous section, we had CERN SLC 6.9 so this is the "slc6" platform ... C compilers available are gcc 6.2 or 7 ... and you can either have the software compiled with debugging enabled (dbg) or with optimisation turned on (opt).

Run the setup script

The views script will set things up for you. Note that as of this writing, it doesn't work correctly with zsh but does with bash. 

$ . /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_91 x86_64-slc6-gcc7-opt
After running that command, you can check that things are set up:
$ gcc --version
gcc (GCC) 7.1.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ root -b
  ------------------------------------------------------------
 | Welcome to ROOT 6.10/06                http://root.cern.ch |
 |                               (c) 1995-2017, The ROOT Team |
 | Built for linuxx8664gcc                                    |
 | From tag v6-10-06, 19 September 2017                       |
 | Try '.help', '.demo', '.license', '.credits', '.quit'/'.q' |
  ------------------------------------------------------------

root [0] .q
$ python
Python 2.7.13 (default, May  9 2017, 16:41:38)
[GCC 7.1.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
$

Compile your code

Now you'll have the right stuff in your path, etc to use that version of gcc. Enjoy that prize. Don't forget to mention the PDP group in your speech.

Retrieved from "https://wiki.nikhef.nl/grid/index.php?title=Modern_C_Compiler,_ROOT,_Geant&oldid=10922"