Difference between revisions of "Modern C Compiler, ROOT, Geant"

From PDP/Grid Wiki
Jump to navigationJump to search
Line 29: Line 29:
 
=== Poke around in SFT and find the relevant release and compiler ===
 
=== Poke around in SFT and find the relevant release and compiler ===
  
stbc-i2:~> cd /cvmfs/sft.cern.ch/lcg/releases
 
stbc-i2:releases> ls -d LCG_*
 
LCG_71     LCG_81c     LCG_82rootaas4  LCG_84swan3  LCG_86
 
LCG_71root6  LCG_81d     LCG_82rootaas5  LCG_84swan4  LCG_87
 
LCG_72a      LCG_81e     LCG_82rootaas6  LCG_85   LCG_latest
 
LCG_75root6  LCG_81f     LCG_83     LCG_85a   LCG_rootext20161010
 
LCG_78root6  LCG_82     LCG_84     LCG_85b   LCG_rootext20161129
 
LCG_79     LCG_82rootaas1  LCG_84a     LCG_85swan1
 
LCG_80     LCG_82rootaas2  LCG_84swan1    LCG_85swan2
 
LCG_81b      LCG_82rootaas3  LCG_84swan2    LCG_85swan3
 
Note the LCG87 is the highest number ... this is generally the latest full release.  It will likely be different than what you now see, as there are new versions about once a month. Take a look at [http://lcginfo.cern.ch/ this page] for some documentation on what the various releases include.
 
 
Note I've replaced below, "LCG_87" with "LCG_XXX" in commands to help avoid mistakes made via blind copy-and-pasting.  Change the XXX to whatever you find the most recent release to be!
 
 
stbc-022:releases> cd /cvmfs/sft.cern.ch/lcg/releases/LCG_XXX
 
stbc-i2:LCG_XXX> ls
 
4suite gmp       LCG_externals_x86_64-ubuntu1604-gcc54-dbg.txt  pacparser        requests
 
AIDA gperftools       LCG_externals_x86_64-ubuntu1604-gcc54-opt.txt  pandas       ROOT
 
astroid graphviz       LCG_generators_x86_64-centos7-gcc62-dbg.txt    pathlib2       root_numpy
 
autoconf Grid       LCG_generators_x86_64-centos7-gcc62-opt.txt    pathos       rootpy
 
automake GSL       LCG_generators_x86_64-slc6-gcc49-dbg.txt       pcre       rpy2
 
backports gtest       LCG_generators_x86_64-slc6-gcc49-opt.txt       pexpect       scikitlearn
 
blas hadoop       LCG_generators_x86_64-slc6-gcc62-dbg.txt       pickleshare      scipy
 
Boost hepdata_converter       LCG_generators_x86_64-slc6-gcc62-opt.txt       pip       setuptools
 
castor hepdata_validator       LCG_generators_x86_64-ubuntu1604-gcc54-dbg.txt  pkg_config      simplegeneric
 
ccache HepMC       LCG_generators_x86_64-ubuntu1604-gcc54-opt.txt  pox       simplejson
 
certifi HepPDT       lcov       ppft       singledispatch
 
clhep hive       lhapdfsets       prettytable      sip
 
CMake igprof       libaio       processing      six
 
cmaketools ipykernel       libgit2       prompt_toolkit  sollya
 
cmmnbuild ipython       libsvm       protobuf       soqt
 
cmt ipython_genutils       libtool       psutil       spark
 
coin3d ipywidgets       libunwind       ptyprocess      sqlalchemy
 
configparser java       libxml2       py       sqlite
 
COOL jemalloc       libxslt       py2neo       stomppy
 
CORAL Jinja2       logilabcommon       py4j       storm
 
CouchDB joblib       lxml       pyanalysis      subprocess32
 
coverage jpype       m4       pyapigitlab      swig
 
cppgsl jsonc       MarkupSafe       pydot       sympy
 
CppUnit jsoncpp       matplotlib       pydot_ng       tbb
 
curl jsonschema       maven       pygments       terminado
 
cx_oracle jupyter       MCGenerators       pygraphics      theano
 
cycler jupyter_client       messaging       pygsi       tornado
 
cython jupyter_console       metakernel       pylint       traitlets
 
Davix jupyter_contrib_core       mistune       pyminuit       urllib3
 
DD4hep jupyter_contrib_nbextensions       mock       pyparsing        uuid
 
decorator jupyter_core       mpfi       pyqt       valgrind
 
dill jupyter_nbextensions_configurator       mpfr       pyqt5       vdt
 
distribute keras       mpich2       pytest       VecGeom
 
doxygen lapack       multiprocess       Python       vectorclass
 
eigen LCGCMT       multiprocessing       python_dateutil  wcwidth
 
elasticsearch LCG_contrib_x86_64-centos7-gcc62-dbg.txt    mysql       python_gitlab    wheel
 
entrypoints LCG_contrib_x86_64-centos7-gcc62-opt.txt    mysql_python       pytimber       widgetsnbextension
 
expat LCG_contrib_x86_64-slc6-gcc49-dbg.txt       nbconvert       pytools       xapian
 
fastjet LCG_contrib_x86_64-slc6-gcc49-opt.txt       nbformat       pytz       XercesC
 
fftw3 LCG_contrib_x86_64-slc6-gcc62-dbg.txt       networkx       pyxml       xqilla
 
fjcontrib LCG_contrib_x86_64-slc6-gcc62-opt.txt       neurobayes       pyyaml       xrootd
 
fplll LCG_contrib_x86_64-ubuntu1604-gcc54-dbg.txt  neurobayes_expert       pyzmq       xrootd_python
 
freetype LCG_contrib_x86_64-ubuntu1604-gcc54-opt.txt  ninja       QMtest       xz
 
frontier_client  lcgenv       nose       qt       yamlcpp
 
ftjam LCG_externals_x86_64-centos7-gcc62-dbg.txt  notebook       qt5       zeromq
 
future LCG_externals_x86_64-centos7-gcc62-opt.txt  numexpr       qtconsole        zlib
 
futures LCG_externals_x86_64-slc6-gcc49-dbg.txt      numpy       qwt
 
gcc LCG_externals_x86_64-slc6-gcc49-opt.txt      omniorb       R
 
Geant4 LCG_externals_x86_64-slc6-gcc62-dbg.txt      openssl       rangev3
 
genshi LCG_externals_x86_64-slc6-gcc62-opt.txt      oracle       RELAX
 
 
There is a lot of stuff in here that might be interesting to you (eg ROOT and Geant4). See that gcc is also listed, list that directory to see the options:
 
stbc-i2:LCG_XXX> ls gcc
 
4.9.1  4.9.3  6.2.0
 
 
The release exists in several versions, each one a specific combination of unix platform (red hat, ubuntu, etc) and C compiler version, as you can only be certain that software will work together if it's all been compiled on the same platform with the same C compiler. To see what's available, use the views script, which we will use again in the next section:
 
 
  stbc-i2:lcg> . /cvmfs/sft.cern.ch/lcg/views/setupViews.sh
 
  stbc-i2:lcg> . /cvmfs/sft.cern.ch/lcg/views/setupViews.sh
 
  Usage:
 
  Usage:
Line 160: Line 88:
 
   
 
   
 
  stbc-i2:lcg>
 
  stbc-i2:lcg>
 +
Each of these LCG_* tags is a release of a bunch of software such as Tensorflow, Python, ROOT, scikitlearn, etc.  To find out what is available inside each release (including versions) you can take a look at
 +
[http://lcginfo.cern.ch/ this page].
 +
 +
The release exists in several versions, each one a specific combination of unix platform (red hat, ubuntu, etc) and C compiler version, as you can only be certain that software will work together if it's all been compiled on the same platform with the same C compiler. The views script will tell you which platform is available, if you select a LCG release:
 +
stbc-i2:lcg> . /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_91
 +
Available <arch-os-complier> for LCG_91 :
 +
  x86_64-centos7-gcc62-dbg
 +
  x86_64-centos7-gcc62-opt
 +
  x86_64-centos7-gcc7-dbg
 +
  x86_64-centos7-gcc7-opt
 +
  x86_64-slc6-gcc62-dbg
 +
  x86_64-slc6-gcc62-opt
 +
  x86_64-slc6-gcc7-dbg
 +
  x86_64-slc6-gcc7-opt
 +
  x86_64-ubuntu1604-gcc54-dbg
 +
  x86_64-ubuntu1604-gcc54-opt
 +
Going back to the previous section,
  
 
=== Run the setup script ===
 
=== Run the setup script ===

Revision as of 10:59, 28 November 2017

Introduction

So you want a modern C compiler and/or some other tool. As of this writing (28 November 2017) the stoomboot interactive nodes have

gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)

as the default compiler. This is not recent enough for much of the HEP code ecosystem.

Modern C compilers can be found in a repository called "SFT" which is maintained by CERN, and distributed via something called CVMFS which is installed on stoomboot machines and most of the desktop machines (older ones may not have it). You can check whether you have it with:

ls /cvmfs/sft.cern.ch

if you don't get an error (and it says "lcg") then you're good. What you'll need to do to set up the compiler is:

  1. figure out what kind of platform you'll be running on
  2. poke around in SFT to find out what the current release is
  3. run the setup script
  4. compile your code and win the Nobel Prize.

Figure out your platform

You'll generally need to compile for a specific platform; go (one of) the machine(s) on which you'll be running your code and type

cat /etc/issue

Today on one of the stoomboot worker nodes, it says

Scientific Linux CERN SLC release 6.9 (Carbon)
Kernel \r on an \m

Remember that first line, that tells you the platform, you'll need it shortly.

Poke around in SFT and find the relevant release and compiler

stbc-i2:lcg> . /cvmfs/sft.cern.ch/lcg/views/setupViews.sh
Usage:
 setupViews <LCG_directory> <arch-os-complier>
  where <LCG_directory> can be any of these

 LCG_71
 LCG_71root6
 LCG_72a
 LCG_79
 LCG_81c
 LCG_81d
 LCG_81e
 LCG_81f
 LCG_82rootaas2
 LCG_82rootaas3
 LCG_82rootaas4
 LCG_82rootaas5
 LCG_82rootaas6
 LCG_83
 LCG_84
 LCG_84swan1
 LCG_84swan2
 LCG_84swan3
 LCG_84swan4
 LCG_85
 LCG_85a
 LCG_85b
 LCG_85swan1
 LCG_85swan2
 LCG_85swan3
 LCG_86
 LCG_87
 LCG_88
 LCG_88Py3
 LCG_89
 LCG_89python3
 LCG_90
 LCG_90a
 LCG_90python3
 LCG_91
 LCG_91python3
 LCG_gaudiext20170622
 LCG_latest
 LCG_rootext20161010
 LCG_rootext20161129
 LCG_rootext20170508
 LCG_rootext20170929
 LCG_rootext20171122
 dev3
 dev3arch
 dev3python3
 dev4
 devgeantv
 geantv
 ml/1.0

 and <arch-os-complier> depends on what is available in <LCG_directory>

stbc-i2:lcg>

Each of these LCG_* tags is a release of a bunch of software such as Tensorflow, Python, ROOT, scikitlearn, etc. To find out what is available inside each release (including versions) you can take a look at this page.

The release exists in several versions, each one a specific combination of unix platform (red hat, ubuntu, etc) and C compiler version, as you can only be certain that software will work together if it's all been compiled on the same platform with the same C compiler. The views script will tell you which platform is available, if you select a LCG release:

stbc-i2:lcg> . /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_91
Available <arch-os-complier> for LCG_91 :
 x86_64-centos7-gcc62-dbg
 x86_64-centos7-gcc62-opt
 x86_64-centos7-gcc7-dbg
 x86_64-centos7-gcc7-opt
 x86_64-slc6-gcc62-dbg
 x86_64-slc6-gcc62-opt
 x86_64-slc6-gcc7-dbg
 x86_64-slc6-gcc7-opt
 x86_64-ubuntu1604-gcc54-dbg
 x86_64-ubuntu1604-gcc54-opt

Going back to the previous section,

Run the setup script

You want that setup.sh file ..

source /cvmfs/sft.cern.ch/lcg/releases/LCG_XXX/gcc/X.Y.Z/x86_64-platform/setup.sh

Change XXX to the right LCG release tag, X.Y.Z to the right gcc version you found, and "platform" to the right platform you found.

note: csh users need the setup.csh file We (PDP) assume that the csh setup works correctly, none of us use csh, hence we don't know for sure ;-)

Compile your code

Now you'll have the right stuff in your path, etc to use this version of gcc:

stbc-022:x86_64-slc6> gcc --version
gcc (GCC) 4.9.3
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

stbc-022:x86_64-slc6> g++ --version
g++ (GCC) 4.9.3
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Enjoy that prize. Don't forget to mention the PDP group in your speech.