User:Dennisvd@nikhef.nl/VASP

Compiling Openmpi on stoomboot

• Make sure the settings for the Intel compiler are correct. I've added this snippet to my .bash_profile:

if [ -f /opt/intel/composer_xe_2013.5.192/bin/compilervars.sh ]; then
    . /opt/intel/composer_xe_2013.5.192/bin/compilervars.sh intel64
fi

• Download the latest stable version of OpenMPI.

wget http://www.open-mpi.org/software/ompi/v1.6/downloads/openmpi-1.6.5.tar.bz2

• Unpack and configure

tar xfj openmpi-1.6.5.tar.bz2
cd openmpi-1.6.5
mkdir build
cd build
../configure --disable-shared --enable-static --prefix=/project/detrd/vasp/openmpi-1.6.5 CC=icc CXX=icpc F77=ifort FC=ifort

• Build and deploy

make all
make install

Note: I'm not sure if openmpi still depends on torque-devel for tm.h. I've installed torque-devel on stbc-i1. (It appears this is still required.) The 'shared' and 'static' options are to make sure we do not end up with a run-time dependency on the Intel libraries which are not on the worker nodes.

The following snippet sets the paths for openmpi use.

if [ -f /project/detrd/vasp/openmpi-1.6.5 ]; then
    export LD_LIBRARY_PATH="/project/detrd/vasp/openmpi-1.6.5/lib:$LD_LIBRARY_PATH"
    export PATH="/project/detrd/vasp/openmpi-1.6.5/bin:$PATH"
fi

Test an openmpi program. Examples are found in the examples directory. I use ring_f90.

The compilation warns about missing functionality:

/opt/intel/composer_xe_2013.5.192/compiler/lib/intel64/libimf.so: warning: warning: feupdateenv is not implemented and will always fail

But this can probably be ignored (VASP doesn't use these C99 construct AFAIK).

A Torque submit script testrun-openmpi.pbs:

#!/bin/sh
#PBS -N vasp-openmpi-test
#PBS -l nodes=2:ppn=1
mpirun ${HOME}/vasp/ring_f90

And then:

qsub testrun-openmpi.pbs

Building VASP with OpenMPI support

The Makefile needs two small edits compared to the original. In the MPI section:

FC=mpif90
FCL=$(FC)
CPP += -DMPI