Difference between revisions of "User:Dennisvd@nikhef.nl/VASP"

From PDP/Grid Wiki
Jump to navigationJump to search
 
(11 intermediate revisions by the same user not shown)
Line 12: Line 12:
 
  mkdir build
 
  mkdir build
 
  cd build
 
  cd build
−
  ../configure --disable-shared --enable-static --prefix=/project/detrd/vasp/openmpi-1.6.5 CC=icc CXX=icpc F77=ifort FC=ifort
+
  ../configure --disable-shared --enable-static --prefix=/project/detrd/vasp/openmpi-1.6.5 --with-tm CC=icc CXX=icpc F77=ifort FC=ifort
 
* Build and  deploy
 
* Build and  deploy
 
  make all
 
  make all
 
  make install
 
  make install
  
−
Note: I'm not sure if openmpi still depends on torque-devel for tm.h. I've installed torque-devel on stbc-i1. (It appears this is still required.)
+
The torque-devel package should be installed so integration with the torque batch system will work.
−
The 'shared' and 'static' options are to make sure we do not end up with a run-time dependency on the Intel libraries which are not on the worker nodes.
+
The ''shared'' and ''static'' options are to make sure we do not end up with a run-time dependency on the Intel libraries which are not on the worker nodes. Unfortunately this makes the install base of openmpi ''very large'' (�600 MB).
  
 
The following snippet sets the paths for openmpi use.
 
The following snippet sets the paths for openmpi use.
−
  if [ -f /project/detrd/vasp/openmpi-1.6.5 ]; then
+
  if [ -e /project/detrd/vasp/openmpi-1.6.5 ]; then
 
     export LD_LIBRARY_PATH="/project/detrd/vasp/openmpi-1.6.5/lib:$LD_LIBRARY_PATH"
 
     export LD_LIBRARY_PATH="/project/detrd/vasp/openmpi-1.6.5/lib:$LD_LIBRARY_PATH"
 
     export PATH="/project/detrd/vasp/openmpi-1.6.5/bin:$PATH"
 
     export PATH="/project/detrd/vasp/openmpi-1.6.5/bin:$PATH"
Line 37: Line 37:
 
  #PBS -N vasp-openmpi-test
 
  #PBS -N vasp-openmpi-test
 
  #PBS -l nodes=2:ppn=1
 
  #PBS -l nodes=2:ppn=1
 +
LD_LIBRARY_PATH="/project/detrd/vasp/openmpi-1.6.5/lib:$LD_LIBRARY_PATH"
 +
export LD_LIBRARY_PATH
 +
PATH="/project/detrd/vasp/openmpi-1.6.5/bin:$PATH"
 +
export PATH
 
  mpirun ${HOME}/vasp/ring_f90
 
  mpirun ${HOME}/vasp/ring_f90
  
 
And then:
 
And then:
 
  qsub testrun-openmpi.pbs
 
  qsub testrun-openmpi.pbs
−
 
  
 
= Building VASP with OpenMPI support =
 
= Building VASP with OpenMPI support =
Line 49: Line 52:
 
to start fresh. I copied the vasp.5.lib dir as-is from /project/detrd/vasp/src/vasp.5.lib/.
 
to start fresh. I copied the vasp.5.lib dir as-is from /project/detrd/vasp/src/vasp.5.lib/.
  
−
The Makefile needs two small edits compared to the original. In the MPI section:
+
To use the Intel MKL FFT wrapper library, I had to build it myself (in the same directory):
 +
make -f /opt/intel/composer_xe_2013.5.192/mkl/interfaces/fftw3xf/makefile libintel64
 +
 
 +
The Makefile needs some edits compared to the original. In the MPI section:
 
  FC=mpif90
 
  FC=mpif90
 
  FCL=$(FC)
 
  FCL=$(FC)
 
  CPP += -DMPI
 
  CPP += -DMPI
  
−
and include support for parallel FFT:
+
and include support for parallel FFT with the Intel wrapper:
−
  FFT3D  = fftmpi.o fftmpi_map.o fft3dfurth.o fft3dlib.o  
+
  FFT3D  = fftmpiw.o fftmpi_map.o fftw3d.o fft3dlib.o   libfftw3xf_intel.a
 +
INCS = -I$(MKLROOT)/include/fftw
 +
 
 +
The prepocessor needs to be set to use NGZhalf:
 +
CPP    = $(CPP_)  -DHOST=\"LinuxIFC\" \
 +
          -DCACHE_SIZE=12000 -DPGF90 -Davoidalloc -DNGZhalf \
 +
 
  
 
The alternate relative path to the lib dir requires another change:
 
The alternate relative path to the lib dir requires another change:
Line 64: Line 76:
 
The compiler '''crashed''' on one file, but replacing -O3 by -O2 worked around that:
 
The compiler '''crashed''' on one file, but replacing -O3 by -O2 worked around that:
  
−
  ifort -FR -names lowercase -assume byterecl -heap-arrays 64 -O3 -ip -parallel -I/opt/intel/composer_xe_2013.5.192/mkl/include -I/opt/intel/composer_xe_2013.5.192/mkl/include/fftw  -c local_field.f90
+
  ifort -FR -names lowercase -assume byterecl -heap-arrays 64 -O3 -ip -parallel \
−
  : catastrophic error: **Internal compiler error: segmentation violation signal raised** Please report this error along with the circumstances in which it occurred in a Software Problem Report.  Note: File and line given may not be explicit cause of this error.
+
-I/opt/intel/composer_xe_2013.5.192/mkl/include \
 +
-I/opt/intel/composer_xe_2013.5.192/mkl/include/fftw  -c local_field.f90
 +
  : catastrophic error: **Internal compiler error: segmentation violation signal raised** \
 +
Please report this error along with the circumstances in which it occurred in a Software \
 +
Problem Report.  Note: File and line given may not be explicit cause of this error.
 
  compilation aborted for local_field.f90 (code 1)
 
  compilation aborted for local_field.f90 (code 1)
 +
 +
A line added to the makefile:
 +
local_field.o: OFLAG=-O2 -ip -parallel
 +
 +
Other lines for individual files were present that turned optimization down in a different way.
 +
 +
 +
== Building VASP 5.4.1 ==
 +
 +
(Jan 2016)
 +
 +
New version has a better layout of the source tree, so more straightforward to build.
 +
 +
Newer Intel compiler installation in /opt/intel (actually /public/public_linux/intel but symlinked because the original path was /opt/intel and that's embedded in nearly every script).
 +
 +
I've made a few changes to the VASP makefile.include:
 +
 +
# Precompiler options
 +
CPP_OPTIONS= -DMPI -DHOST=\"IFC91_ompi\" -DIFC \
 +
              -DCACHE_SIZE=4000 -DPGF90 -Davoidalloc \
 +
              -DMPI_BLOCK=8000 -DscaLAPACK -Duse_collective \
 +
              -DnoAugXCmeta -Duse_bse_te \
 +
              -Duse_shmem -Dtbdyn
 +
 +
CPP        = fpp -f_com=no -free -w0  $*$(FUFFIX) $*$(SUFFIX) $(CPP_OPTIONS)
 +
 +
FC        = mpif90
 +
FCL        = mpif90 -mkl
 +
 +
FREE      = -free -names lowercase
 +
 +
FFLAGS    = -assume byterecl
 +
OFLAG      = -O2
 +
OFLAG_IN  = $(OFLAG)
 +
DEBUG      = -O0
 +
 +
MKL_PATH  = $(MKLROOT)/lib/intel64
 +
BLAS      =
 +
LAPACK    =
 +
BLACS      = -lmkl_blacs_custom_lp64
 +
SCALAPACK  = $(MKL_PATH)/libmkl_scalapack_lp64.a $(BLACS)
 +
 +
OBJECTS    = fftmpiw.o fftmpi_map.o fftw3d.o fft3dlib.o \
 +
              libfftw3xf_intel.a
 +
INCS      =-I$(MKLROOT)/include/fftw
 +
 +
LLIBS      = $(SCALAPACK) $(LAPACK) $(BLAS)
 +
 +
OBJECTS_O1 += fft3dfurth.o fftw3d.o fftmpi.o fftmpiw.o
 +
OBJECTS_O2 += fft3dlib.o
 +
 +
# For what used to be vasp.5.lib
 +
CPP_LIB    = $(CPP)
 +
FC_LIB    = $(FC)
 +
CC_LIB    = icc
 +
CFLAGS_LIB = -O
 +
FFLAGS_LIB = -O1
 +
FREE_LIB  = $(FREE)
 +
 +
OBJECTS_LIB= linpack_double.o getshmem.o
 +
 +
# Normally no need to change this
 +
SRCDIR    = ../../src
 +
BINDIR    = ../../bin
 +
 +
 +
The blacs_custom is my own openmpi based build
 +
 +
/opt/intel/mkl/lib/intel64_lin/libmkl_blacs_custom_lp64.so
 +
 +
with the help of Intel's support pages. (And Ton helped install it along with the other blacs libraries.)

Latest revision as of 14:32, 28 January 2016

Retrieved from "https://wiki.nikhef.nl/grid/index.php?title=User:Dennisvd@nikhef.nl/VASP&oldid=9902"