AMD GPU on IBM POWER

From PDP/Grid Wiki
Revision as of 20:55, 15 June 2020 by Tsuerink@nikhef.nl (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

The reason for this setup is to test if it is possible to combine AMD MI GPU's together with the PPC64le architecture.

Physical setup

Hardware shoppinglist

  • 1x IBM IC922 with dual socket 16 core cpu's and 128GB ram
  • 1x AMD MI50 GPU
  • Possibility to cut metal

Physical changes

The riser cards of the IC922 has a backplate with a hole for the power of the GPU. But this hole is too small to fit 2x 8-pin PCI-e connectors. So with some help of our mechanical department we now got a hole that's big enough. So make sure that you've got the tools or know somebody that can help you make this hole bigger. The MI50 GPU fits in the riser nicely and don't forget to hookup the power for the GPU.

Basic setup

Make sure that the fans in the IC922 are running at maximum speed all the time. The IC922 and the AMD Mi50 doesn't communicate with each other about how warm the GPU is and how much cooling it needs.

Configuration and Software

Installation and Configuration

We've taken Ubuntu 18.04 for this test. But you'll need to do a few modifications to get this working. 

Download the 5.6 kernel files for PPC64el from here and install them via dpkg.
Reboot the system to activate the new kernel.
git clone https://github.com/ROCm-Developer-Tools/aomp
cd aomp
export AOMP=/opt/rocm/
make install

This will take a while and every other dependency needed will be fetched by the installer. If everything went well, you'll should have ROCm and AOMP (clang +OpenMP) available in /opt/rocm/

Usage

Make sure that the following vaiables are set before using the software: 

export AOMP=/opt/rocm/
export PATH=$PATH:/opt/rocm/bin/

Example

Retrieved from "https://wiki.nikhef.nl/grid/index.php?title=AMD_GPU_on_IBM_POWER&oldid=10889"