User:Dennisvd@nikhef.nl/UserSpacePackageManagement

Managing software packages on the Grid is complicated, because there is no central control over the setup of the operating environment across clusters. Even in a relatively small, tightly coordinated collection of clusters such as BiG Grid, the offering of packages is not tailored to the user's needs.

There has always been the option for users (or rather, VOs) to place their own software in special locations on a per-cluster basis. This is a low-level fallback, basically offering just file system access and nothing else. To do proper package management in these so-called VO software areas is something that needs to be looked at.

There are various possibilities, but the one explored here concerns pkgsrc by NetBSD. It is a widely available open-source system that fetches and builds software from source whenever needed.

Installation steps

1. download pkgsrc (choose the most recent quarterly release; approximate size is 30MB)
2. Let's assume the the VO software area is /data/esia/pvier. Unpack the pkgsrc distribution there.

   tar xfz pkgsrc.tar.gz

   cd pkgsrc

3. bootstrap it for the local system. This creates a directory called 'work'.

   ./bootstrap --prefix /data/esia/pvier/pkg --unprivileged

   this will install the basics in prefix already.

   (I should try this on a minimalist system to see which build dependencies this brings).

4. Add the bin and sbin directories to the PATH and install any package

   export PATH=/data/esia/pvier/pkg/bin:/data/esia/pvier/pkg/sbin:$PATH

5. build something: cd lang/python26 && bmake
6. bmake install
7. bmake package

Bootstrapping

Trying to create a binary package of the bootstrapping phase, with ./bootstrap --gzip-binary-kit /tmp/pkgsrc-bootstrap.tar.gz --workdir /tmp/bootstrap --unprivileged --prefix=/tmp/foo