SAC software procedures

This page contains several procedures regarding the release process of the Grid Security Middleware.

Source distribution release procedure

This is the Software release procedure for Grid Security Middleware. This page describes that what to do when your software is ready for public distribution as source tarball. These tarballs can be used by anyone who likes to do their own local builds, but they are also used to generate RPM (Red Hat) and deb (Debian) packages.

Mandatory Check list

1. make distcheck: sh bootstrap && ./configure && make distcheck must result in "Ready for distribution"
2. Configure.ac: Update the version of the component in the configure.ac
3. BUGS: Updated the BUGS file in the component directory with known issues
4. NEWS Updated the NEWS file: write release notes like information. Hint which version solves the problem.
5. ChangeLog: a "svn log > ChangeLog" for the time being.
6. INSTALL: Update installation instruction file (if applicable)
7. README: Update the README file with more useful information
8. create a tag in the version control system.

Extra info: Source repository details

The main source repository for these components is https://ndpfsvn.nikhef.nl/cgi-bin/viewvc.cgi/mwsec/, which can be used with SVN in the following ways:

SVN with SSH public key authentication (NB. username is always svn):

svn+ssh://svn@ndpfsvn.nikhef.nl/repos/mwsec/

SVN with https access:

https://ndpfsvn.nikhef.nl/repos/mwsec/

SVN with http access:

http://ndpfsvn.nikhef.nl/ro/mwsec/

Creating a tag in Grid Security Middleware

We take the example for glexec, but this will be a valid set of sets for any component:

• Go to the trunk, and make sure it's up to date:

# For glexec:
cd mwsec/trunk
svn update
cd ..

• Check out the tags directory for the component exclusively:

svn co --depth=empty svn+ssh://svn@ndpfsvn.nikhef.nl/repos/mwsec/tags
cd tags
svn co --depth=immediates svn+ssh://svn@ndpfsvn.nikhef.nl/repos/mwsec/tags/glexec
cd glexec

• Copy the HEAD from trunk into the tag directory, the directory name is only named by it's version:

svn copy ../../trunk/glexec 0_8_2
cd 0_8_2

• Make a list of the used SVN externals. Resolve the externals as local files for each tag:

svn propget svn:externals > my_externals.txt

• Make a list of the files associated to each external and remove the externals in the current SVN directory:

svn propdel svn:externals

• For each entry in the my_externals.txt file and each file associated to them do something like:

rm -rf m4
rm -rf src/{safefile-1.0,environ,realpath,fileutil}
svn copy --parents ../../../trunk/m4/lcmaps.m4 m4/lcmaps.m4
svn copy ../../../trunk/cgul/{safefile-1.0,environ,realpath,fileutil} src/
rm my_externals.txt
cd ..

• Commit!

svn commit

Release software as tarball

The software tarballs should be released and made available for download in http://software.nikhef.nl/security. The exact procedure is as follows:

• Make a clean/fresh check out the tag:

cd /tmp
svn co svn+ssh://svn@ndpfsvn.nikhef.nl/repos/mwsec/tags/glexec/0_8_10 glexec_0_8_10
cd glexec_0_8_10

• Do a clean build from the software from there on a Red Hat 5 (compatible) system, by running:

./bootstrap
./configure

This is required to have consistency in the versions of the autotools that are used in the distributed software.

• Create release tarbal:

make dist

• Copy the tarball to:

software.nikhef.nl:/project/srv/www/site/software/html/security/<component>/release-candidate/

• generate a SHA1 and SHA256 checksum of the tarball and place it in a file with the same name but with the .sha1 extension.

sha1sum <component>-<version>.tar.gz > <componen>-<version>.tar.gz.sha1
sha256sum <component>-<version>.tar.gz > <componen>-<version>.tar.gz.sha256

• Verify that this tarball can be used for packaging, and that the packaged software works.
• After sufficient certification, release the software by moving it to to the parent directory, and have the release manager sign the hashes by putting them in a GPG-signed e-mail to grid-mw-security@nikhef.nl.

Generating RPM packages from the distributed tarballs

The creation of RPM packages is automated with Cruisecontrol; this set-up guarantees a clean, reproducible build. The following procedure should be followed to trigger the generation of a new component build.

The creation of RPM packages from source tarball is done with the central build system called cruisecontrol.testbed. Access to this machine is restricted to users of the CTB testbed.

The build system runs a Cruisecontrol instance that monitors several spec file directories of the old VL-e project. The sources for this setup rarely need to be touched, only the files

centos5-i386-projects-mwsec 
centos5-x86_64-projects-mwsec

are relevant as they contain the base component lists for the packages to be build. Cruisecontrol monitors the SPEC file directory

span.nikhef.nl:/srv/project/rpmbuild/SPECS-MWSEC/

to see if any file is touched; this will trigger a new build of the corresponding component. This directory is automatically kept up-to-date (from within cruisecontrol itself) by tracking the SVN repository at

https://ndpfsvn.nikhef.nl/repos/mwsec/packaging/fedora/trunk

Note: the build cycle of Cruisecontrol is set to 5 minutes, so it may take up to 5 minutes to update the spec file, and another 5 minutes from there to trigger the actual build. To monitor the status of Cruisecontrol at http://cruisecontrol.testbed:8080/ or http://cruisecontrol.testbed:8080/dashboard/dashboard you may need to tune the network access parameters of your browser; here is an example in Firefox:

network.proxy.no_proxies_on localhost, 127.0.0.1
network.proxy.socks 127.0.0.1
network.proxy.socks_port 8888
network.proxy.socks_remote_dns true
network.proxy.socks_version 5
network.proxy.type 1

This should be combined with a SOCKS proxy to a testbed machine, e.g.:

ssh -n -N -f -D 8888 span.nikhef.nl

The following steps must be taken under the usercc account.

Getting the sources

All output should show up on span.nikhef.nl under /srv/project/rpmbuild/(S)RPMS/...; from this a yum-installable repo needs to be composed by copying the right files to /srv/project/mwsec/el5/mwsec/ and running

cd /srv/project/mwsec/el5/mwsec/x86_64/ && createrepo .
cd /srv/project/mwsec/el5/mwsec/x386/ && createrepo .
cd /srv/project/mwsec/el5/mwsec/SRPMS/ && createrepo .

After this step, the repositories need to be synced with software.nikhef.nl:

rsync -rLt /srv/project/mwsec/ software.nikhef.nl:/project/srv/www/site/software/html/dist/redhat/

Generating RPMs manually

The above procedure produces 'official' RPMS, in a reproducible way from a clean build environment. However, it is possible to generate RPM packages manually with a simple recipe.

(recipe here TODO)

Generating Debian packages from the distributed source tarballs

TODO: this section is coming soon.