IGTF:Distribution Build

The International Grid Trust Federastion (IGTF) maintains a distribution of trust anchors for use by relying parties, based on a Common Source maintained by the chairs or trusted introducers of the member PMAs: David Groep for the EUGridPMA, Mike Helm for the TAGPMA, and Yoshio Tanaka for the APGridPMA APGridPMA. The Common Source is hosted on the EUGridPMA CVS Repository, alongside the build tools.

Working with the CVS CA Repository

The CVS Repository is hosted at cvs.eugridpma.org at the Nikhef Data Processing facility in the secured services network (194.171.96.67). The CVS settings are

CVSROOT=:ext:username@cvs.eugridpma.org:/cvs/eugridpma
CVS_RSH=ssh

where username is the one assigned for ssh login (via cvs only!) to the TI for the PMA. The CVS repository contains two branches

• CA repository carep/ hosting the trust anchors and the IGTF build tools
• utilities provided by the IGTF, including the old (v2) fetch-crl, the LCG and EGI build tools, the PMA expiration-warning tool, and the GFD.125 compliance test suite

When working with the CA repository, make sure it remains consistent and secure, and periodically check for intrusions by comparing with a local trusted host. The typical work flow is:

• go to your own secure build host
• do a "cvs update -A -R -d ." in the carep/ directory
• review manually any changes imported!
• apply your own updates and carefully review them for compliance with the PMA status, GFD.125 and a valid signing_policy and namespaces file (for all accredited CAs)
• update the CHANGES file in the top-level directory
• do a "cvs commit" of the tree, checking if all changes made were found, and the CHANGES file updated

Build tools

Go to the carep/buildtools directory. Until at least mid-2012, we will build two formats of the IGTF distribution: a dual-hash OpenSSLv1 compliance version (with debian support), and a old-style single-hash version (without Debian support). Each of these formats uses its own build tool:

cabuild3.pl

dual-hash distribution with all formats. This is the version being actively developed and maintained, and any new formats or unit tests should be added here

cabuild.pl

single-hash old distribution for legacy/OSG/VOMS-Admin compatibility.

cabuild3

• go to the buildtools/ directory
• make sure the VERSION file has the correct version number (e.g. "1.99")
• make sure the RPM signing key (keyID 3CDBBC71) is available and defined in the rpmmacros file. Insert the secure USB key from the safe in the build host
• make sure an RPM build directory structure is available
• run the build tool, with signing ("-s"), overwriting any previous content ("-f"), with Debian support ("--mkdeb"), and writing the distribution output to a directory ("-o dirname")

./cabuild3.pl --version=AUTO -s -f  -o ~/1.99 --mkdeb

• give the passphrase
• copy the new distribution to the preview or target web site

rsync -e ssh -rav --delete ~/1.99 webegp@weikuip:/project/srv/www/site/eugridpma-dist/html/distribution/tests/PMA-PRIVATE-PREVIEW/

• remove the USB signing key from the build host and put it back in the safe (or bujild the legacy distro or EGI release now)

cabuild

• build the new-style distribution first, to run the unit tests
• run the old-build command, writing to a new directory ("-old")

./cabuild.pl --version=AUTO -s -f  -o ~/1.99-old

• copy the new "old" distribution to the preview or target web site

rsync -e ssh -rav --delete ~/1.40-old webegp@weikuip:/project/srv/www/site/eugridpma-dist/html/distribution/tests/PMA-PRIVATE-PREVIEW/

• remove the USB signing key from the build host and put it back in the safe (or bujild the legacy distro or EGI release now)

Signing keys

The IGTF uses the EUGridPMA Signing Key #3 for signing the distribution, RPMs and Debian repo files:

pub   1024D/3CDBBC71 2005-07-12
      Key fingerprint = D12E 9228 22BE 64D5 0146  188B C32D 99C8 3CDB BC71
uid                  EUGridPMA Distribution Signing Key 3 <info@eugridpma.org>

The RPM macros file should read:

%_topdir                /home/davidg/rpmbuild
%_tmppath               /home/davidg/rpmbuild/tmp
%_signature gpg
%_gpg_path /home/davidg/.gnupg/
%_gpg_name 3CDBBC71

where ~/.gnupg is a symlink to the USB mount point of the secure key, and the following directories must exist for rpmbuild to succeed

mkdir -p /home/davidg/rpmbuild/
mkdir -p /home/davidg/rpmbuild/SOURCES
mkdir -p /home/davidg/rpmbuild/SPECS
mkdir -p /home/davidg/rpmbuild/SRPMS
mkdir -p /home/davidg/rpmbuild/RPMS
mkdir -p /home/davidg/rpmbuild/RPMS/noarch
mkdir -p /home/davidg/rpmbuild/BUILD
mkdir -p /home/davidg/rpmbuild/tmp

Publication Repositories

The publication repository (dist.eugridpma.info) is hosted on the NDPF secure services network at 194.171.96.74 (with a luke-warm space at our fail-over location in Haarlem at lama.nikhefhousing.nl). Both services should be in synch, and this has to be done manually (see later).

Location of the web site data

All of the "distribution/" bit of the web site is contained at

/project/srv/www/site/eugridpma-dist/html/distribution/

as a flat-file repository. The web site cannot do any dynamic content (on purpose, it has no scripting language installed), so what you see is what you get. The IGTF distribution is contained in the igtf/subdirectory.

To install the distribution, on the secure build host build both old and new formats, and then do (for e.g. 1.99):

rsync -e ssh -rav --delete ~/1.99 webegp@weikuip:/project/srv/www/site/eugridpma-dist/html/distribution/igtf/
rsync -e ssh -rav --delete ~/1.99-old webegp@weikuip:/project/srv/www/site/eugridpma-dist/html/distribution/igtf/

then go to the distribution web site host and update the links to make it active:

rm current current-new current-old 1.98-is-current ; ln -s 1.99 current ; ln -s 1.99 current-new ; ln -s 1.99-old current-old ; touch 1.99-is-current

and then immediately go to the public web site (on zeis, www.eugridpma.org) and upload

• the news letter:
  • take the previous one as a template
  • update the version numbers everywhere (global replace)
  • put in the new changelog file
  • upload the newsletter to the www.eugridpma.org web site under newsletters/
• update the home page of the eugridpma to update the vresion number everywhere (use vi ;-)
• update the news block on the right on the same home page (still using vi)
• sync the public web site (on zeis) to the warm-spare public site (dodo) in haarlem using ./push-to-haarlem)
• send out the newsletter (mind the date/time) to announce@eugridpma.org (you should get a posting ack from mailman)

Syncing the luke-warm spare

Login to the master node as the service user, and from its home directory run the synch script after every change on the master host:

./push-dist-to-haarlem

You as a TI will be able to access the warm-spare with the same SSH public key and the same service account name.

And the same for the public web site (on zeis) when you change it:

./push-to-haarlem

EGI Interactions

Once the IGTF distibution is uploaded (to test or public area), you can build the EGI distribution from this one. You MUST to the IGTF distro first, since the EGI build process relies on its output

Building the EGI distro

Logon to the same secure build host as you used for the IGTF distro, and then

• go to util/patches/ca-policy-egi/
• create a documentation file for both LCG and EGI under doc/, using the previous version as a template and updating the CHANGES section
• make sure the USB signing key is in the machine, RPMbuild setup correctly, etc.
• build the EGI distribution, based on the IGTF public web site, and copy the output to the SWrel upstream repo

./builddist-egi.pl -r 1 --srcurl=http://dist.eugridpma.info/distribution/igtf/1.99 -v -t 'davidg@mestkar.nikhef.nl:/project/srv/www/site/egi-igtf/html/distribution/egi-1.99'

• build a historic EGI distrubtion for VOMS-Admin, without Debian, like above

./builddist-egi.pl -r 1 --srcurl=http://dist.eugridpma.info/distribution/igtf/1.99-old -v -t 'davidg@mestkar.nikhef.nl:/project/srv/www/site/egi-igtf/html/distribution/egi-1.99-old' --debian=0

• login to the SWrel upstream repo service (can be any web site), and make sure the content of the NRSW XML file matches the URL. Typically logon to mestkar and do

cd /project/srv/www/site/egi-igtf/html/distribution
mv egi-1.99-old/ca-policy-egi-core-1.99-1 egi-1.99/ca-policy-egi-core-1.99-1-old
cd egi-1.99
ln -s ca-policy-egi-core-1.99-1 current
ln -s ca-policy-egi-core-1.99-1-old current-old
ln -s current/GPG-KEY-EUGridPMA-RPM-3 GPG-KEY-EUGridPMA-RPM-3 
ls -lR | sed -e 's/davidg emin/nobody   nobody /g' > ls-lR
cd ..

• move the "egi/" link to point to the new version if the Nagios probes are all new everywhere

rm egi ; ln -s egi-1.99 egi

• synch to the standby location

rsync -rav -e ssh --delete \
       /project/srv/www/site/egi-igtf/html/ \
       lama.nikhefhousing.nl:/project/srv/www/site/egi-igtf/html/

• then start the release process as described at https://wiki.egi.eu/wiki/EGI_IGTF_Release_Process