Requesting or Renewing Host certificates

IMPORTANT NOTE: All nodes except UI, WN and BDII require the host certificate/key files to be installed.

Terena Certificates

IMPORTANT NOTE: All nodes except UI, WN and BDII require the host certificate/key files to be installed.

So far the certificates and the private keys are sent by encrypted email and then stored at vlaai:/export/perm/share/grid-security/terena-certs/$year, where $year is the current year when the certificate was issued. Under the directory terena-certs there are the following directories:

-keys: The private keys for the different certificates.

-$year: The certificates of the server for the different years.

-scripts: The scripts for copying the certificates to the different server directories in vlaai (/export/perm/share/grid-security/$host) and rolling back to the previous year ones.

Request a new host certificate

To request a new host certificate, just send an email either to Elly van Kerkhoff or Paul Kuipers with the name of the server(s) and if these certificates have to be Standard or eScience. If the server hosts any kind of Grid service, the certificate has to be eScience.

Don't forget to send your public key, so that they can encrypt the email that will contain the requested certificate(s) and the private key(s).

Renewing an existing (valid) host certificate

The procedure is similar to the previous one, but in this case only the certificate will be sent back. The private key to use it is the old one.

Installing the key/certificate pair

The script cpterenacerts.sh can be used, in order to copy the certificates to the different server directories at vlaai. It performs the following steps:

- It gets the list with all the certificates (Since the server name is included in the certificate name, it's straight-forward to know to which server corresponds which certificate).

- It creates a directory $year for every host directory, in order to store the certificate.

- It copies the private key from the $year-1 directory.

- It deletes the symbolic links pointing to the certificate and key in the directory $year-1, and it creates new symbolic links pointing to those in the directory $year.

- It creates a file release.state with a recent time stamp (still to be done as root in the directory /export/perm/share/grid-security/$host):

- It SSHs into the host to install the certificate and key in the host under /etc/grid-security/, and in extra directories for those hosts that need additional copies of the certificate and key.

If you run into any problems when installing the certificate and key, it is possible to manually run the following lines at vlaai:/export/perm/share/grid-security/$host:

# touch release.state
# chown apache:apache release.state

And then to follow the same steps as for the installation of [Dutchgrid Certificates].

Rollback to the old certificate

This can be done with the script rollback.sh. This script just deletes the symbolic links pointing to the certificate and key of the current year and it points to the ones of the previous year. Then it performs the same steps as for the installation of [Dutchgrid Certificates].

It is quite useful, in case something goes wrong in the installation of the new certificate and it is necessary to reinstall the old one.

Dutchgrid Certificates (Deprecated)

IMPORTANT NOTE: All nodes except UI, WN and BDII require the host certificate/key files to be installed.

This guide describes how to request a new host certificate or renew an existing one, and what to do with the new/renewed certificate.

If the host already has a valid certificate, skip step 1a) and continue with step 1b)

Request a new host certificate

To request a new host certificate, follow the procedure on the Dutchgrid CA website. Part of this procedure involves downloading and executing a shell script. Executing the script has to be done as user root on server vlaai, in directory /export/perm/share/grid-security/hostname/year (which should probably be created first!). For example, for host graszode in year 2008:

# mkdir -p /export/perm/share/grid-security/graszode/2008
# cd /export/perm/share/grid-security/graszode/2008
# sh ./makerequest.sh .

This will create a few files, including one called userkey.pem

Complete the administrative procedure as described on the CA web page and continue with step 2).

Renewing an existing (valid) host certificate

To renew a host certificate, follow the procedure on the Dutchgrid CA website. Part of this procedure involves downloading and executing a shell script. Executing the script has to be done as user root on server vlaai. Create a new directory /export/perm/share/grid-security/hostname/year to hold the new key/certificate pair. Then execute the downloaded script, providing the existing certificate as parameter (according to convention, it should be present in the directory corresponding to the previous year) and use options -d to store the output files in the desired directory:

# mkdir -p /export/perm/share/grid-security/tbn01/2008
# cd /export/perm/share/grid-security/tbn01/2008
# ls -l ../2007/usercert.pem
-rw-r--r-- 1 root root 5146 Jan 30  2007 ../2007/usercert.pem
# dca-rekey-pack.sh -d . ../2007/usercert.pem

This will create a few files, including one called userkey.pem

A one liner to do a renewal:

# cd /export/perm/share/grid-security/<machine name>
# mkdir $((  `date +%Y` )) && cd $((  `date +%Y` )) && wget http://ca.dutchgrid.nl/info/dca-rekey-pack.sh && sh dca-rekey-pack.sh ../$((  `date +%Y` - 1 ))/usercert.pem && mail ca@dutchgrid.nl < ./newrekeypack.txt && mv newkey.pem userkey.pem

Complete the administrative procedure as described on the CA web page and continue with step 2).

Installing the key/certificate pair on the NFS Server

After some time, the CA will send a mail with the location of the generated host certificate. As root on vlaai, download the certificate from the given location into the directory where the output files of the request are stored. Rename the downloaded certificate to usercert.pem.

As a oneliner:

# cd /export/perm/share/grid-security/<machine name>/$((  `date +%Y` ))
# B="<public key number>" && wget http://www.dutchgrid.nl/ca/medium/details/newcerts/$B.pem && mv $B.pem usercert.pem && cd .. && rm -rdf user* ; ln -s $((  `date +%Y` ))/usercert.pem && ln -s $((  `date +%Y` ))/userkey.pem && touch release.state && chown apache:apache release.state

# cd /export/perm/share/grid-security/tbn01/2008
# wget link-to-host-certificate
# mv WXYZ.pem usercert.pem

In the parent directory, /export/perm/share/grid-security/hostname, two symbolic links need to be created that point to the most recent host key and certificate (if they already exist, remove the links first).

# cd ..
# ls -l user*.pem
lrwxrwxrwx 1 root root 17 Aug  4 18:25 usercert.pem -> 2007/usercert.pem
lrwxrwxrwx 1 root root 16 Aug  4 18:25 userkey.pem -> 2007/userkey.pem
# rm -f usercert.pem userkey.pem
# ln -s 2008/usercert.pem
# ln -s 2008/userkey.pem

In order to allow the host to retrieve its host key (and certificate), create a file release.state with a recent time stamp (still to be done as root in the directory /export/perm/share/grid-security/hostname):

# touch release.state
# chown apache:apache release.state

Continue with the next step to install the new/renewed key/certificate pair on the host. This needs to be within the expiration time of 15 minutes.

Installing the key/certificate pair on the host

The key/certificate pair can can be fetched using quattor tools. If the key/certificate pair has been made accessible as described in the previous step, the following command will install the pair:

ncm-ncd --configure gethostkey

This command requires root privileges.

Certain node types that are managed by Yaim need additional copies of the key/certificate pair. For those hosts, the quattor command should read:

/bin/rm /etc/siteinfo/lcg-quattor-site-info.def ; ncm-ncd --configure yaim

(the first part is required to force execution of the Yaim component).

Both of the above ncm-ncd commands may be replaced by a global reconfiguration:

ncm-ncd --configure --all

(note that removing Yaim's configuration file may be required).

The quattor component gethostkey is provided by the package trustedget. This package also provides the script /usr/local/sbin/trustedget, which does the real work. It may be installed on hosts that are not managed by quattor, enabling to download the key/certificate pair for such hosts too.

The following steps are executed by the quattor component, but may be performed manually on all nodes.

Verify that the key/certificate pair can be downloaded:

# /usr/local/sbin/trustedget https://vlaai.nikhef.nl/retrieve.php?willdo
200 Will release tbn01

This means that it is possible to download the host key. If the resulting message is "403 Release for tbn01 expired", then the file release.state at the server either does not exist, is not owned by user and group apache, or has a timestamp that is too far in the past. In that case, downloading the key is not possible. The certificate may always be downloaded.

To download the certificate:

# /usr/local/sbin/trustedget https://vlaai.nikhef.nl/retrieve.php?cert > /etc/grid-security/hostcert.pem

Ensure the permissions on the certificate are 0644 (world-readable).

To download the key:

# /usr/local/sbin/trustedget https://vlaai.nikhef.nl/retrieve.php?key > /etc/grid-security/hostkey.pem

Ensure the permissions on the key are 0400 (only readable for root). After downloading the key, the timestamp of the key file on the web server is reset to "1 Jan 1970" and downloading of the key is no longer possible.

Scripts for automated process

There are available several scripts to automate the process:

Install the certificate at one server

In this case the script has to be run with two parameters ($1 = hostname, $2 = This is the certificate id and it's included in the email at the end of the URL - without the '.pem'):

#!/bin/bash
server=$1
B=$2
j=`echo $server | cut -f 1 -d'.'`
ssh root@vlaai "cd /export/perm/share/grid-security/$j/$((  `date +%Y` )) && wget http://www.dutchgrid.nl/ca/medium/details/newcerts/$B.pem && mv $B.pem usercert.pem && cd .. && rm -rf  user* ; ln -s $((  `date +%Y` ))/usercert.pem && ln -s $((  `date +%Y` ))/userkey.pem && touch release.state && chown apache:apache release.state"
ssh root@$server "/bin/rm /etc/siteinfo/lcg-quattor-site-info.def ; ncm-ncd --configure --all"

Install the certificates at several servers

This script reads from a text file with the following structure:

<HOST1> <CERT_ID1>
<HOST2> <CERT_ID2>
...
<HOST3> <CERT_ID3>

The script to be run is:

#!/bin/bash
FILE="<FILE_NAME>"
server_list=`awk '{print $1}' $FILE`
cert_id_list=`awk '{print $2}' $FILE`
for i in $server_list
do
   ssh $i 'hostname'
   j=`echo $i | cut -f 1 -d'.'`
   B=`awk "/$i/ {print $2}" $FILE | cut -f 2 -d' '`
   ssh root@vlaai "cd /export/perm/share/grid-security/$j/$((  `date +%Y` )) && wget http://www.dutchgrid.nl/ca/medium/details/newcerts/$B.pem && mv $B.pem usercert.pem && cd .. && rm -rf user* ; ln -s $((  `date +%Y` ))/usercert.pem && ln -s $((  `date +%Y` ))/userkey.pem && touch release.state && chown apache:apache release.state"
   ssh root@$i "/bin/rm /etc/siteinfo/lcg-quattor-site-info.def ; ncm-ncd --configure --all"
done