How to handle OpenSSL and not get hurt using the CLI

From PDP/Grid Wiki
Jump to navigationJump to search

Standard OpenSSL stuff

I'm not going to copy/paste everything in here which is supplied in a nice and orderly fashion elsewhere.

s_client foo

example: openssl s_client -connect 127.0.0.1:13050 -msg -nbio -ssl3 -CApath ~/dvl/ca/ -cert ~/dvl/ca/newcert.pem -key ~/dvl/ca/newkey.pem

-CAfile vs. -CApath

Using the -CAfile <specific CA file> will send this certificate over the wire to the server-side. This will typically fail the verification of the certificate chain at the server-side, because it is not allowed to transfer the self-signed certificates. The trust-anchors should be installed at the service, not transfered by the client (for obvious reasons). The -CAfile <file> will also be used for the verification of the server-side certificate, but it's safer to use the -CApath <path to one or more CA certificates> option.

How to calculate the hash value used by CA file names

OpenSSL CLI and the OpenSSL library functions will search in a default path and/or a given path to the needed (installed) CA files when it needs to verify a certificate chain. By convention a client (and server) will never provide the (final) CA certificate to the connected peer. The trust in the peer certificate (chain) has to be completed by adding the CA certificate(s) to the chain for it to verify completely.

This means that the OpenSSL CLI tool and/or library functions need to search one or more (stated) paths for the use CA files by the peers. OpenSSL will search in the -CApath directory by the hash of the used CA.

Run the following command:

openssl x509 -hash -noout -in cacert.pem
0e52ca4f

Copy or rename the cacert.pem file to 0e52ca4f.0. The .0 indicates that it is the root CA. A .1 extension indicates a subordinate CA. The .1 doesn't always work because of differences in the OpenSSL implementations between its versions. Also Java libraries will handle the .1 each in a different way.

Using proxy certificates and s_client

Setting up a mutually authenticated SSL connection means that you'll receive the server-side certificate for your (automated) verification and that your (client-side) credentials are to be passed over to the server-side for authentication (in that order).

To perform a proper verification of the certificate chain, the server-side must be capable of constructing the complete certificate chain and through a challenge-response mechanism the private key used must fit the final certificate. When using a complex certificate chain where only the CA certificates are distributed to the service nodes a client must send its EEC and proxy certificates to the server-side.

To send the EEC and the proxy to the server-side with OpenSSL s_client you can use -CAfile to fill the gap in the certificate chain:

openssl s_client -connect 127.0.0.1:13050 \
     -debug -state -nbio \
     -CApath /etc/grid-security/certificates/ \
     -CAfile $HOME/.globus/usercert.pem \
     -cert /tmp/x509up_u501 \
     -key /tmp/x509up_u501

The -cert is used for the proxy certificate, -key for the private key (which is in the same file as the proxy certificate), the -CAfile is used for your EEC (your personal certificate) and the -CApath refers to the directory filled with root CA certificates. The -debug, -state and -nbio are not relevant here.

Downloading the host, service or user certificate from an OpenSSL session

Copy and execute the following little script. Rename the SERVER and PORT variable and you will end up with the certificate of the service in PEM format on file.

#!/bin/sh
OPENSSL=openssl
SED=sed
CAT=cat
TIMEOUT=1
SERVER=kuiken.nikhef.nl
PORT=8443
CERTDIR=/etc/grid-security/certificates/
PROXY_FILE=/tmp/x509up_u501
TMP_FILE=/tmp/cert_fetch.tmp
OUTPUTFILE=certificate_${SERVER}_${PORT}.pem
$OPENSSL s_client -connect ${SERVER}:${PORT} -CApath ${CERTDIR}  -key ${PROXY_FILE} -CAfile ${PROXY_FILE} -cert ${PROXY_FILE} -showcerts > ${TMP_FILE} &
PID=$!
sleep $TIMEOUT
kill -TERM $PID 2>/dev/null
$CAT $TMP_FILE | $SED -ne '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' > $OUTPUTFILE
echo The host certificate is written in: $OUTPUTFILE
exit 0

File Creation

Creating a CA

First create a random file, as random input seed. WARNING! I'm using /dev/urandom which is a speedy short cut. Do NOT use in production CAs!

SEED="private/.rand"
dd bs=4096 count=2048 if=/dev/urandom of=${SEED}

Create a new CA with sha1 for 5 years:

PRIVATE_KEY="cakey.pem"
PUBLIC_CA_CERT="cacert.pem"
DAYS=`echo "(365 + 1) * 5" | bc`
openssl req -new -x509 -sha1 -keyout ${PRIVATE_KEY} -rand ${SEED} -out ${PUBLIC_CA_CERT} -days ${DAYS}

Creating a CRL file

To create a CRL file you'll need the OpenSSL config file used by your CA to generate the CRL file:

CRL_OUT="output.r0"
CONFIG_FILE="config.file"
openssl ca -gencrl -config ${CONFIG_FILE} -out ${CRL_OUT}

File Conversion

A couple of file handling and format conversion examples.

Making a p12 file from pem

You can do this in many way and for different reasons. At this moment I want to have all the CA certificates in a p12 files to be used on my iPhone. I've downloaded the current CA distribution (in a classic, mics and slcs tarball) from the EUGridPMA website and used the following command to create a p12 file only for the CA certificates:

LINE="openssl pkcs12" ; for i in `ls *.0`; do LINE="$LINE -in $i"; done ; COMMAND="$LINE -cacerts -nokeys -export -out IGTF-classic-mics-slcs.p12"; $COMMAND

Making a PEM file from a p12 file

input: cert.p12 – certificate in .p12 format output: cert.pem – certificate in .pem format (use it as both X509_USER_KEY and X509_USER_CERT)

Only the certificate(s):

openssl pkcs12 -in cert.p12 -clcerts -out cert.pem

Certificate plus private key:

openssl pkcs12 -in cert.p12 -out cert.pem

Note: This option will also ask for a new Passphrase for the to be written private key.

Converting CRL from PEM to DER

Take the CRL file in PEM format as input, and the output will be in DER:

IN_CRL="input.r0"
OUT_CRL="output_in_der.r0"
openssl crl -in ${IN_CRL} -outform DER -out ${OUT_CRL}

Converting OpenSSH public keys to PEM

Goto: http://blog.oddbit.com/2011/05/converting-openssh-public-keys.html

Verification examples

How to verify your certificate, private key and other details to be valid, matching or properly formatted.

Show effective certificate content in a file

Show the public certificate

CERT=certificate.pem
openssl x509 -noout -in ${CERT} -text

Show certificate file by its split content

Goto https://github.com/okoeroo/certificate-handling and use the show-split-certs.py script. Example:

./show-split-certs.py /etc/grid-security/hostcert.pem
./show-split-certs.py <other file>
[okoeroo@localhost ~]$ ./show-split-certs.py mkproxy-x509-voms 
Checking file : mkproxy-x509-voms
subject= /DC=org/DC=terena/DC=tcs/C=NL/O=Nikhef/CN=Oscar Koeroo okoeroo@nikhef.nl/CN=proxy
issuer= /DC=org/DC=terena/DC=tcs/C=NL/O=Nikhef/CN=Oscar Koeroo okoeroo@nikhef.nl
notBefore=Jan 26 09:19:43 2012 GMT
notAfter=Jan 27 09:24:43 2012 GMT

subject= /DC=org/DC=terena/DC=tcs/C=NL/O=Nikhef/CN=Oscar Koeroo okoeroo@nikhef.nl
issuer= /C=NL/O=TERENA/CN=TERENA eScience Personal CA
notBefore=Jul  4 00:00:00 2011 GMT
notAfter=Aug  2 23:59:59 2012 GMT

Verifying your certificate against the installed CA certificates and CRLs

This is a basic check to verify your certificate:

# The usercert.pem is your user certificate file
# The /etc/grid-security/certificates/ is the directory that holds the CA certificates and CRL files. Adjust this to another directory if needed.
openssl verify -CApath /etc/grid-security/certificates/ usercert.pem

This can also be used for host certificates.

Matching the private key file with the public certificate file

The modulus and the public exponent portions in the key and the Certificate must match. But since the public exponent is usually 65537 and it's bothering comparing long modulus you can use the following approach:

openssl x509 -noout -modulus -in usercert.pem | openssl md5
openssl rsa -noout -modulus -in userkey.pem   | openssl md5

And then compare these really shorter numbers. With overwhelming probability they will differ if the keys are different. As a "one-liner":

openssl x509 -noout -modulus -in usercert.pem | openssl md5 ;\
openssl rsa -noout -modulus -in userkey.pem   | openssl md5

The usercert.pem is your user certificate file. This can also be used for host certificates.

grid-proxy-verify

A simple but powerful tool to diagnose a certificate chain with old style/classic or RFC3820 compliant proxy certificates.

http://www.nikhef.nl/~janjust/proxy-verify/

Some grid computing specific tips

Like the subsection tells you.

Proxy certificate generation with pure OpenSSL and Bash

Pure bash implementation of grid-proxy-init using only OpenSSL CLI tools and bash. It can construct old style/classic, gt3 and RFC3820 style proxies:

http://www.nikhef.nl/~janjust/proxy-verify/

Create an .lsc file for VOMS

Take the host certificate of the VOMS server and run it through the following code snippet:

HOSTCERT=/etc/grid-security/hostcert.pem
VONAME="MyVO"
VOMSDIR=/etc/grid-security/vomsdir
openssl x509 -noout -subject -issuer -in ${HOSTCERT} | cut -d " " -f 2- -s > ${VOMSDIR}/${VONAME}/${VONAME}.lsc

It will set the PEM file as input, set the VO name for which it works and sets the default VOMSDIR and write the openssl wrapped output into a file with the .lsc extension in the right location.

Walk your CA certificates

I've pushed a piece of code scribbles to GitHub that might be useful to view meta-data on the CAs installed:https://github.com/okoeroo/certificate-handling

The read_cert_chains.py will provide a nice humanly understandable ASCII-artish view on the installed CAs. By default it will look for .pem files in the /etc/grid-security/certificates/

"/C=GB/ST=Greater Manchester/L=Salford/O=Comodo CA Limited/CN=AAA Certificate Services"
    |   |-> File             : AAACertificateServices.pem
    |   |-> Depth            : 0
    |   |-> Not Before       : 20040101000000Z
    |   |-> Not After        : 20281231235959Z
    |   |-> Serial number    : 1
    |   |-> Is valid         : no
    |   #
    \___| -> "/C=US/ST=UT/L=Salt Lake City/O=The USERTRUST Network/OU=http://www.usertrust.com/CN=UTN-USERFirst-Client Authentication and Email"
        |   |-> File             : UTNAAAClient.pem
        |   |-> Depth            : 1
        |   |-> Not Before       : 20040101000000Z
        |   |-> Not After        : 20281231235959Z
        |   |-> Serial number    : 165470017604460690276855178217067231472
        |   |-> Is valid         : no
        |   #
        \___| -> "/C=NL/O=TERENA/CN=TERENA eScience Personal CA"
            |   |-> File             : TERENAeSciencePersonalCA.pem
            |   |-> Depth            : 2
            |   |-> Not Before       : 20090518000000Z
            |   |-> Not After        : 20281231235959Z
            |   |-> Serial number    : 124943880517709597336269004402877481672
            |   |-> Is valid         : no
            |   #
"/C=NL/O=NIKHEF/CN=NIKHEF medium-security certification auth"
    |   |-> File             : NIKHEF.pem
    |   |-> Depth            : 0
    |   |-> Not Before       : 20010921000000Z
    |   |-> Not After        : 20210209000000Z
    |   |-> Serial number    : 0
    |   |-> Is valid         : no
    |   #

DrSSL

DrSSL - "Diagnose your SSL" - is intended to setup an SSL connection, check all kinds of SSL and certificate related things, report about that and diagnose problems: https://github.com/okoeroo/drssl